This post was updated with advanced tips (at the bottom) 2/12/2016 For scientific projects we often using knitr and R markdown in RStudio to share results and code either in HTML or PDF format. These are great tools for reproducible research. We find, however, that keeping all of our R code in an R markdown […]
Continue reading →For most of our projects that involve large datasets we use either PostgreSQL or MySQL to conduct analysis and feed data-driven web sites. In general, we’re happy with these tools, particularly PostgreSQL and its spatial functionality, and find that they meet our data needs. PostgreSQL has also been fairly innovative, recently adding functionality focused on […]
Continue reading →We have moved many of our geographic analyses to QGIS from ESRI’s ArcGIS but we have yet to find an adequate replacement for ESRI’s Network Analyst extension. Network Analyst, itself, can be clunky and tricky to use (not to mention expensive!), but it has enabled us to compute network buffers and minimum network distance fairly […]
Continue reading →PostgreSQL allows you to easily dump tables for backup purposes into plain text files using code such as: pg_dump mytable > mytable_backup.sql But this process is not very flexible and can create very large output files. As an alternative, PostgreSQL has a nice little feature that allows users to export to what is called a […]
Continue reading →gdalAlthough we still use ESRI products such as ArcGIS for geographic analysis, more and more we find ourselves turning to open source software like PostGIS and QGIS. In many cases, though, even PostGIS and QGIS are more software than necessary for a given task and we prefer to skip the ‘middleman’ in favor of the […]
Continue reading →For a recent project I was conducting some fairly dry quality assurance/quality control. In particular, I was comparing whether the population-weighted centroids of ZIP codes (ZCTA, to be specific) we computed in PostGIS matched those computed using QGIS. Given that I was creating temporary, internal maps, to make things more fun, I used the OpenLayers […]
Continue reading →AngularJS is a powerful tool for creating single page web applications, particularly data driven applications. One of the most powerful features of Angular is its two-way data binding, the automatic synchronization of data between the view and the application itself. In this example, we demonstrate the two-way data binding using a table and LeafletJS map […]
Continue reading →The Python library BeautifulSoup is an incredible tool for pulling out information from a webpage. You can use it not only to extract tables and lists but you can also use to to pull out very specific elements like a paragraph with a green font color. To briefly illustrate this functionality and in honor of […]
Continue reading →For a recent project we were provided with a GIS shapefile of developments (primarily houses) in a suburban neighborhood in Ithaca, NY. We created static maps to show change through time with color coded dots for different time windows but we thought that the dataset was a perfect opportunity to show off CartoDB’s time series […]
Continue reading →ggplot2When you’re plotting a lot of data overplotting can sometimes obscure important patterns. In situations like this it can be useful at the exploratory data analysis phase to ‘jitter’ the data so that underlying data can be viewed making it easier to see patterns. In addition, jittering can be a way of ‘anonymizing’ spatial data […]
Continue reading →