For a recent project I was conducting some fairly dry quality assurance/quality control. In particular, I was comparing whether the population-weighted centroids of ZIP codes (ZCTA, to be specific) we computed in PostGIS matched those computed using QGIS. Given that I was creating temporary, internal maps, to make things more fun, I used the OpenLayers […]
Continue reading →AngularJS is a powerful tool for creating single page web applications, particularly data driven applications. One of the most powerful features of Angular is its two-way data binding, the automatic synchronization of data between the view and the application itself. In this example, we demonstrate the two-way data binding using a table and LeafletJS map […]
Continue reading →The Python library BeautifulSoup is an incredible tool for pulling out information from a webpage. You can use it not only to extract tables and lists but you can also use to to pull out very specific elements like a paragraph with a green font color. To briefly illustrate this functionality and in honor of […]
Continue reading →For a recent project we were provided with a GIS shapefile of developments (primarily houses) in a suburban neighborhood in Ithaca, NY. We created static maps to show change through time with color coded dots for different time windows but we thought that the dataset was a perfect opportunity to show off CartoDB’s time series […]
Continue reading →ggplot2When you’re plotting a lot of data overplotting can sometimes obscure important patterns. In situations like this it can be useful at the exploratory data analysis phase to ‘jitter’ the data so that underlying data can be viewed making it easier to see patterns. In addition, jittering can be a way of ‘anonymizing’ spatial data […]
Continue reading →CartoDB is a great service for generating online maps and is particularly useful if you’re working with big data that might not render quickly using APIs such as Leaflet and Google Maps. For details on CartoDB we did a write-up of the nice functionality here. In working with CartoDB layers, though, one of the challenges […]
Continue reading →dplyr's function inner_join() is much faster than merge()Using the merge() function in R on big tables can be time consuming. Luckily the join functions in the new package dplyr are much faster. The package offers four different joins: inner_join (similar to merge with all.x=F and all.y=F) left_join (similar to merge with all.x=T and all.y=F) semi_join (not really an equivalent in merge() unless […]
Continue reading →Although many data science-related projects can be completed with a single software tool we often find that decisions about what tool to use for a project involve weighing a combination of what tool would be “best” for the job, what tools we're most familiar with and whether we already have scripts we can use. As […]
Continue reading →A lot of government agencies and other institutions make map layers available through ArcGIS Server map services. The United States Geological Survey’s National Map Program, for example, makes services available for adding basemap, topography, weather and other layers. These map layers can often be a useful addition to your own online maps and a relatively […]
Continue reading →For a current project we needed to render, on the web, a geographic file with approximately six thousand features. The shapefile itself is approximately 11 megabytes, far too big to be handled speedily in a web application. We were planning to render the features using GeoServer and map tiles but decided to investigate slimming down […]
Continue reading →