How can I create automated data downloads?
Data can be incredibly useful. By analysing or interpreting the information contained in data, better decisions can be made.
Ordnance Survey is one of the world’s oldest data organisations. We have gathered information about Great Britain since 1791, organising, analysing and disseminating maps and other information. As information technologies have evolved from tables and charts and pen-and-paper calculations to databases, vector graphics and sophisticated machine learning algorithms, OS has been a leader in adopting and innovating new ways to capture, store, process and share valuable location information.
The challenge – managing data
As the scale of data grows, so does its potential usefulness – and the complexity of managing it. Understanding a small table of numbers is easy, less so for a complex database with dozens of tables and millions of rows. With code though, it has never been easier for programmers to retrieve and manipulate large volumes of data and extract and understand the valuable information they contain.
As every data scientist and web developer knows, well-structured datasets are difficult to find and even harder to create. Data wrangling is a major task and is often overlooked by those less familiar with the data science process. For algorithms to extract accurate and meaningful information from a dataset – especially using a powerful analysis libraries openly available like Python’s pandas – the data needs to be consistently structured and clean.
The answer – automated data downloads
With the OS Maps, OS Vector Tile and OS Features APIs, users authenticate their request with an API key created on their OS Data Hub account where they can dynamically fetch the data they need when they need it. This is powerfully useful in web maps, data science workflows and desktop GIS applications. These APIs enable developers and spatial analysts to create and customise interactive maps, and to fetch location data with rich attribution – information about a feature useful for visualisation and analysis purposes.
Unlocking value by improving access
The OS Data Hub APIs are the result of a major effort at OS to make our National Geographic Database easier to access than ever before. A broad range of users in Great Britain from civil servants to NGO workers, academic researchers, entrepreneurs and large companies, could enhance the quality and efficiency of their work with the insights they could derive from OS data. We’re working hard to make sure the path from data to decision-making is as short as possible. Automated data downloads are one of the ways we’re doing that.
We’ve written a tutorial to get you started with automated data downloads from the OS Downloads API using NodeJS – find it here.