Fork me on GitHub

Baseline for Federal Government Open Data and API Portals

08/20/2013

I have a whole list of projects around open data and APIs in the federal government. Additionally I have numerous other open data and API projects I’d like to tackle across other federal agencies. As I do with other areas of my work, I needed a standardized way to stabilize the datasets and APis I will need for my projects, in the same way any open data and API provider should do for their consumers.

To help support my work, and hopefully the work of others I wanted to create a baseline portal that I could use at any agency for showing what is possible when hanging open datasets and APIs, in a full featured portal. The success of any open data and / or API portal starts with the technical building blocks, like data and APIs, but have a set of business and political building blocks that are essential to their adoption and growth.

I’ve spent the last three years studying the business and politics of APIs. During these three years I’ve looked at almost 10,000 API developer portals, and established a base set of what I consider the building blocks of successful API portals, with a handful in which I consider essential to success. I’ve always wanted a simple API portal template that would reflect this research, and my new Dev Hub is the first step towards achieving this.

Dev Hub is an early stage prototype, I’ve built to satisfy this need of mine, to deploy gov data. I will polishing this portal and replicating as a single template that can be used for any API and / or open data portal. This portal exists purely as a Github repository and runs on Github Pages, using a Jekyll for managing its pages and blog. Everything else is HTML, CSS, Javascript and JSON, allowing it to be able to run on any server, including other cloud services like Dropbox or Amazon S3.

To start with I’ve included 20 of what I consider essential API building blocks:

Beyond this project running completely using open formats and standards, deployable in easy to stand-up cloud environments like Github, Dropbox and Amazon S3, everything is machine readable by default. The portal starts with a sitemap.json, which links to any other building blocks of the open data and API portal, including a data.json and api.json which provide programmatic access to all resources included in the site:

Next I wanted all essential building blocks of the portal to also be machine readable by default:

This project is the product of 3 years of research, and 3 days worth of work. So it is rough. I was able to capture my initial vision in 3 days, but still a lot of work to do. I need to let it simmer a little bit, and round off some of the rough edges and apply some polish. Then the next step is to generate a generic template from it, that will become the baseline for any other portal I derive from this work.

If there is anything you’d like to see in the future, go ahead and use any of the support channels available on the site. Once I get the base template setup, we can use that as a top level project to guide the future of this baseline for open data and API portals.