Metadata Repository Aggregation


This is a metadata repository aggregator. It contains information about datasets culled from a variety of sources and makes them searchable and inspectable in a uniform way. In addition to the human user interface, it produces machine readable output using RDF and queriable with the SPARQL language so that automated processes may easily make use of the same information.

The data presented here is harvested from the various metadata repositories only periodically so it may take some time for changes made at the source to show up here -- typically up to a day for CKAN instances.

Where the datasets are themselves RDF and a description of them and their neighbours is available, a navigable visualisation is available of their immediate neighbourhood. For curated groups of datasets, a similarly navigable visualisation is provided as well.

A subset of the CKAN API is implemented, namely the package list, search and retrieve operations, rooted at /api/ and can be used by the usual tools that do read operations on the CKAN API to do them across all aggregated instances.

The web user interface that you are looking at is made almost entirely in Javascript using jQuery and a modified version of sparql.js. The visualisations use Protovis. The data processing is done with the Go language using 4store and raptor. The source code is available here.

This aggregregator contains information from: