I'm going to add a datapackage path to the bulk data service, allowing for the POSTing and processing of a datapackage.json file:
{
"name": "a-unique-human-readable-and-url-usable-identifier",
"datapackage_version": "1.0-beta",
"title": "A nice title",
"description": "...",
"version": "2.0",
"keywords": ["name", "My new keyword"],
"licenses": [{
"url": "http://opendatacommons.org/licenses/pddl/",
"name": "Open Data Commons Public Domain",
"version": "1.0",
"id": "odc-pddl"
}],
"sources": [{
"name": "World Bank and OECD",
"web": "http://data.worldbank.org/indicator/NY.GDP.MKTP.CD"
}],
"contributors":[ {
"name": "Joe Bloggs",
"email": "joe@bloggs.com",
"web": "http://www.bloggs.com"
}],
"maintainers": [ {
"name": "Joe Bloggs",
"email": "joe@bloggs.com",
"web": "http://www.bloggs.com"
}],
"publishers": [ {
"name": "Joe Bloggs",
"email": "joe@bloggs.com",
"web": "http://www.bloggs.com"
}],
"resources": [
{
"path": "relative-path-to-file",
"url": "online url"
}
]
}
I will handle the same as bulk JSON POSTS, process each record, and submit as JSON Job which can then be processed on schedule or events.