These are chat archives for frictionlessdata/chat

17th
Oct 2017
Rufus Pollock
@rufuspollock
Oct 17 2017 08:41

Indeed, looking forward to get in gear. But first, we have a hackathon to run - and that means making more data packages! https://discuss.okfn.org/t/october-27-open-tourism-data-hackathon/5860

@loleg that's great and you can start pushing your data packages to the new https://datahub.io/ - if you are interested in being an alpha publisher user just sign up and then fill in the short questionnaire ...

jobarratt
@jobarratt
Oct 17 2017 09:08
looks great @loleg ! if there is something you are working on that is FD related always let us (especially @callmealien ) know here so we can give it a bit of an extra promotion push! And you may already be in touch with the OKI comms team but if not and you want to pitch a blog we'll always be happy to support you with it
Meiran Zhiyenbayev
@Mikanebu
Oct 17 2017 10:08

Data Package v1 Specifications. What has Changed and how to Upgrade

This post walks you through the major changes in the Data Package v1 specs compared to pre-v1. It covers changes in the full suite of Data Package specifications including Data Resources and Table Schema. It is particularly valuable if:

  • you were using Data Packages pre v1 and want to know how to upgrade your datasets
  • if you are implementing Data Package related tooling and want to know how to upgrade your tools or want to support or auto-upgrade pre-v1 Data Packages for backwards compatibility

You can find the entire blogpost here http://datahub.io/blog/upgrade-to-data-package-specs-v1

Stephen Gates
@Stephen-Gates
Oct 17 2017 10:46
What's the difference between sources in the Data Resource spec and sources in a Data Package? Sources in the resource don't explicitly inherit from the package like licences do. So why have both?
Meiran Zhiyenbayev
@Mikanebu
Oct 17 2017 11:17
@Stephen-Gates Thanks for asking this question. We will provide an answer soon, reading through the discussion in specs.
Rufus Pollock
@rufuspollock
Oct 17 2017 11:18
@Mikanebu you can have sources in both and there is no specific semantic on inheritance. sources in data package can be taken as sources for whole data package whilst for a given resource they are just for that resoruces ...
Meiran Zhiyenbayev
@Mikanebu
Oct 17 2017 11:20
@rufuspollock Thanks for clarifying this
Stephen Gates
@Stephen-Gates
Oct 17 2017 11:21
@rufuspollock if that's the case, why not have a statement similar to licenses, licenses: as for Data Package metadata. If not specified the resource inherits from the data package.
Rufus Pollock
@rufuspollock
Oct 17 2017 11:22

@Stephen-Gates because i don't think the specific resource inherits in a defined sense like licenses. sources are a less specific in that sense - whereas licenses obviously filter down the sources you specify may apply to some resources but not others etc.

I guess my question is more to you: what semantics do you want and why :-) ?

Stephen Gates
@Stephen-Gates
Oct 17 2017 11:49
@rufuspollock From a convenience perspective, I think think you should be able to define a licence or sources once at the package level and explicitly say resources inherit. If sources vary at the resource level, specify at that level and don't specify at the package level. Given licence compatibility issues, you could you specify different licences at the resource and not have a licence at the package level. The Specs support this apart from explicit inheritance of sources from the package. This could be fixed in the data resource spec by source: as for Data Package metadata. If not specified the resource inherits from the data package.
Stephen Gates
@Stephen-Gates
Oct 17 2017 14:49
Logged at frictionlessdata/specs#541
Byron Ruth
@bruth
Oct 17 2017 16:58
Good afternoon. I am reviewing the various specifications and had two questions. First have you come across a use case where a "query" is being represented as a data resource? The assumption being that the dataset is a function of the query at the time it is executed. And second, are there any support/examples for including and/or deriving provenance (PROV or otherwise) from data resources?