@roll @vitorbaptista do we have a common repo(s) where we have sample data files e.g. sample csv, sample data packages etc?
If not what do you think of maintaining one? It would be super useful for the community to have a common set of test data.
Thinking about requirement for example data packages:
- As a Tutorial writer I want a set of data files and data packages I can use in my tutorials so that i can embed them and point users to them to play with them themselves
- As a Developer writing a library I want to have a set of standard test data files and data packages as a reference for my implementation tests
- As a new Publisher of data packages i want to see examples that i can copy and use so that I can move quickly and understand what is involved
- As a Consumer of data packages I want to see some examples for use
My sense is that the "exemplar" and "test" use cases are somewhat different. 1+3+4 are exemplar and want "nice' data packages". 2 (+1) are more test and are about testing the real range of sitautions and being super simple for testing.
My sense is that the key here to focus on is the test (lib developer) case to start with.
They probably want versioning and ability to git submodule so they can pin the data they are developing against (e.g. if data package spec gets upgraded they can still keep old spec versions if they need them).
wdyt?
pm repo has been deprecated / deleted https://github.com/frictionlessdata/pm 404s ...
About issues current system is we have two main issue trackers aside concrete libs:
- https://github.com/frictionlessdata/specs/issues - everything related to the specs (well known=)
- https://github.com/frictionlessdata/implementations/issues - everything related to the implementations
The second on I think could be used as just for everything else except the specs. Also the FD workboard lives on this repo:
goodtables-py/CLI as a backend for goodtables-js - frictionlessdata/goodtables-js#19 It's a pretty trivial feature to implement which allow to validate data in JavaScript locally using Python CLI to generate data quality reports. Having pure goodtables-js is cool but we should take into account that it could mean really a lot of work because in JavaScript we miss both tabulator and goodtables. Which I would say the most complex software in the Python stack. Also it could be always a problem that JavaScript implementation will be possibly always very limited in compare to the Python one. Python for example already support advanced checks and other advanced concepts.
@nathanxmeyer first, great to hear from you. To answer your questions:
My first question is whether the frictionless data schema specifications for datapackage and table are fairly stable.
Yes, the schema specs are definitely very stable - they are now v1.0 and have been refined for ~5y. Of course, they will continue to evolve gently but backwards compatibility will be maintained etc.
My second question is whether the goodtables python implementation is something that has future viability
Yes, absolutely. These have been heavily developed and will be continue to be!