These are chat archives for frictionlessdata/chat

20th
Dec 2018
Zane Selvans
@zaneselvans
Dec 20 2018 13:27
I have some postgres database tables which I want to turn into tabular data resources in a tabular data package. I feel like this should be really easy, since the columns already have nice clean names and data types as well as comments for use as field descriptions. But somehow I'm not finding documentation on how one might do this. Am I missing something? Going directly from a pandas dataframe to a tabular data resource would also work well.
roll
@roll
Dec 20 2018 14:58
@zaneselvans Hi
# pip install tableschema-sql
package = Package(storage='sql', engnine=sqlalchemy.create_engine(DATABASE_URL))
Zane Selvans
@zaneselvans
Dec 20 2018 15:20
I don't want the entire database, I want to specify a subset of the tables. So I imagine I would use Resource() instead. But I don't see any documentation for how that works.
Rufus Pollock
@rufuspollock
Dec 20 2018 15:26
@roll any thoughts on @zaneselvans question - seems a natural approach ...
Zane Selvans
@zaneselvans
Dec 20 2018 15:35
Will it grab the comments associated with the database columns and use them as descriptions for the fields in the resource? And assign the appropriate field types? And allowable values for Enum columns? Even just generating the descriptor would be great. Pulling the table (or a subset thereof) and exporting it as a CSV is easy.
Zane Selvans
@zaneselvans
Dec 20 2018 15:57
it's ironic that getting our data into datapackages feels so friction filled. :-/
roll
@roll
Dec 20 2018 16:49
@zaneselvans I need to get back to my computer to check and write an example but you can provide a datapackage descriptor to call above to cherry-pick tables that should be mapped to resources
resources: [{name: table1}, ...]
Rufus Pollock
@rufuspollock
Dec 20 2018 17:50

it's ironic that getting our data into datapackages feels so friction filled. :-/

Building great tooling takes investment, a community and patience :smile:

We really appreciate the feedback and we’re keeping iterating on this and i agree it could be lots better. These are often things we know - it just takes time/money to make it better.

If you’d be up for contributing to the tool stack we’d love to have it and we’d be have to have PRs, tutorials, etc :smile: