These are chat archives for frictionlessdata/chat

21st
Dec 2018
roll
@roll
Dec 21 2018 08:29

@zaneselvans Actually it's as simple as

descriptor = {'resources': [{'path': 'table_name_1'}, {'path': 'table_name_2'}]}
package = Package(descriptor, storage='sql', engine=create_engine(DATABASE_URL))
package.infer()
package.save('descriptor.json') # types, constraints, primary/foreign keys from DB

Or you can create resources:

resource = Resource({'path': 'table_name_1'}, storage='sql', engine=create_engine(DATABASE_URL))
# ...

The software is quite there but we lack high-level documentation on these topics.

Rufus Pollock
@rufuspollock
Dec 21 2018 10:33
:thumbsup:
Zane Selvans
@zaneselvans
Dec 21 2018 11:17
Hmm, both of these give me StorageError: Storage.connect is not available on concrete implemetations
Zane Selvans
@zaneselvans
Dec 21 2018 11:35
Reloaded my kernel and tried again, and now instead when I try to infer, I get StorageError: Type "ferc1_pudl_fuel_codes" is not supported That's one of the Enum fields, containing several strings.
Zane Selvans
@zaneselvans
Dec 21 2018 11:44
So I guess it doesn't like our database definition. How would I use a pandas dataframe to create a resource and infer its descriptor instead?
roll
@roll
Dec 21 2018 13:01
I think it's because of the custom types. For pandas interface is the same only difference should be Package(storage='pandas', dataframes=dataframes)
Zane Selvans
@zaneselvans
Dec 21 2018 13:05
Hmm. Postgres requires Enum instances to be named (and the name of the Enum is what shows up as the Type it doesn't like), but In the sqlalchemy MetaData object the type comes up as Enum, and it lists out the allowable values.
roll
@roll
Dec 21 2018 15:54
@zaneselvans Would you like to PR this feature? I think it only requires a change in the SQL mapper - https://github.com/frictionlessdata/tableschema-sql-py/blob/master/tableschema_sql/mapper.py#L234