These are chat archives for frictionlessdata/chat

27th
Feb 2017
roll
@roll
Feb 27 2017 09:29
@rufuspollock data type casting?
Adam Kariv
@akariv
Feb 27 2017 12:40
@roll quick consultation re jsontableschema-sql:
I want to be able to update an existing DB table with new data.
Currently the only way to do that is to delete() it and then write() the new data into it. This is problematic in the case the write fails and we're left without a valid dataset. Even if the write succeeds, we still have a down time of the data for the writing period (could be many minutes).
There are a few solutions for this:
  • Add a delete_all parameter to the write() method, which deletes all rows and writes them back in the same transaction. It's not very efficient but might work.
  • Add an update_by parameter, which allows you to define a set of columns as a 'key' - new rows are tested to see if an existing row with the same key already exists. If it does, data is updated. If it doesn't a new row is added.
Adam Kariv
@akariv
Feb 27 2017 12:45
I'm more inclined towards the 2nd approach. @roll , wdyt?
roll
@roll
Feb 27 2017 12:49
@akariv I would say second approach seems better option. For example dataset library uses for it keys parameter - https://dataset.readthedocs.io/en/latest/api.html#dataset.Table.update
Adam Kariv
@akariv
Feb 27 2017 12:52
Okay @roll - so I will start working on implementing that approach and will send a PR.
roll
@roll
Feb 27 2017 12:52
@akariv cool :+1: