These are chat archives for frictionlessdata/chat

23rd
Mar 2017
Rufus Pollock
@rufuspollock
Mar 23 2017 08:29

frictionlessdata/specs#391

@pwalsh i’m concerned that the new setup where data can’t be actual JSON is going to be a bit of a pain …

Rather than:

"data": [
        {"x": 1,  "y": 28}, {"x": 2,  "y": 55},
        {"x": 3,  "y": 43}, {"x": 4,  "y": 91},
      ]

I have to do:

data": // some json pointer to _values (and i don’t even know how json pointer works!
// arbitrary naming!
"_values": [
        {"x": 1,  "y": 28}, {"x": 2,  "y": 55},
        {"x": 3,  "y": 43}, {"x": 4,  "y": 91},
      ]
Paul Walsh
@pwalsh
Mar 23 2017 08:33
@rufuspollock there are two reasons:
  1. Consistency for implementations (as @Stiivi pointed out, we overload many things, and in response I made many properties more explicit/consistent. Following, having "inline data or a reference to data" here is quite complicated to implement).
  2. There are really close to zero data packages in the wild with inline data. The type of people who are inlining data are, I think, the type of people who can reference the data on another property.
Rufus Pollock
@rufuspollock
Mar 23 2017 08:34
@pwalsh it is a commonly used approach and it is exactly the approach being used in the jupyter/nteract stuff (and I use it). When you actually start manipulating dps in memory is very common ...
My gut and some experience indicates this will be significant so we do need to think about it - having to go read the json pointer spec every time i want local data is quite a deal IMO ...
Paul Walsh
@pwalsh
Mar 23 2017 08:37
@rufuspollock ok. I did not make this change in a vacuum :). It was discussed. However, manipulating dps in memory is not a concern of the spec - once you are manipulating dps in memory, then you also have complete control over where you access the materialised views on the data that result from the manipulations, no?
Rufus Pollock
@rufuspollock
Mar 23 2017 08:49
@pwalsh let’s chat about this on a call soon ...
Kuyawa Kata
@kuyawa
Mar 23 2017 14:03
Hi guys, I just applied for the Swift conversion, wish me luck
Paul Walsh
@pwalsh
Mar 23 2017 14:03
@kuyawa good luck!
Kuyawa Kata
@kuyawa
Mar 23 2017 14:03
:D
Rufus Pollock
@rufuspollock
Mar 23 2017 19:52
@gnestor welcome - great to speak yesterday!
Jonathan Mitchell
@Jonatha24699848_twitter
Mar 23 2017 20:46
couldn't help but laugh... I've been spending all day opening excel files, formatting columns, removing headers/footers, saving as csv, and uploading them to database tables - then I get an email from a customer saying they're having trouble working with the CSV file we sent them... these standards and libraries can't come quickly enough :)
Rufus Pollock
@rufuspollock
Mar 23 2017 20:46
@Jonatha24699848_twitter ;-) and i hear you!