These are chat archives for frictionlessdata/chat

11th
Dec 2016
Yuri Astrakhan
@nyurik
Dec 11 2016 00:30
@rgrp, hi, so i got a bit confused about nullability. My goal is to simply mark any columns as "nullable", similar to SQL
the "default" values has a somewhat different semantic meaning - its like adding an automatic SQL collation on top of a null value - something that might be useful in general, but not required in the first pass
also, BIG ANNOUNCEMENT: tabular data has launched on all of Wikipedia. I am still in the process of making a demo, so we haven't announced it widely yet. https://commons.wikimedia.org/wiki/Data:Weather/New_York_City.tab is some data for an example
Rufus Pollock
@rufuspollock
Dec 11 2016 09:40

@nyurik i’m not sure i understand. You want to have columns that are null right? That is exactly what missingValue definition is for. As the spec states http://specs.frictionlessdata.io/json-table-schema/#missing-values:

By “missing” we simply mean null or “not present for whatever reason”. Many datasets arrive with missing data values, either because a value was not collected or it never existed.

The missing value stuff is not about collation. If you are reading frictionlessdata/specs#97 we are addressing the second of two options that the Stiivi identifies:

Other use is to tell the analytical/datamining software which values are considered as empty (NULLs). For example a string field with a content string "NULL" or "(empty)" will be converted to real NULL value.

@nyurik very exciting to see this go live on Wikipedia!
@nyurik you might also be interested in the work we are doing on a visualization / view spec attached to data packages that builds on vega / vega-lite - something for us to discuss in the near future
Rufus Pollock
@rufuspollock
Dec 11 2016 10:01
@pwalsh have you seen the updated proposal re metadata translations here: frictionlessdata/specs#42 - https://github.com/frictionlessdata/specs/issues/42#issuecomment-264258755 - interested in your thoughts. I think we are close to something that is “ready to merge"
Paul Walsh
@pwalsh
Dec 11 2016 13:51
@rgrp yes, i’m happy with it. Just some comments on languages property.