Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    JD Bothma
    @jbothma
    let's try os-data-importers - shouldn't rely on goodtables
    JD Bothma
    @jbothma
    I've managed to process enough of the columbian datapackage in the simple branch using os-data-importers to be able to see the default bar chart but when I try to filter the API doesn't respond - any ideas why that might be?
    Oscar Montiel
    @tlacoyodefrijol
    @jbothma Elastic Search is down. This is giving some problems with all of the OS components. We’re working on bringing the services back up
    JD Bothma
    @jbothma
    awesome - thanks - wasn't sure what's happening - I'll let you focus on that
    JD Bothma
    @jbothma
    :clap: looks like it's alive again!
    JD Bothma
    @jbothma
    Hey guys, it looks like OS Packager is still hitting failing to do the initial dataset check which was previously a problem with goodtables
    Any chance you can look into that? Or is it faster to just add https://github.com/vulekamali/os-source-specs#vulekamali to os-data-importers?
    I'm pretty sure the pipelines in that branch don't work properly but if it's cool to add so long I can get the new pipeline I need working in there later today
    lolziac
    @lolziac
    It looks live...but OS Packager is not working :(
    when trying to upload the file I get this Error: Can't get a report from API. Reason: "Error: Reached maximum attemtps"
    JD Bothma
    @jbothma
    Same
    lolziac
    @lolziac
    hope somebody from the team can check this
    JD Bothma
    @jbothma
    in the network log I then see a request to /packager/proxy?url=undefined which has come up before openspending/openspending#1378
    lolziac
    @lolziac
    same here
    Oscar Montiel
    @tlacoyodefrijol
    Hi @jbothma @lolziac. I'll open a ticket to look into the Packager issue. For now it's a good idea to add the data to the pipeline
    lolziac
    @lolziac
    HI @tlacoyodefrijol thanks for treating this Packager issue, regarding adding data to the pipeline that you are mentioning it, where can I read more or try it out ?!?!
    lolziac
    @lolziac
    everything seems OK now
    JD Bothma
    @jbothma
    hey folks! Don't know if you know vega already, but I'm incredibly excited about it - just found out about it
    I think it's a declarative way of making D3 visualisations, with vega-lite being a very simple high level version of vega, which is much more verbose but also powerful
    here's an example pulling directly from OpenSpending https://codepen.io/jbothma/pen/zYOEbWv?editors=1000
    I haven't explored its ability to slice and dice data - I don't think it can generate data URLs dynamically, but at the very least I can generate URLs dynamically for it, and perhaps let it do basic filtering of disaggregated OpenSpending queries
    JD Bothma
    @jbothma
    Has anyone built a Tableau Web Data Connector for OpenSpending yet? https://help.tableau.com/current/pro/desktop/en-us/examples_web_data_connector.htm
    JD Bothma
    @jbothma
    I built a quick attempt - it ends up getting errors from the OpenSpending API - probably hitting it a bit hard. It should probably just use the model to provide a schema to Tableau, then load the data by streaming the CSV file https://github.com/vulekamali/tableau-web-data-connector
    On another topic, it'd be amazing to have the OpenSpending API URL exposed by OpenSpending viewer - we've been building some interactive visualisations using D3 and still look at the browser network log in OS Viewer to put API URLs together :)
    Oscar Montiel
    @tlacoyodefrijol
    The API being exposed is definitely something to look into
    @jbothma what will that connector feed into?
    JD Bothma
    @jbothma
    The connector could make it easy to use data in OpenSpending in Tableau for visualisation https://cdns.tblsft.com/sites/default/files/pages/wdc-socrata.gif
    for vulekamali.gov.za that's interesting because there could be an adaptor specific to vulekamali which lists just the vulekamali datasets, and if we update a dataset in openspending, peoples' visualisations could update automatically overnight
    obviousy we'd need to make it clear if that might happen. it makes sense for an in-year spending tracker. it might be surprising for a once-off analysis
    JD Bothma
    @jbothma
    @tlacoyodefrijol just checking you're aware it looks like we're getting gateway timeouts from the API
    Adam Kariv
    @akariv
    Yes, I'm currently handling it.
    JD Bothma
    @jbothma
    Thanks Adam
    Adam Kariv
    @akariv
    It's back up now
    JD Bothma
    @jbothma
    Woo! thanks Adam
    jjalcolea
    @jjalcolea
    Hi, OpenSpending gurus
    We're developing an Open Fiscal Data Package conversion script for public universities, but are facing a technical problem and need some help
    I've tried the forums, but they won't allow me to post anything (¿?), so here I am...
    image.png
    This is a date field as defined in the JSON file. As you can see, we specify the format
    And here is the error we get when applying the JSON to our CSV in the online OS Packager:
    image.png
    As you can see, it seems to be ignoring the "format" information, and tries to apply a different date mask than the one we specify in the JSON
    We are not able to deal with this. Can anybody lend us a hand?
    Thanks!
    jjalcolea
    @jjalcolea
    @akariv I'm trying to spot the exact point where the failure happens, but I'm not sure which is the correct repo, is it https://github.com/openspending/dataflows-openspending ?
    jjalcolea
    @jjalcolea
    OK: I've just discovered it's a known (and still unsolved) bug... :-(
    Adam Kariv
    @akariv
    Hey @jjalcolea - if you’d like, we can get on a call sometime next week so you can explain to me what you’re trying to do and I can guide you through the openspending codebase and how to best use it for your needs.
    jjalcolea
    @jjalcolea
    Thanks @akariv , I was just trying to isolate where the bug occurs in the code to better inform the bug report and facilitate the diagnosis; for the moment we don't have the resources to get involved in development; we're just users of the tools at openspending.org