Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 31 2019 19:27
    rufuspollock commented #241
  • Dec 14 2018 15:27
    StephenAbbott opened #246
  • Dec 03 2018 09:12
    rufuspollock commented #245
  • Nov 26 2018 14:51
    StephenAbbott opened #245
  • Nov 08 2018 08:31
    zelima commented #243
  • Nov 08 2018 08:05
    zelima closed #244
  • Nov 08 2018 08:05
    zelima commented #244
  • Nov 08 2018 07:57
    zaneselvans commented #244
  • Nov 07 2018 07:22
    zelima commented #244
  • Nov 07 2018 07:16
    akariv commented #244
  • Nov 07 2018 07:10
    akariv commented #234
  • Nov 06 2018 16:56
    parrottsquawk commented #234
  • Nov 01 2018 13:25
    zelima commented #244
  • Nov 01 2018 13:25
    zelima commented #244
  • Nov 01 2018 13:23
    zelima commented #244
  • Nov 01 2018 08:29
    anuveyatsu commented #244
  • Nov 01 2018 08:29
    anuveyatsu commented #244
  • Oct 24 2018 19:03
    zaneselvans opened #244
  • Oct 23 2018 09:40
    geraldb starred datahq/datahub-qa
  • Oct 19 2018 08:22

    Branko-Dj on master

    [travis][s]: Added update comma… (compare)

M. Ali Naqvi
@MAliNaqvi
Folks, I just spent a couple of hours uploading 43 datasets. It was a very frustrating to find that only 3 of those datasets made it to the datahub website, even though the data utility uploaded everything without an issue. Here are the results:
| dataset                                                            | url                                                                                                    | AVAILABLE |
|--------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+-----------|
| nasa-temperature-anomalies-by-latitude-bands-time-series-1880-2017 | https://datahub.io/JohnSnowLabs/nasa-temperature-anomalies-by-latitude-bands-time-series-1880-2017/v/2 | No        |
| chicago-annual-taxpayer-location-list                              | https://datahub.io/JohnSnowLabs/chicago-annual-taxpayer-location-list/v/2                              | No        |
| nasa-global-temperature-anomalies-time-series-1880-2018            | https://datahub.io/JohnSnowLabs/nasa-global-temperature-anomalies-time-series-1880-2018/v/2            | No        |
| nj-residents-leading-causes-of-death                               | https://datahub.io/JohnSnowLabs/nj-residents-leading-causes-of-death/v/2                               | No        |
| uk-properties-for-sale-by-ministry-of-defense                      | https://datahub.io/JohnSnowLabs/uk-properties-for-sale-by-ministry-of-defense/v/2                      | No        |
| tree-debris-requested-by-311-service                               | https://datahub.io/JohnSnowLabs/tree-debris-requested-by-311-service/v/2                               | No        |
| tree-trims-requested-by-311-service                                | https://datahub.io/JohnSnowLabs/tree-trims-requested-by-311-service/v/2                                | No        |
| garbage-carts-requested-by-311-service                             | https://datahub.io/JohnSnowLabs/garbage-carts-requested-by-311-service/v/2                             | No        |
| pot-holes-reported-by-311-service                                  | https://datahub.io/JohnSnowLabs/pot-holes-reported-by-311-service/v/2                                  | No        |
| eicu-collaborative-research-admissions-summary-statistics          | https://datahub.io/JohnSnowLabs/eicu-collaborative-research-admissions-summary-statistics/v/1          | Yes       |
| chicago-taxi-trips                                                 | https://datahub.io/JohnSnowLabs/chicago-taxi-trips/v/2                                                 | No        |
| chicago-beach-weather-stations-automated-sensors                   | https://datahub.io/JohnSnowLabs/chicago-beach-weather-stations-automated-sensors/v/2                   | No        |
| chicago-beach-water-quality-automated-sensors-report               | https://datahub.io/JohnSnowLabs/chicago-beach-water-quality-automated-sensors-report/v/2               | No        |
| all-countries-latitude-longitude                                   | https://datahub.io/JohnSnowLabs/all-countries-latitude-longitude/v/4                                   | No        |
| estimates-emissions-of-co2-at-country-and-global-level             | https://datahub.io/JohnSnowLabs/estimates-emissions-of-co2-at-country-and-global-level/v/2             | No        |
| energy-consumption-by-mode-of-transportation-and-type-of-energy    | https://datahub.io/JohnSnowLabs/energy-consumption-by-mode-of-transportation-and-type-of-energy/v/2    | No        |
| relocated-vehicles-in-chicago-last-90-days                         | https://datahub.io/JohnSnowLabs/relocated-vehicles-in-chicago-last-90-days/v/1                         | No        |
| nys-english-and-mathematics-exam                                   | https://datahub.io/JohnSnowLabs/nys-english-and-mathematics-exam/v/2                                   | No        |
| schools-for-life-safety-evaluations                                | https://datahub.io/JohnSnowLabs/schools-for-life-safety-evaluations/v/2                                | No        |
| food-affordability-for-households-led-by-females                   | https://datahub.io/JohnSnowLabs/food-affordability-for-households-led-by-females/v/2                   | No        |
| chicago-business-licenses                                          | https://datahub.io/JohnSnowLabs/chicago-business-licenses/v/1                                          | No        |
| city-population-annual-time-series                                 | https://datahub.io/JohnSnowLabs/city-population-annual-time-series/v/3                                 | No        |
| bloomington-animal-care-and-control-adopted-animals                | https://datahub.io/JohnSnowLabs/bloomington-animal-care-and-control-adopted-animals/v/2                | No        |
| legally-operating-businesses                                       | https://datahub.io/JohnSnowLabs/legally-operating-businesses/v/2                                       | No        |
| cta-ridership-bus-routes                                           | https://datahub.io/JohnSnowLabs/cta-ridership-bus-routes/v/1                                           | Yes       |
| most-popular-baby-names-by-gender-and-mother-ethnic-group          | https://datahub.io/JohnSnowLabs/most-popular-baby-names-by-gender-and-mother-ethnic-group/v/2          | No        |
| eicu-collaborative-research-available-tables-and-data              | https://datahub.io/JohnSnowLabs/eicu-collaborative-research-available-tables-and-data/v/1              | Yes       |
| nj-traffic-counts-data                                             | https://datahub.io/JohnSnowLabs/nj-traffic-counts-data/v/2                                             | No        |
| austin-adult-and-children-vaccinations                             | https://datahub.io/JohnSnowLabs/austin-adult-and-children-vaccinations/v/2                             | No        |
| euro-4-cars-emissions-traded-on-uk-market-2000-2012                | https://datahub.io/JohnSnowLabs/euro-4-cars-emissions-traded-on-uk-market-2000-2012/v/2                | No        |
| lobbyist-agency-report                                             | https://datahub.io/JohnSnowLabs/lobbyist-agency-report/v/2                                             | No        |
| windsor-transit-bus-stops                                          | https://datahub.io/JohnSnowLabs/windsor-transit-bus-stops/v/2                                          | No        |
| omha-receipts-for-fiscal-year-2011-2013                            | https://datahub.io/JohnSnowLabs/omha-receipts-for-fiscal-year-2011-2013/v/2                            | No        |
| impaired-driving-death-rate-by-age-and-race                        | https://datahub.io/JohnSnowLabs/impaired-driving-death-rate-by-age-and-race/v/2                        | No        |
| chicago-red-light-and-speed-camera-violations                      | https://datahub.io/JohnSnowLabs/chicago-red-light-and-speed-camera-violations/v/2                      | No        |
| us-employment-and-unemployment-rates                               | https://datahub.io/JohnSnowLabs/us-employment-and-unemployment-rates/v/2                               | No        |
| chicago-affordable-rental-housing-developments                     | https://datahub.io/JohnSnowLabs/chicago-affordable-rental-housing-developments/v/2                     | No        |
| vehicle-occupant-safety-data                                       | https://datahub.io/JohnSnowLabs/vehicle-occupant-safety-data/v/2                                       | No        |
| chicago-traffic-tracker                                            | https://datahub.io/JohnSnowLabs/chicago-traffic-tracker/v/2                                            | No        |
| imf-world-economic-outlook-database                                | https://datahub.io/JohnSnowLabs/imf-world-economic-outlook-database/v/2                                | No        |
| chicago-bike-racks-map                                             | https://datahub.io/JohnSnowLabs/chicago-bike-racks-map/v/2                                             | No        |
| us-states-and-territories                                          | https://datahub.io/JohnSnowLabs/us-states-and-territories/v/2                                          | No        |
| chicago-alternative-fuel-locations                                 | https://datahub.io/JohnSnowLabs/chicago-alternative-fuel-locations/v/2                                 | No        |
Anuar Ustayev
@anuveyatsu

Folks, I just spent a couple of hours uploading 43 datasets. It was a very frustrating to find that only 3 of those datasets made it to the datahub website, even though the data utility uploaded everything without an issue. Here are the results:

@MAliNaqvi Hi Ali! As I can see all datasets was uploaded successfully, however, most of them have validation/processing issues. You need to be logged in to see those errors. I know that you’re using an org account so the best way to check would be to pass your JWT within query params, e.g., try this https://datahub.io/JohnSnowLabs/chicago-traffic-tracker/v/2?jwt=<your-jwt> so that you are able to see FAILED dataset page.

Malikah
@Malikah95
hi
I'm thinking of a data set for each disease, his different levels and his symptoms, in order to design a tool for medical diagnostic.
medical diagnostic. or for only one disease like heart and like that ?
Anuar Ustayev
@anuveyatsu
@Malikah95 Hi! Could you please send a request using our service here - https://datahub.io/requests ?
You could also take a look at existing datasets, e.g., some can be found here - https://datahub.io/machine-learning
Irakli Mchedlishvili
@zelima

@akariv dataflows' sort_by processor does not seem to be working as expected, any ideas?

from dataflows import Flow, printer, sort_rows

data = [
    {'data': 'B'},
    {'data': 'E'},
    {'data': 'C'},
    {'data': 'D'},
    {'data': 'A'},
]

f = Flow(
      data,
      sort_rows('data'),
      printer()
)
f.process()

results with

res_1:
  #  data
     (string)
---  ----------
  1  B
  2  E
  3  C
  4  D
  5  A
hm... OK, so looking at the docs here seems like I need {} around data https://github.com/frictionlessdata/datapackage-pipelines#sort
from dataflows import Flow, printer, sort_rows

data = [
    {'data': 'B'},
    {'data': 'E'},
    {'data': 'C'},
    {'data': 'D'},
    {'data': 'A'},
]

f = Flow(
      data,
      sort_rows('{data}'),
      printer()
)
f.process()
Irakli Mchedlishvili
@zelima
Great, now I've got
res_1:
  #  data
     (string)
---  ----------
  1  A
  2  B
  3  C
  4  D
  5  E
Shreenikhil.m.c
@shreenikhil_twitter
I would like to know to know the original source of this dataset how do I know it
and the name of the dataset is 2016 Survey of Consumer Finances (Summary Extract) and the link is
Irakli Mchedlishvili
@zelima
@shreenikhil_twitter hi, unfortunately owner of the dataset has not provided source of the dataset in non of metadata of it or README. I would try and @ owner and wait for him/her to respond.
Usually usernames on DataHub and GitHub match as people register with their GitHub accounts
Konrad Höffner
@KonradHoeffner
Hello, I would like to apply for an organization account.
Anuar Ustayev
@anuveyatsu
Hi @KonradHoeffner could you please fill in this form - https://datahub.io/docs/features/teams-and-permissions
Konrad Höffner
@KonradHoeffner
Will do, thanks for the extremely quick response! :-)
Anuar Ustayev
@anuveyatsu
You’re welcome :smile:
Michael Decklever
@MDeck06_twitter
Hello. I found a wonderful dataset of S&P Financial data. Would you happen to have this same dataset over the past 10 years archived?
S&P 500 Companies with Financial Information
Mostafa Senousy
@Senousy_gitlab
I need datasets for Eczema disease ( images processing)
Is there any help
all my appreciated for your cooperation
estebanruseler
@estebanruseler
@MDeck06_twitter Thanks for writing to us. Our data engineer @Branko-Dj has done some work in this area and will get back to you tomorrow. We can respond on this thread or if you fill out a data request we can get back to you via email https://datahub.io/requests
@Senousy_gitlab Thanks for reaching out. On this one the best option is to send us a https://datahub.io/requests and one of our data engineers will get back to you
Mostafa Senousy
@Senousy_gitlab
Thanks
I wait you
Branko
@Branko-Dj
@MDeck06_twitter We are currently working on the dataset regarding this specific data. We will let you know once we have it prepared
EGBODOFO ADEBAYO
@GREATLIBERTY_gitlab
hello, please i need API for Laravel to set up my data account
Michael Decklever
@MDeck06_twitter
@Branko-Dj Thank you Branko!
Branko
@Branko-Dj
@MDeck06_twitter you're welcome :)
Anuar Ustayev
@anuveyatsu
@GREATLIBERTY_gitlab Hi, not sure what you mean. You can setup datahub account using the data CLI tool or on our website.
sebaswm
@sebaswm
Hello community :) I am looking for an API of European food and beverage producers filtered by product, sector & revenue. Does anybody know if this information is available openly?
Zane Selvans
@zaneselvans
Hey there. I changed which email address is listed as the primary address on my GitHub account, which is linked to my DataHub account, and now when I log in to DataHub w/ GitHub... I am a different person with a different username, and all of my uploaded datasets have disappeared. This seems like it might not be the desired behavior.
Gaurav Gireesh
@gaurav-gireesh
Hi gitters! I was trying to play around with GMSL dataset:
https://datahub.io/core/sea-level-rise/r/csiro_alt_gmsl_mo_2015.csv
I was trying to train a simple neural network to carry out poisson regression on GMSL data. As a loss function I am using PoissonNLLLoss function. While training I can see the loss decreasing with time, which is as expected. However, I am not able to understand, what I am trying to predict here and how do I test accuracy of this model?
Any ideas are welcome!
Thanks,
Gaurav
Anuar Ustayev
@anuveyatsu
@zaneselvans thanks for letting us know! We’ll need to discuss it with the team. Would you mind to open an issue here https://github.com/datahq/datahub-qa/issues ?
@sebaswm Hey! Not sure if such APIs are available at all. Try to look through existing questions here may be it’ll help - https://opendata.stackexchange.com/search?q=food
@gaurav-gireesh Hi there! Good to know you’re playing around with it :smile: :+1: Unfortunately, I’m not an expert on it so cannot help but I hope somebody in the community will :pray:
@Branko-Dj any ideas?
Zane Selvans
@zaneselvans
@anuveyatsu done! datahq/datahub-qa#244
Anuar Ustayev
@anuveyatsu
@zaneselvans thank you very much :+1:
Branko
@Branko-Dj
@anuveyatsu @gaurav-gireesh I am not sure what you are trying to do. Could you elaborate on what are you trying to accomplish?
Johan Richer
@johanricher
@sebaswm Partial answer to your question (better than nothing I guess): https://openfoodfacts.org
(it's a French project but increasingly popular, I can introduce to the team if needed)
José Ferraz Neto
@netoferraz
Hello! There is a way to use data over a company proxy ?
lapidus
@lapidus
Hi! Appreciated if anyone can shine some more light on Datapackages and DBMS.
We have 100s of datapackages with different schemas and want to make them accessible behind one API with at least basic filtering across available dimensions (for production web application use). Are there some readymade solutions for this? (Does for example https://datahub.io/data-factory help with parts of this or are there other solutions to automatically ingest into Postgres, MongoDB etc?)
Anuar Ustayev
@anuveyatsu

Hi @lapidus great to hear from you!

I believe Data Factory could be a good fit for what you’re trying to accomplish. @akariv could you please suggest?

lapidus
@lapidus
This message was deleted
Thank you @anuveyatsu! @akariv: Any recommendations very appreciated here or on DM.
sebaswm
@sebaswm
@johanricher Wow, very interesting openfacts.or., thank you very much. Would it be possible to speak with the team ?
Adam Kariv
@akariv
@lapidus hey, so - you can use dataflows to easily load datapackages into a relational database (postgres and the like). Just create a flow with the load and dump_to_sql processors.
For a good api, I recommend using babbage, which provides facts and aggregation endpoints over such DBs.