These are chat archives for gindeleo/climate

15th
Mar 2016
Oliver Gindele
@gindeleo
Mar 15 2016 11:01
hey jo
Gregor
@echna
Mar 15 2016 11:07
moin
Oliver Gindele
@gindeleo
Mar 15 2016 11:09
seen this:

Nice. Also this is only the global data. There is plenty more for the cities and the states. I think you have to download it yourself from Kaggle since one file was to big to upload on github.

Let's see how many entries there are per city (e.g. how many dates) and if we need to do some clean up for that.

Then we could have a look at what the warmest/coldest places are and where most T change happened lately.

Gregor
@echna
Mar 15 2016 11:09
I did.
Oliver Gindele
@gindeleo
Mar 15 2016 11:10
loading all the data takes quite some time now
Gregor
@echna
Mar 15 2016 11:11
I think we should formulate a question for ourselves before jumping into cleaning all the data
Oliver Gindele
@gindeleo
Mar 15 2016 11:11
yup. ideas?
besides it might be fast to pickle the data then load it from there and not as cvs
csv
Gregor
@echna
Mar 15 2016 11:12
pickle?
Oliver Gindele
@gindeleo
Mar 15 2016 11:12
it saves the whole python object in binary, so it can be loaded again
Gregor
@echna
Mar 15 2016 11:12
just found i t
there is cPickle too :)
Oliver Gindele
@gindeleo
Mar 15 2016 11:14
yep
Gregor
@echna
Mar 15 2016 11:47
I doubt we can do anything too exciting too easily with this data. Maybe one could combine it with some other data set and see if there is some correlation ( with food production, natural disasters or just something seemingly unrelated). Or we can of course just look at this data itself. We might for example compare temps on different continents.
For example average all the data we have for europe, from cities, states, etc and compare it to the average in north america and see if there is anything interesting popping up
Oliver Gindele
@gindeleo
Mar 15 2016 11:51
yep, agreed. It might also make sense to focus on one data set. e.g cities or states.
Oliver Gindele
@gindeleo
Mar 15 2016 17:00
yep, pickling speeds it up a lot.
24 s vs 4 s for major city data
Oliver Gindele
@gindeleo
Mar 15 2016 17:06
major city has 100 cities with about 2000 temperatures (dates) each
Oliver Gindele
@gindeleo
Mar 15 2016 17:20
3448 cities in the city file
Gregor
@echna
Mar 15 2016 18:51
Nice!
I'll be at UCL tmr for a careers fair and to talk to my Prof.
Hopefully I can get the corrections out of the way soon. One less thing to to.