These are chat archives for FreeCodeCamp/DataScience

22nd
Mar 2016
evaristoc
@evaristoc
Mar 22 2016 13:51
@Lightwaves I did it! But I had to implement sooo maaaany unexpected tricks... using not pure d3.js but having to use jQuery... for some reason d3.js has been unable to cascade attributes when working with lines... I am really wondering why my final solution is working instead of what looked a typical solution to this... weeeeeeiiiiiirrdd....
network_graph_chat_201603221452.png
evaristoc
@evaristoc
Mar 22 2016 13:57
now is only selecting the first degree nodes attached to any selected node AND hiding those links and nodes that were not of first degree (before was only hiding the nodes but not the links)
Anyway: at least working... I am planning a possible viz combining Network Graphs with time: see how they change along the time.... but it is possible that a viz like that wouldn't have analytical value but aesthetic value... I hope it works... not priority though...
evaristoc
@evaristoc
Mar 22 2016 14:03
@samosale: we are working on trying to get all the campsites through a scrap... We would like to complement your project so it is not only aesthetic but also could provide useful info for the CoreTeam... I know you are busy so the idea is you owning the viz, while we probably adding other ones + stats... I am not sure if we could work on your repo, so we might need you to receive and accept PR's...
Otherwise we would fork your project, giving you the well deserved recognition for your project...
Aleksandar B.
@aleksandar-b
Mar 22 2016 14:08
Whatever you wish @evaristoc . :+1: Glad to see someone is working on dataviz
evaristoc
@evaristoc
Mar 22 2016 15:56

Hi People,
As usual, a bunch of links about topics I am reading...

This time, an unfortunate hot topic at this side of the world (and recurrent one for any part in the world, more sadly...). Social Media Analysis and Security. For some of us, that could be considered an opportunity, which I consider a bit ironic but it is what it is. A link about it:
http://www.computerweekly.com/feature/Social-media-a-security-challenge-and-opportunity

CamperBot
@camperbot
Mar 22 2016 15:56

welcome to FreeCodeCamp @evaristoc!

Alice Jiang
@becausealice2
Mar 22 2016 15:57
Hey @evaristoc :D
evaristoc
@evaristoc
Mar 22 2016 15:57
Hey!!! How is it going?
Alice Jiang
@becausealice2
Mar 22 2016 15:58
The scraper is frustrating me so I'm taking a break ;)
I can't get the data to drop escaped characters :(
evaristoc
@evaristoc
Mar 22 2016 15:58
Let's work on it together tomorrow if you want... You may need a bit of regex then...
Alice Jiang
@becausealice2
Mar 22 2016 15:59
I tried regex. I even ran it on refiddle to test that the expressions work but they didn't affect the data
evaristoc
@evaristoc
Mar 22 2016 16:01
Ok... let's see it together tomorrow? We can try to arrange a pairing session for that if you want... I am in Amsterdam
Alice Jiang
@becausealice2
Mar 22 2016 16:02
Sure. I'm free all day.
evaristoc
@evaristoc
Mar 22 2016 16:10

People,
Another link about Social Media, more general and with a marketing focus.
http://www.deloittedigital.com/us/blog/social-media-analytics-what-its-good-for-and-what-its-not


OBS:

Although the use of Social Media for marketing is commonly stressed, be aware that these types of analyses have a broad application. You may already know about the use of Social Media Analytics Apps for catastrophe management, for example.

You might think that given the numerous applications you as developer will be already successful by trying a general utility. Unfortunately that is difficult to achieve. Sometimes the specialisation is the key.

For example, I was just listening to the radio where someone was speaking about its use in Security and the person said something that we should take in consideration. Simply put, the person said:

The existing information is so vast that nowadays you have to specialise.

This is an interesting observation that I already mentioned to @darwinrc in a previous post.

However there are many common places that would allow different specialised implementations to share the same room (the DataScience Room?).

So post about your interests!

evaristoc
@evaristoc
Mar 22 2016 16:21
It is not that generalist solutions might not be successful, but in my opinion they should be somehow more disruptive...

I will post about technology management and innovation topics in future posts too
evaristoc
@evaristoc
Mar 22 2016 19:54

People,

This one about algorithms and NP-hard problems. Taking this one from the python room:
http://nbviewer.jupyter.org/url/norvig.com/ipython/xkcd1313.ipynb

Nice approach to the solution of that problem! Link suggested by @alayek

Aleksandar B.
@aleksandar-b
Mar 22 2016 19:59

@evaristoc I see you are having problems with parsing data from https://github.com/FreeCodeCamp/freecodecamp/wiki/List-of-Free-Code-Camp-city-based-Campsites

I submited PR to github that contains 950 cities with coordinates https://github.com/samosale/wiki/blob/ccb84dce126a4019c753c09aef141d27cb6261f5/Campsites.json

The data was last updated 1 and a half monts ago so I guess it is outdated, but you could easely update it. I also see that you are having problems with \n. Then you could just require that file in your local repository and split it by \n. Then the rest is easy.

evaristoc
@evaristoc
Mar 22 2016 20:08

@samosale: @alicejiang1 is estimating there are more than 1000 campsites. I thing we are going to try to get as many cities as there are in the file and then use the Google Maps API I mentioned to you.

If that is all what we need to run your code, great! But then I was to suggest to take those numbers to make some additional crunching stats. For example, by (sub-)continent, main countries, growth estimates, etc... This is what will take your viz a bit further...

That was my idea... what do you think?

Alice Jiang
@becausealice2
Mar 22 2016 20:08
Maybe I should ask what the end goal for the data is and clarify that if the end goal is to completely automate the task of writing out the JSON file for the data viz @samosale did, it needs more work. Otherwise if all we need is a list of cities/facebook links that's done
There's 1150 by my count
evaristoc
@evaristoc
Mar 22 2016 20:09
@alicejiang1 with your code there will be just a step to create an update, don't worry...
We can share all between the three...
Alice Jiang
@becausealice2
Mar 22 2016 20:10
But I can't get geographic coordinates without also pulling regions/countries to avoid confusing cities with same names, and the markdown is throwing off my scraper.
evaristoc
@evaristoc
Mar 22 2016 20:10
You reach as far as you want and can reach...
Google Maps has an API for reverse geolocation
Alice Jiang
@becausealice2
Mar 22 2016 20:11
I'm a bored housewife. I've got time and interest to stretch a bit ;)
evaristoc
@evaristoc
Mar 22 2016 20:11
Hahahahahahahahahahahaha!!!!
Aleksandar B.
@aleksandar-b
Mar 22 2016 20:11
@alicejiang1 yes you are right. One of the problems with my data is that it mixes up same city names from different states
evaristoc
@evaristoc
Mar 22 2016 20:11
Hahahahahahahaha!!!!!!!!!!
Alice Jiang
@becausealice2
Mar 22 2016 20:11
Right I was planning to use that but if two cities have the same name Google will return wrong coords
Aleksandar B.
@aleksandar-b
Mar 22 2016 20:12
exactly
Alice Jiang
@becausealice2
Mar 22 2016 20:12
@samosale exactly. And with 1150 cities I am not doing that by hand :/
Aleksandar B.
@aleksandar-b
Mar 22 2016 20:12
@alicejiang1 did you menage to scrap the site
evaristoc
@evaristoc
Mar 22 2016 20:12
Good point, but I think you can also include other parameter, like continent.. SORRY: country
Anyway: we can not ask for perfection neither
But it is indeed a good point
Alice Jiang
@becausealice2
Mar 22 2016 20:12
Yep
I can get a clean list of city names and Facebook group links but when I try to add regions and countries it gets real ugly real quick
evaristoc
@evaristoc
Mar 22 2016 20:14
I will check that people: tomorrow? Today I want to get into the code comparison project a bit more...
(bored housewife!!!! hahahahahahaha!)
Aleksandar B.
@aleksandar-b
Mar 22 2016 20:14
@alicejiang1 hm, do you remember bonfire Find the largest number in the arrays, or something like that?
i think you can do that similary
evaristoc
@evaristoc
Mar 22 2016 20:15
@samosale ??? interesting!!! (I haven't solved any algos, but it is really nice that you are using them as reference!!!!)
Aleksandar B.
@aleksandar-b
Mar 22 2016 20:16
@evaristoc yeah i remember that algo it was really pretty exercise
Alice Jiang
@becausealice2
Mar 22 2016 20:16
@evaristoc it's true! I'm in a new city in a neighborhood where I can't go out often, I don't like my neighbors, and my husband is a full time student. This, math, and cooking is kind of all I have to do right now hahaha
evaristoc
@evaristoc
Mar 22 2016 20:17
I havent seen the wiki page as much as you... You know better than me so far... @samosale great man!!!
Alice Jiang
@becausealice2
Mar 22 2016 20:17
@samosale I don't remember that, I'll go see if I can find it
Aleksandar B.
@aleksandar-b
Mar 22 2016 20:17
@evaristoc he tnx man :D
CamperBot
@camperbot
Mar 22 2016 20:17
samosale sends brownie points to @evaristoc :sparkles: :thumbsup: :sparkles:
:star: 240 | @evaristoc | http://www.freecodecamp.com/evaristoc
evaristoc
@evaristoc
Mar 22 2016 20:18
@alicejiang1 I live in Amsterdam... sorry I can't help you...
Difficult to get bored here, even if you have freaky neighbours...
@alicejiang1 But, you see? You are maybe going in the same direction than Newton, Galileo Galilei and all those guys that didn't live in Amsterdam by that time...
Math... Computers... Food... probably painting a dead body...
You will discover something new for sure!!!
Alice Jiang
@becausealice2
Mar 22 2016 20:22
My neighbors are all stereotypical white college students. Like in movies, frat boys and sorority sisters.... Parties every weekend.... So I stay home by myself and learn how to data science :)
Painting a dead body what
Amsterdam sounds like my kind of place ;)
evaristoc
@evaristoc
Mar 22 2016 20:22
I don't know: here every one is stereotypical white college students. Like in movies, frat boys and sorority sisters.... Parties every weekend....
I don't know how the things are at @samosale place...
But I guess it is the same as here...
No, well, a lot of museum of people who used to paint dead bodies too... in Amsterdam...
Alice Jiang
@becausealice2
Mar 22 2016 20:25
Sounds like a good time to me!
evaristoc
@evaristoc
Mar 22 2016 20:25
Indeed it is!
Beautiful city!!!!!!!!!!!!!!!!!!!!!!
Alice Jiang
@becausealice2
Mar 22 2016 20:28
I will have to add it to my list of places to visit ;)
evaristoc
@evaristoc
Mar 22 2016 20:30

Yes!!! You really really really have to!

People trying to get something done soon so take care! I hope we can grasp this stuff by the three of us, learning something new and having a bit of fun!

C u around!!!

Darwin RC
@darwinrc
Mar 22 2016 22:38
I found this rather old page (2013), does anyone have taken a look at it? http://www.gapminder.org/world
evaristoc
@evaristoc
Mar 22 2016 23:15

@darwinrc going to sleep... but the guy is a CLASSIC on how to make vivid presentations: Try to find him at TED...

See you!

Lightwaves
@Lightwaves
Mar 22 2016 23:17
@evaristoc haha all the freaks live in amsterdam?
evaristoc
@evaristoc
Mar 22 2016 23:17
:)
Lightwaves
@Lightwaves
Mar 22 2016 23:18

from what @alicejiang1 described sounds like they are living in an area similar to me.

Not exactly the best place to go out at night alone.

Man or Woman