These are chat archives for FreeCodeCamp/DataScience

19th
Aug 2016
Eric Leung
@erictleung
Aug 19 2016 01:07
@QuincyLarson I don't know if you can answer the second question with the survey, unless I'm missing something. You only have country they reside in and region, which doesn't really help. Were you hoping to use country along with the remote variable together, implying they work in a different country?
Rplot01.png

@QuincyLarson I removed missing answers and compared people who responded "Full-time remote" or "Part-time remote" and compared them to the "I rarely work remotely" and the "Never"

I also normalized them by remote or not. (There are way more non-remote people than remote.)

Lightwaves
@Lightwaves
Aug 19 2016 01:09
That's a interesting trend, so remote developers seem to make more then nonremote developers
Eric Leung
@erictleung
Aug 19 2016 01:13

Here's my code to make that plot above in case people are interested

https://gist.github.com/erictleung/0ccf9a71b5e88b35ee93e6c0221453a1

@Lightwaves @QuincyLarson straight by raw numbers, non-remote developers make more just because there are more of them. But it appears from my plot that, relatively speaking, there are more remote developers on the higher salary end than non-remote developers.
Eric Leung
@erictleung
Aug 19 2016 01:25
FYI, the StackOverflow developer survey data is 66.5 MB unzipped.
Quincy Larson
@QuincyLarson
Aug 19 2016 06:49
@erictleung wow - that was fast!
@erictleung great - do you know what the median salaries were for remote VS non-remote? Based on bands of years of experience?
I am really impressed you got this together so quickly.
Eric Leung
@erictleung
Aug 19 2016 07:29

@QuincyLarson R makes it easy to get what you want :smiley:

Remote:

  • Med: $55,000
  • Standard Deviation: $51,199.51
  • Number of people who responded with salary info: 10,583

Non-Remote:

  • Med: $45,000
  • Standard Deviation: $43,727.47
  • Number of people who responded with salary info: 25,413

Note: the salary numbers here are midpoints of the ranges they asked. So these median values are actual median values of ranges. Just something to keep in mind.

@QuincyLarson

Based on bands of years of experience?

What did you mean by this? I'm heading to bed now. Let me know if you needed any other quick stat from the dataset.

Quincy Larson
@QuincyLarson
Aug 19 2016 08:47
@erictleung Awesome! This is really helpful. I was wondering whether at some point in your career (years experience) it starts to pay more to work remotely than on-site. If you could figure out the median salary for candidates within specific bands of years of experience, this would help illustrate whether such a phenomenon exists.
evaristoc
@evaristoc
Aug 19 2016 09:18

@QuincyLarson Yes, I haven't reproduced the data yet but I was just working on something similar... I will come to you later but so far, when comparing current salary against working remotely there are two points at which working remotely seems to occur:

  • Apparently very low salary will trend to find jobs remotely more frequently, possibly indicating no benefits and part-time jobs
  • Then those who are earning average salary will seldom work remotely; this is a possible indication of critical positions where being close to the source is relevant...
  • ... until your salary increase, then, you will trend to work mostly sometimes remotely, possibly indicating more mobility for work reasons, or perhaps having the benefit of working from home some days per week

I think we cannot extract from this data how far the person work, if that is what interest you: the data only suggests if the person works remotely or not, but not from where. We should have to assume things or maybe using other external sources to confirm the findings.

Another aspect that we could try to solve is to which sector the person works. And then country of course. Better to take as reference the most important countries (we could suggest which could be our focus based on GA information, for example) and obtain an accumulative for the rest of the world.

I am trying to work on a quick d3.js although it would be easier to show my results in R or any other tool...
evaristoc
@evaristoc
Aug 19 2016 09:23
@erictleung what I am doing is, instead of between, comparing percentages within. The samples have different sizes, and the within information is more relevant for what @QuincyLarson is expecting...
evaristoc
@evaristoc
Aug 19 2016 10:16
salaryvsremote.png
@QuincyLarson @erictleung : a quick chart to support what I was previously saying...
evaristoc
@evaristoc
Aug 19 2016 11:23

@QuincyLarson although the results are suggesting a trend, that trend is led by possibly an overwhelming number of responses coming from few countries, like US, India, Canada, or UK. However there are countries that for either job culture or infrastructure or other reasons the remote work won't be encouraged and even allowed.

To elaborate better on those trends we could try to analyse the data per country segments. We have some data about country key measures used for an article I made for medium at the beginning of this year. If you are interested we can try to use that to see what pops up...

The other question that I think we should ask you is: how do you want those graphs to be made? d3.js, or simple svg/png?

evaristoc
@evaristoc
Aug 19 2016 11:49
@QuincyLarson @erictleung I am preparing a similar draft of a chart like above but in d3.js, it is 80% ready but need few adjustments - have to go so we are in contact later today?
evaristoc
@evaristoc
Aug 19 2016 11:54

People

JUPYTER and Polyglotism

I am using the Jupyter (although not Anaconda yet) platform to code in python, R and now some d3.js to prepare the project suggested by @QuincyLarson, all in the same notebook. Although not the best code ( :( ) I will make part of that notebook available, as an example of how to write in several languages on Jupyter notebook.

Not sure if a real asset. Polyglot approaches could be also very confusing and likely less efficient that using only one central language. And many languages are incorporating ways to handle other languages and still look like native. A close example is R which has incorporated new libraries to write in d3.js.

In any case, notice that we are already being polyglots when using the Databricks platform, for those who are following the Spark course. SQL, python, some R for those who want to, and Spark, all on the same interface: Jupyter Notebooks.

Eric Leung
@erictleung
Aug 19 2016 16:57
@evaristoc ah I see. Yeah, that's another way to slice the data that makes more sense for the question. It took me a second to figure out what the plot is saying relative to my plot because the trend is different than mine. But it is showing something different so that's why :laughing:
@evaristoc here's an interesting site discussing the benefits and drawbacks of polyglot programming, among other things https://deanwampler.github.io/polyglotprogramming/
evaristoc
@evaristoc
Aug 19 2016 18:49
@erictleung :+1:!! Thanks!
CamperBot
@camperbot
Aug 19 2016 18:49
evaristoc sends brownie points to @erictleung :sparkles: :thumbsup: :sparkles:
:cookie: 402 | @erictleung |http://www.freecodecamp.com/erictleung
evaristoc
@evaristoc
Aug 19 2016 19:09
@QuincyLarson @erictleung:
Quick codepen, adding some interactivity:
http://codepen.io/ecccs/full/rLbJyG/
evaristoc
@evaristoc
Aug 19 2016 19:14
@erictleung the link is very good... I guess you already knew the owner of the blog is also talking about big data and Spark, functional programming and Reactive Extensions?
@erictleung... also QCon... didn't know about that either! Very good place to check!
evaristoc
@evaristoc
Aug 19 2016 19:31
People:
Following @erictleung link about pro's and con's of polyglot approach, a presentation by D Wampler in 2009 that it is worth watching if you are interested in this topic!
https://www.infoq.com/presentations/polyglot-polyparadigm-programming
Eric Leung
@erictleung
Aug 19 2016 21:17
@evaristoc cool interactive plot of the salaries! :+1: I'm jealous of your D3 skills :stuck_out_tongue:
evaristoc
@evaristoc
Aug 19 2016 21:40

@erictleung thanks, man! It is not really much but surely better than when I started here for sure :) , at least allows for quick prototyping at the moment...

and to return the favour --->> I have been learning some very good R stuff from you lately ;) .

CamperBot
@camperbot
Aug 19 2016 21:40
evaristoc sends brownie points to @erictleung :sparkles: :thumbsup: :sparkles:
:cookie: 403 | @erictleung |http://www.freecodecamp.com/erictleung
evaristoc
@evaristoc
Aug 19 2016 21:41
Going, take it easy everyone!