These are chat archives for FreeCodeCamp/DataScience

Aug 2016
Aug 21 2016 08:21
@kymanikd :) !
Farooq Abdul Rehman
Aug 21 2016 08:34
@kymanikd No I will never leave FCC, I've already done frontend but It would be hard to find a job locally
Farooq Abdul Rehman
Aug 21 2016 08:54
@evaristoc I joined FCC in May and have done frontend uptill now.. I found FCC on Googling ... If there is really a peak than that's very good ...
Aug 21 2016 09:17


Nice! You are progressing very fast! Yes, I hope you will be able to find new more people from your country by the FCC channels and probably organise an encounter there? That would be great!

Similarly I also hope you are enjoying the rest of the int'l community as well!!

Aug 21 2016 10:45


What can we expect in the future of the Big Data / Data Science industry?

Having studies in Innovation and Technology Topics this is an interesting topic for me. My question would be:

What could we expect from the job market of Big Data / Data Science in the future taking in consideration some current trends in the industry?

No more than 6 years ago I remember a friend who was completing studies to become a Clinical Data Manager (CDM). A CDM is a person who has to follow the collection and transfer of the data for clinical trials from the start to the end of the trial. It is a very specialised job. About 6 years ago the job market was still healthy in Western Europe (there are numerous clinical trials in the continent) and the positions well paid. But suddenly, this friend ended up not finding jobs, and all the positions suddenly disappeared. Why? There was a centralisation of the activities and all the data collected in Europe and around the world for a clinical trial would be managed from a single operator in US, for example.

Boom. No more jobs as CDM and that happened very quickly, just in a period of 2-3 years.

In the Big Data sector the trend is apparently this:

  • Big Data = Big Players: IaaS. Some companies are investing in the infrastructure required for Big Data and offering the infrastructure in cloud for Big Data warehousing. Amazon, Google, MS, Teradata and IBM are some of the best examples. Clients using those services won't need the engineering for the infrastructure: administrators will come handy enough. Nowadays is becoming really expensive to reach the service scale of the existing big players, becoming the required capital investment a strong barrier of entry.
  • PaaS / SaaS: On top of that, companies are offering more mechanisms that will make easier to interact with your data using the tools of your preference. Here is a real fierce competition for USERS: getting a critical mass of users will be key to define who becomes the de facto standard, so there is a campaign to let you start using platforms for free. A list of some:
    • IBMl's DataScientistWorkBench and BlueMix (who is very close to corporative market)
    • Google's TensorFlow and Google CloudMachineLearning
    • MS's Azure (also very close to corporative market)
    • Databricks-Spark on Amazon
    • Cloudera-Hadoop ML (originally Yahoo! tech) tools on Amazon (losing terrain to Spark)
    • another simple example of a new entrant: (tough! only niche for now, and no matter how good the technology could eventually be left just like concept proposal...)

What that means? It is possible that some leaders in the sector are pushing to centralization. Companies like Teradata, Pentaho, Datos, or DataDriven already offer centralised data analytics services since some time ago. A hardcore indication of this trend comes from companies dedicated to automated online marketing, or those in the stock market sector, offering automated bidding mechanisms. Those bidding mechanisms are not perfect though, but good enough for commodity services.

So questions that you would like to ask yourself now:

  • Which technology should I learn first?
  • If I get a job, is that position threatened by centralization? (there will be some Merger&Acquisition at some point...)
  • If my position is threatened by centralization, how much time do I have before that happens and then what am I gonna do to pay my rent after that!!!????

The point is: this sector as ALL sectors in IT could be changing VERY VERY fast, faster than you think... For what I have seen how companies change, you could expect that:

  • Most of the companies delay adoption until technology matures and they are prepared for...
  • ... but once that happens, many of those will trend to adopt rather quickly, in few years, in order not to stay behind

So making good selections and planning ahead could be very helpful, I guess...

Aug 21 2016 11:23

Here an opinion from an specialist about the competition between Google vs MS ML Cloud for adoption for their new offers:

Really interesting...

Eric Leung
Aug 21 2016 17:06

@evaristoc here's some general R resources I've yet to fully utilize but seem worthwhile

This one is specific to dplyr and tidyr

Aug 21 2016 22:15
@erictleung :+1: Thanks!
Aug 21 2016 22:15
evaristoc sends brownie points to @erictleung :sparkles: :thumbsup: :sparkles:
:cookie: 405 | @erictleung |