sclinede on master
Fix undef method when Bestgems … (compare)
The main problem of machine learning here is to proof that it provide valid results.
Of course, when it is obvious whether or not our ML gives us good results it make sense.
If we will feed the ML with bad data that is not prepared properly and data that is not correlate with our decision we'll receive unexpected result and I don't think it will give us much profit.
I've started with Decision Trees and some metrics and found it is not working well.
So the next move was to discover more raw metrics and provide valuable higher order metrics upon them.
And here I found that we have many corner cases that will not be covered by interview of 3K people and 30 packges, believe me.
I'm sure that ML will help us, but not now.
At first we need to understand how to prepare as much clean stats as possible before feeding ML.
That's my point, but if you'll have working solution - that's great!
I didn't though about interviews but I think it will be more simple if we will have a prototype and feedback form, where interviewers could say what is missing for their decision. And... we already have a prototype just need to add feedback form and start the whole process)
As I understood, you'll face the problem of subjectivness in your solution as decisions would be made by human. That means that even if Dan Abramov and other huge part in our reference group think that some project is bad it could be caused just by their feelings about its community or hype or something else and not about quality of work that is done for it.
I agree with idea of asking people. That is why I started that project, I want as much people as possible to think about what is good open-source is and create a system that can measure it.
I think it would be much better to prepare somehow a questionnaire to see what do people use to make their decisions above their subjectivness and intuition, any marks or trends that they check to answer Why that project is so good/bad.