scikit-learn/scikit-learn

scikit-learn: machine learning in Python. Please feel free to ask specific questions about scikit-learn. Please try to keep the discussion focused on scikit-learn usage and immediately related open source projects from the Python ecosystem.

Andreas Mueller

@amueller

I'd recommend using conda and doing conda install numpy scipy cython matplotlib pytest flake8 sphinx sphinx-gallery or something like that

Kristiyan Katsarov

@katsar0v

@amueller by the way, numpy and scipy from conda perform somehow faster than from pip

but I still haven't found out why

Andreas Mueller

@amueller

@katsar0v thats mkl vs openblas possibly

but could also be how they are configured by default

i.e. how many threads they use etc

Kristiyan Katsarov

@katsar0v

@amueller how can I reconfigure numpy and scipy to use max threads e.g. 6?

I have no mkl (from conda or pip)

Andreas Mueller

@amueller

pip has no mkl ;)

(so far)

https://stackoverflow.com/questions/30791550/limit-number-of-threads-in-numpy

Kristiyan Katsarov

@katsar0v

https://pypi.org/project/mkl/

Andreas Mueller

@amueller

@katsar0v I don't think that helps given that numpy and scipy will not be linked against it

Kristiyan Katsarov

@katsar0v

this saved my life @amueller

Andreas Mueller

@amueller

well in your script n and m are way too small to show anything useful

Kristiyan Katsarov

@katsar0v

it reduced my validation curve

from 500s to 15 seconds

@amueller this is a life saver

Andreas Mueller

@amueller

what did?

Kristiyan Katsarov

@katsar0v

https://stackoverflow.com/questions/30791550/limit-number-of-threads-in-numpy

these envs

Andreas Mueller

@amueller

well stackoverflow saved your live

*life

Kristiyan Katsarov

@katsar0v

It's good for performance tweaks

Samesh Lakhotia

@sameshl

How should I build the docs for harversine_distances in my local repo? I ran python setup.py install but still I can't find it under doc/modules/

Guillaume Lemaitre

@glemaitre

The documentation is another command line

cd doc
make html

should work all OS I think

then it will create a _build/html folder and you can search for the index.html

Loïc Estève

@lesteve

@sameshl note this part of the contributing scikit-learn doc: https://scikit-learn.org/stable/developers/contributing.html#documentation

If you see ways the contributing doc can be improved while you face this "setup" issues, let us know or/and open PRs to improve the contributing docs!

Samesh Lakhotia

@sameshl

@lesteve Sure. Thanks for the help.

As a beginner contributor to this organisation, the arrangements of the docs did feel a bit tough to navigate. I will put my thoughts about it more concisely and then open a issue and PR for the same

Adrin Jalali

@adrinjalali

We're working on improving our contributing docs @sameshl, there's some discussion under #14582

Samesh Lakhotia

@sameshl

Thats great. Would love to contribute on scikit-learn/scikit-learn#14582

I am working on scikit-learn/scikit-learn#14575. So I found the corresponding example under sklearn/metrics/pairwise.py. My question is, are the examples run in the doc building process and output is generated or I am supposed to manually write the output of the example in the docstring of a function?

Adrin Jalali

@adrinjalali

you should write the output in the example. The doc build will run the code and check if the generated output is the same as the one you put there. See https://docs.python.org/3.5/library/doctest.html for more info

Samesh Lakhotia

@sameshl

Thanks @adrinjalali !

Vishesh Mangla

@Teut2711

Does anyone here knows a good source to learn rnn structure ?

Is it like replacing every hidden node with a rnn cell?

Samesh Lakhotia

@sameshl

I am working on scikit-learn/scikit-learn#14131 . So, I thought that I could append a note in the docstring of KDTree regarding the issue. But I looked into sklearn/neighbors/kd_tree.pyx and it looks like KDTree is inheriting its docstring from BinaryTree. So can someone tell me an elegant way to append my note docstring to the inherited docstring of KDTree or if I could do something else to solve this issue.

Manish Aradwad

@ManishAradwad

Currently working on #14081.
I am supposed to create a pitfalls section which includes practices not to be followed by users. Quite confused about how should I approach it, should I create a whole new section in documentation.html or is there another way to do this??
Thanks for the help!!!

Peng Yu

@yupbank

Hey channel, i’ve being working on vectorizing regression tree with Numpy, and i have achieved some speed up against the cython version of sklearn. in case anyone is interested, here is the link https://github.com/yupbank/np_decision_tree#regression-with-mae

Peng Yu

@yupbank

on median data(10000*100), with MAE criteria, achieved 20 times speed up :)

Adrin Jalali

@adrinjalali

still haven't checked the code in depth. But it's definitely interesting @yupbank . What do you think @NicolasHug ?

Peng Yu

@yupbank

i haven’t clean the code yet, and also working on a blog post explainning what i did, and add some CI to it. But i would love to have some extra inputs before i proceed, e.g. reviews.

Adrin Jalali

@adrinjalali

I don't think it'd be easy, but I'd love to see if it actually passes our tree tests, and if it doesn't why not and which tests. Feel free to ping me when you write the blog post.

Peng Yu

@yupbank

sure.. that would be nice,

Nicolas Hug

@NicolasHug

@yupbank pretty cool stuff! I took a quick glance at the tree grower and the greedy_split function and it looks good as far as I can tell. I wouldn't advertise benchmarks with only max_depth=1 though ;)
Please definitely ping us when you write the blog post!!

Peng Yu

@yupbank

lol, you are right, actually with max_depth=10, i only get 5 times faster.

Where communities thrive

People

Repo info

Activity