Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 06:02
    github-actions[bot] locked #10305
  • 06:02
    github-actions[bot] labeled #10305
  • 06:02
    github-actions[bot] commented #10305
  • 06:02
    github-actions[bot] locked #10298
  • 06:02
    github-actions[bot] labeled #10298
  • 06:02
    github-actions[bot] commented #10298
  • 06:02
    github-actions[bot] locked #10315
  • 06:02
    github-actions[bot] labeled #10315
  • 06:02
    github-actions[bot] commented #10315
  • 06:02
    github-actions[bot] locked #10313
  • 06:02
    github-actions[bot] labeled #10313
  • 06:02
    github-actions[bot] commented #10313
  • 06:02
    github-actions[bot] locked #10316
  • 06:02
    github-actions[bot] labeled #10316
  • 06:02
    github-actions[bot] commented #10316
  • 06:02
    github-actions[bot] locked #4739
  • 06:02
    github-actions[bot] labeled #4739
  • 06:02
    github-actions[bot] commented #4739
  • 06:02
    github-actions[bot] locked #4937
  • 06:02
    github-actions[bot] labeled #4937
Dave Hirschfeld
@dhirschfeld
...so I'm not sure what you're actually asking.
Ólavur Mortensen
@olavurmortensen
I want to write this in a YAML file, specifying each channel-package pair specifically, using the channel::package syntax. The thing is that Bioconda (https://bioconda.github.io/) packages require both the bioconda as well as the conda-forge channel (don't ask me why, I do not know). Therefore, I need to somehow specify bioconda::package and conda-forge::package simultaneously.
Uwe L. Korn
@xhochy
bioconda requires both channels because it uses a lot of packages from conda-forge as dependencies but a package is normally either on bioconda or conda-forge (if it's on both, the version on conda-forge should be preferred)
Thus you should use bioconda:: for the packages that are on bioconda and conda-forge:: for the other ones.
conda list --explicit shows you from where each package is coming.
Ólavur Mortensen
@olavurmortensen

Thanks @xhochy.

conda list --explicit says a particular package is from bioconda, but when I install from bioconda, without conda-forge installation fails (incompatible packages). Only when I specify both channels does the installation work.

So, somehow, this package depends on both channels, rather than just one of them, as you say bioconda packages should.
Uwe L. Korn
@xhochy
Yes, you will also need its dependencies
You are looking for conda install -c conda-forge -c bioconda bioconda::package
Alternatively conda install bioconda::package conda-forge::dependency_1 conda-forge::dependency_2 …
The package depends on both channels but is only part of one.
Ólavur Mortensen
@olavurmortensen
Alright, thanks @xhochy, those are useful alternatives. I should be able to make this work somehow.
mattchan-tencent
@mattchan-tencent
I'm building a docker image with a conda package in it. The MKL library is really blowing up the size of the image (700mb for mkl alone when a typical docker image is <300mb for everything). Has anyone tried removing the redundant mkl .so libraries from /lib? I straced it and it only loads like 5 of the 70 in the mkl package...
I know it changes the lib loading depending on the processor, but I'm pretty certain I won't ever be running this image on a MIC arch for example.
Jonathan J. Helmus
@jjhelmus
If you are concerned about size you may want to use the openblas library rather than MKL
conda will select the openblas variants of package is you specify blas=*=openblas in as a specification. For example conda create -n test python=3.7 numpy blas=*=openblas
mattchan-tencent
@mattchan-tencent
@jjhelmus I can't use openblas unfortunately... There's a package which is hosted in a user conda channel and it is only built with MKL
Also, it seems that openblas has fallen behind MKL fairly significantly as of late. 4x speedup on matrix muls with mkl...
mattchan-tencent
@mattchan-tencent
So I figured it out... Not sure if it's a good idea in the long run, but I found a listing of what each .so does
So I deleted all the shared objects for libs which weren't applicable to the architectures I'd be running on
Shaved a good 500mb off the image!
Wolf Vollprecht
@wolfv
nice idea!
I think conda is slowly getting support to have some features enabled by CPU architecture
maybe at some point the MKL package could be split so that one only installs the relevant libs
jakirkham
@jakirkham
Where does repodata get stored locally (after downloading it)?
Nehal J Wani
@nehaljwani
@jakirkham </path/to/package/cache/dir>/cache
jakirkham
@jakirkham

Got it! Thanks Nehal :smile:

It appears these have hashes in the names. What do those mean?

Nehal J Wani
@nehaljwani
The repo URL is hashed using a function inside conda
jakirkham
@jakirkham
:pray:
What are the .q files?
jakirkham
@jakirkham
What's the best place to look for the repodata of defaults? Should I look here?
Replace linux-64 with the subdir that you are interested in. There are also compressed files as well (.json.bz2)
Wolf Vollprecht
@wolfv
@jakirkham the .q files are pickled repodata afaik
jakirkham
@jakirkham
:+1:
David Brochart
@davidbrochart
I'm downloading conda-forge/linux-64 at 8 KB/s, it's been like that for more than 15 minutes now... and it's happening more and more often.
is it a know issue?
I'm from France, usually it works fine in the morning, but it looks like when the US wakes up everything collapses.
just did a quick download test on a web site, I'm at 100 Mbps
David Brochart
@davidbrochart
actually I just installed the same package on a remote machine in Germany, and it was fast, so I'm not sure what's going on in France...
jakirkham
@jakirkham
Is it possible there is a Cloudflare issue? defaults and conda-forge are mirrored to CDN hosted by Cloudflare. So that may be the issue
David Brochart
@davidbrochart
yes, could be
kylekeppler
@kylekeppler
Does conda have a mechanism to switch back to system python (then at some point go back to conda)?
kylekeppler
@kylekeppler
I think I am basically looking for the inverse of conda init
which is conda init --reverse but that didn't reverse $PATH
Anthony Scopatz
@scopatz
@kylekeppler - that sounds like a really bad idea because you are trying to undo conda's isolation
kylekeppler
@kylekeppler
I am aware and I agree. But unfortunately I've got a new project dropped in my lap that has 34945 dependencies that get installed to system python. So hoping I can disable conda for now, and re-enable it in the future when I can work on projects that are more well-behaved.
Anthony Scopatz
@scopatz
Ahh