Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 11:23
    cla-bot[bot] labeled #3812
  • 11:23
    mingwandroid opened #3812
  • 10:13
    marcelotrevisani closed #3794
  • 10:13
    marcelotrevisani commented #3794
  • 09:18
    marcelotrevisani commented #3756
  • 09:17
    mingwandroid commented #3756
  • 09:15
    mingwandroid closed #3756
  • 09:15

    conda-bot on gh-pages

    remove docs for pr 3810 (compare)

  • 09:15

    mingwandroid on master

    Add macOS to Azure Pipelines a… Fix for Xcode binaries Fix detection of Python shebangs and 1 more (compare)

  • 09:15
    mingwandroid closed #3810
  • 07:47
    marcelotrevisani commented #3810
  • 07:47
    marcelotrevisani commented #3810
  • 07:40
    marcelotrevisani commented #3810
  • 00:11
    mingwandroid commented #3810
  • Nov 14 23:14
    mingwandroid commented #3810
  • Nov 14 22:46
    mingwandroid synchronize #3810
  • Nov 14 22:27
    mingwandroid synchronize #3810
  • Nov 14 22:25
    mingwandroid synchronize #3810
  • Nov 14 22:03

    conda-bot on gh-pages

    docs for pr 3810 (compare)

  • Nov 14 22:02
    mingwandroid synchronize #3810
Matthew R. Becker
@beckermr
the conda build ones are too long for most systems to deal with
the fun edge case here is that if you are working on a build on OSX that is doing somewhat dumb things with DYLD_LIBRARY_PATH, then you end up with the /usr/bin/env invoking SIP and breaking everything
blarg
Andrew Annex
@AndrewAnnex
thanks for merging in #3773, I believe that will fix my conda-forge feedstock once a new release is made, how soon until 3.8.11?
Ray Donnelly
@mingwandroid
Well, IMHO /usr/bin/env python is the correct invocation in that we respect and use PATH and not having to rewrite files per-env is a saving.
Jonathan J. Helmus
@jjhelmus
@beckermr That snippet is created by setuptools/pip not conda-build/conda. Entry points created by conda do not use pkg_resources, https://github.com/conda/conda/blob/4.7.12/conda/gateways/disk/create.py#L85-L106
Matthew R. Becker
@beckermr
Most of it is but not the shebang line
Let me show you from some packages
check out the pytest package here: https://anaconda.org/conda-forge/pytest/files
here is the contents of the command line util
(base) localhost:pytest-5.2.1-py37_0 Matt$ cd bin/
(base) localhost:bin Matt$ cat pytest 
#!/opt/anaconda1anaconda2anaconda3/bin/python

# -*- coding: utf-8 -*-
import re
import sys

from py.test import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
Notice that the #! line is the text that conda replaces with the prefix
here is what pytest looks like installed locally on my machine
(I have it in my base env)
(base) localhost:bin Matt$ cat ~/miniconda3/bin/pytest
#!/Users/Matt/miniconda3/bin/python

# -*- coding: utf-8 -*-
import re
import sys

from py.test import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
now it has the exact path to python
Matthew R. Becker
@beckermr
now check out this really silly env
(base) localhost:bin Matt$ conda create -n gggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg python=3.7 pytest
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /Users/Matt/miniconda3/envs/gggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg

  added / updated specs:
    - pytest
    - python=3.7


The following NEW packages will be INSTALLED:

  atomicwrites       conda-forge/noarch::atomicwrites-1.3.0-py_0
  attrs              conda-forge/noarch::attrs-19.3.0-py_0
  bzip2              conda-forge/osx-64::bzip2-1.0.8-h01d97ff_1
  ca-certificates    conda-forge/osx-64::ca-certificates-2019.9.11-hecc5488_0
  certifi            conda-forge/osx-64::certifi-2019.9.11-py37_0
  importlib_metadata conda-forge/osx-64::importlib_metadata-0.23-py37_0
  libcxx             conda-forge/osx-64::libcxx-9.0.0-h89e68fa_1
  libffi             conda-forge/osx-64::libffi-3.2.1-h6de7cb9_1006
  more-itertools     conda-forge/noarch::more-itertools-7.2.0-py_0
  ncurses            conda-forge/osx-64::ncurses-6.1-h0a44026_1002
  openssl            conda-forge/osx-64::openssl-1.1.1c-h01d97ff_0
  packaging          conda-forge/noarch::packaging-19.2-py_0
  pip                conda-forge/osx-64::pip-19.3.1-py37_0
  pluggy             conda-forge/osx-64::pluggy-0.13.0-py37_0
  py                 conda-forge/noarch::py-1.8.0-py_0
  pyparsing          conda-forge/noarch::pyparsing-2.4.2-py_0
  pytest             conda-forge/osx-64::pytest-5.2.1-py37_0
  python             conda-forge/osx-64::python-3.7.3-h93065d6_1
  readline           conda-forge/osx-64::readline-8.0-hcfe32e1_0
  setuptools         conda-forge/osx-64::setuptools-41.4.0-py37_0
  six                conda-forge/osx-64::six-1.12.0-py37_1000
  sqlite             conda-forge/osx-64::sqlite-3.30.1-h93121df_0
  tk                 conda-forge/osx-64::tk-8.6.9-h2573ce8_1003
  wcwidth            conda-forge/noarch::wcwidth-0.1.7-py_1
  wheel              conda-forge/osx-64::wheel-0.33.6-py37_0
  xz                 conda-forge/osx-64::xz-5.2.4-h1de35cc_1001
  zipp               conda-forge/noarch::zipp-0.6.0-py_0
  zlib               conda-forge/osx-64::zlib-1.2.11-h0b31af3_1006


Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate gggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) localhost:bin Matt$ conda activate gggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg
(gggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg) localhost:bin Matt$ cat `which pytest`
#!/usr/bin/env python

# -*- coding: utf-8 -*-
import re
import sys

from py.test import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
notice that for really long env names, conda is putting in /usr/bin/env for the prefix
all of the env names when running conda build exceeed ~128 characters
and hence they get set to /usr/bin/env
and thus on mac they trigger SIP behavior
Matthew R. Becker
@beckermr
One way around this would be to map every env to a short random hash.
But I think this is an edge case that doesn’t need to be handled
Douglas Lowe
@douglowe
@mingwandroid thanks for the link. In this case I don't need the gui package to depend on the cli one (though this might change for the linux or windows packages).
Jonathan J. Helmus
@jjhelmus
https://github.com/conda/conda/blob/4.7.12/conda/core/portability.py#L155-L175 is replacing the long shebang lines for portability. Most platforms/distribution have some limit on the length of the shebang and using #!/usr/bin/env python in those cases. Adding a configuration option to override this rewrite would be reasonable.
Matthew R. Becker
@beckermr
yup I found the old github issue where this was discussed
I like the idea of a config override
OTOH, I put in a custom shim in my build that solved this particular issue
Matthew R. Becker
@beckermr
@jjhelmus there does seem to be some interest in a config option to turn off the replacement
should I make an issue?
Matthew R. Becker
@beckermr
I made an issue on this
deepali-c
@deepali-c

I see below error while building a recipe:

Fixing permissions execute(sys.argv[1:]) File "<mypath>/lib/python2.7/site-packages/conda_build/cli/main_build.py", line 430, in execute verify=args.verify, variants=args.variants) File "<mypath>/lib/python2.7/site-packages/conda_build/api.py", line 201, in build notest=notest, need_source_download=need_source_download, variants=variants) File "<mypath>/lib/python2.7/site-packages/conda_build/build.py", line 2275, in build_tree notest=notest, File "<mypath>/lib/python2.7/site-packages/conda_build/build.py", line 1654, in build newly_built_packages = bundlers[output_d.get('type', 'conda')](output_d, m, env, stats) File "<mypath>/lib/python2.7/site-packages/conda_build/build.py", line 972, in bundle_conda output['checksums'] = create_info_files(metadata, files, prefix=metadata.config.host_prefix) File "<mypath>/lib/python2.7/site-packages/conda_build/build.py", line 665, in create_info_files copy_recipe(m) File "<mypath>/lib/python2.7/site-packages/conda_build/build.py", line 301, in copy_recipe _copy_output_recipe(m, recipe_dir) File "<mypath>/lib/python2.7/site-packages/conda_build/build.py", line 270, in _copy_output_recipe _copy_top_level_recipe(src_dir, m.config, dest_dir, 'parent') File "<mypath>/lib/python2.7/site-packages/conda_build/build.py", line 264, in _copy_top_level_recipe locking=config.locking, clobber=True) File "<mypath>/lib/python2.7/site-packages/conda_build/utils.py", line 502, in copy_into _copy_with_shell_fallback(src, dst_fn) File "<mypath>/lib/python2.7/site-packages/conda_build/utils.py", line 435, in _copy_with_shell_fallback raise OSError("Failed to copy {} to {}. Error was: {}".format(src, dst, e)) OSError: Failed to copy <path>/meta.yaml to <envpath>/info/recipe/parent/meta.yaml. Error was: Command 'cp -a <path>/meta.yaml <envpath>/info/recipe/parent/meta.yaml' returned non-zero exit status 1
Any hints on how to get to the cause of this? I have just shortened the paths here.

Duncan Macleod
@duncanmmacleod
should conda-build be adding ${CONDA_PREFIX}/sbin to path, or is there some other recommendation?
Duncan Macleod
@duncanmmacleod
sorry, this is more of a conda question, not a conda build question
deepali-c
@deepali-c
so shall I post in the conda gitter
Duncan Macleod
@duncanmmacleod
@deepali-c, my question was totally unrelated to yours, sorry for any confusion
Nehal J Wani
@nehaljwani
@duncanmmacleod you should build the package in a manner such that all binaries end up in bin. Any particular reason why you want to use sbin ?
Duncan Macleod
@duncanmmacleod
@nehaljwani, I am just building an existing package that includes daemon scripts that naturally belong in sbin
I understand that sbin is mainly for administrator control, I was just wondering if ‘don’t use sbin’ is a conda policy, or just nobody has tried it until now
Nehal J Wani
@nehaljwani
Conda doesn't add sbin to PATH on activation
Duncan Macleod
@duncanmmacleod
Sorry, my original question was badly worded, I am wondering if that very fact is a choice that was long-considered or just an oversight
Nehal J Wani
@nehaljwani

My understanding is that the sbin folder on Linux systems is for segregating binaries meant to be run by privileged users. (generally speaking, sysadmins)

But conda is a generic package manager and doesn't make such distinctions, so the need for segregating binaries doesn't make much sense here.

Anthony Scopatz
@scopatz
Hi All, is missing_dso_whitelist undocumented?
Seems so
Anthony Scopatz
@scopatz
Yeah that is what I was looking for
or at, rather
Nehal J Wani
@nehaljwani
It is definitely supported though. Are you repackaging something?
Anthony Scopatz
@scopatz
yeah, a haskel package