Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 02 19:52
    rchord synchronize #4338
  • Dec 02 19:44
    rchord synchronize #4338
  • Dec 02 19:42
    rchord synchronize #4338
  • Dec 02 19:33
    rchord synchronize #4338
  • Dec 02 19:28
    rchord synchronize #4338
  • Dec 02 19:21
    rchord synchronize #4338
  • Dec 02 19:09

    kenodegard on master

    Fix keeping old work when workd… (compare)

  • Dec 02 19:09
    kenodegard closed #4341
  • Dec 02 19:08
    kenodegard commented #4341
  • Dec 02 18:32
    rchord synchronize #4338
  • Dec 02 18:20
    rchord synchronize #4338
  • Dec 02 18:19
    rchord synchronize #4338
  • Dec 02 18:17
    rchord synchronize #4338
  • Dec 02 17:36
    Lnaden commented #4341
  • Dec 02 16:56
    brey commented #2438
  • Dec 02 15:16
    anaconda-issue-bot labeled #4341
  • Dec 02 15:16
    Lnaden opened #4341
  • Dec 02 12:27
    jezdez review_requested #4340
  • Dec 01 22:32
    rchord synchronize #4338
  • Dec 01 22:31
    rchord synchronize #4338
Lucas Hosseini
@beauby
An other option being having a separate channel
Ray Donnelly
@mingwandroid
You can use 'labels' on anaconda.org, it's just like nested channels really
But you probably do want different filenames. You can see how we address that in the AD python-recipe here: https://github.com/AnacondaRecipes/python-feedstock/blob/master/recipe/meta.yaml
follow PY_INTERP_DEBUG
Lucas Hosseini
@beauby
@mingwandroid Thanks, I'll look into that! Would I be correct to assume that conda install foo -c mychannel would install only packages tagged with "main", and conda install foo -c mychannel/labels/nightly would install only packages tagged with "nightly"?
Ray Donnelly
@mingwandroid
Yeah, I think so. I don't use labels myself though, probably I should
Lucas Hosseini
@beauby
Pretty cool, if this works the way I think, it's exactly what I'm looking for. Thanks for pointing it out
Ray Donnelly
@mingwandroid
No problem!
Lucas Hosseini
@beauby
Yeah I'm not 100% sure what the role of the build string is – it has to be unique per version, but does it serve a technical purpose?
Cheng H. Lee
@chenghlee
It serves to distinguish multiple builds of the same version of a package. These may be due to changes in recipes to fix build problems, to variants of the same base recipes (e.g., choosing between py36, py37, or py38 builds of numpy), or to whatever other purpose the package maintainer chooses.
Lucas Hosseini
@beauby
@chenghlee Right, but the build string itself is not used to decide which package is "more current", right?
Ray Donnelly
@mingwandroid
There's a longstanding bug in conda here that build/string also helps to avoid. The package cache can have conflicts between e.g. win-32 and win-64 (though we've mitigated that some in recent conda-builds).
Cheng H. Lee
@chenghlee
All other things being equal, the solver considers build 1 more current than build 0.
Lucas Hosseini
@beauby
@chenghlee Right, but by "build 1" you mean the build number is "1", right?
Ray Donnelly
@mingwandroid
If the win-32 package has the same file name as the win-64 package (for example, but it can also happen outside of different arches) then conda will happily unpack one on top of the other and that breaks things sometimes.#
Lucas Hosseini
@beauby
Btw @chenghlee Do you have a link to the bug tracker issue you mentioned the other day (regarding the bug in conda cache)?
2 replies
@mingwandroid I see, so keeping it unique probably still matters quite a bit, but it does not have to be lexicographically increasing between versions/builds
Ray Donnelly
@mingwandroid
If I were doing e.g. a release and debug python package I'd keep the build numbers the same when built from the same source and add -dbg or something like that to the build string to ensure no package cache conflicts, then labels also to facilitate selection.
the package is still then called "python" for both so the solver will be ok.
Lucas Hosseini
@beauby
Right, in my scheme I do add nightly as well to the nightly builds
Cheng H. Lee
@chenghlee
Looking through the solver code, the build string is used as the tie-breaker of last resort; i.e., if two packages somehow end up with the same channel priority, version, build number, sub-directory (noarch vs platform-specific), and build timestamps.
Lucas Hosseini
@beauby
I see, but then it's quite surprising to use a hash in the build string by default
(although I guess it is mainly a matter of the solver being deterministic, rather than something intended to be actually used)
Cheng H. Lee
@chenghlee
I imagine you'd have to be in a really, really odd state where everything other than build string is tied.
The hash in the build string serves a different purpose; it's meant to distinguish different variants of a package. E.g., for numpy, the hash encodes information about what BLAS library the package was linked against (MKL vs OpenBLAS).
Lucas Hosseini
@beauby
Yup, I understand that part, I was just mentioning that the hash won't yield lexicographically increasing strings
Cheng H. Lee
@chenghlee
Right.
But the lexicographical ordering of the build strings is likely irrelevant there, as the package selection would be determined by the BLAS specification of the environment.
Lucas Hosseini
@beauby
True
Krishna Bhogaonker
@00krishna

Hey folks, I was hoping someone could point me in the right direction with this one. I am building a package called nyuki using conda build 3.20.3. The error is happening after the build process as conda build is uploading to Anaconda.org, it seems.

Uploading nyuki-0.0.3-py_0.tar.bz2 to anaconda.org
Traceback (most recent call last):
  File "/media/krishnab/lakshmi/anaconda3/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 474, in main
    execute(sys.argv[1:])
  File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 465, in execute
    verify=args.verify, variants=args.variants)
  File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/site-packages/conda_build/api.py", line 210, in build
    notest=notest, variants=variants)
  File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 3155, in build_tree
    handle_anaconda_upload(tarballs, config=config)
  File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 3236, in handle_anaconda_upload
    subprocess.call(cmd + [package])
  File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/subprocess.py", line 339, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/subprocess.py", line 1465, in _execute_child
    executable = os.fsencode(executable)
  File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/os.py", line 812, in fsencode
    filename = fspath(filename)  # Does type-checking of `filename`.
TypeError: expected str, bytes or os.PathLike object, not list

I am logged in to Anaconda.org through the terminal, so should be no issue there. The tests all seem to pass in the build process. The only strange errors are those ones about ERROR:conda_build.utils:Failed to rename work directory despite sleeping and retrying. This is some Windows file locking mis-bahaviour. Note that I am building on Ubuntu Linux 18.04.

The meta.yaml file is as follows.

{% set name = "nyuki" %}
{% set version = "0.0.3" %}

package:
  name: {{ name|lower }}
  version: {{ version }}

source:
  url: https://github.com/00krishna-tools/nyuki/archive/v{{ version }}.tar.gz
  sha256: f98b3d43bdc89fe4d10dfe5da59d9e961013329df7d799396195b31cf15f84ed

build:
  noarch: python
  number: 0
  entry_points:
    - nyuki = nyuki.nyuki:nyuki

requirements:
  host:
    - python >3.5
    - pip
    - numpy >=1.18.5
    - pandas
    - rasterio
    - click >=7
    - tqdm
    - geopandas
    - pillow
    - libwebp
    - libtiff >4.0
  run:
    - python >3.5
    - numpy >=1.18.5
    - pandas
    - rasterio
    - click >=7
    - tqdm
    - geopandas
    - pillow
    - libwebp
    - libtiff >4.0

test:
  imports:
    - nyuki
  commands:
    - test -f ${PREFIX}/lib/libtiff.a  # [not win]
    - test -f ${PREFIX}/lib/libtiffxx.a  # [not win]
    - test -f ${PREFIX}/lib/libtiff{{ SHLIB_EXT }}  # [not win]
    - test -f ${PREFIX}/lib/libtiffxx{{ SHLIB_EXT }}  # [not win]
    - if not exist %PREFIX%\\Library\\bin\\tiff.dll exit 1  # [win]
    - if not exist %PREFIX%\\Library\\bin\\tiffxx.dll exit 1  # [win]
    - if not exist %PREFIX%\\Library\\bin\\libtiff.dll exit 1  # [win]
    - if not exist %PREFIX%\\Library\\bin\\libtiffxx.dll exit 1  # [win]

about:
  home: https://github.com/00krishna-tools/nyuki
  license: MIT
  license_family: MIT
  license_file: LICENSE.txt
  summary: "A geospatial library for humans. "

  description: "Nyuki: a geospatial library for humans"
  doc_url: https://nyuki-a-geospatial-toolkit-for-humans.readthedocs.io/en/latest/
  dev_url: https://github.com/00krishna-tools/nyuki

extra:
  recipe-maintainers:
    - 00krishna

Any help is appreciated.

Fabien Celier
@fabiencelier

Hi guys,
I want to build 2 packages:

  • one OS dependant with binaries
  • one noarch with only python calling the binaries

In the python package how can I know where the binaries of the first package are located ?

Ray Donnelly
@mingwandroid
Next PR to conda-build will be #4096 for those who like round numbers.
Gabriel Reis
@gabrielcnr
@mingwandroid if I have a noarch python package, but I want to bring an extra run dependency on Windows only, I can't use selectors for # [win] if I'm building on Linux? Is there a way to achieve that?
Ray Donnelly
@mingwandroid
No, there is not. conda doesn't have separate dependency metadata per-platform for a noarch python package.
That works against the idea frankly
You could have a fake package on the other platforms, but that's kind of horrible. Still ...
There's worse
Gabriel Reis
@gabrielcnr
I see the point... thanks
Faustin Carter
@FaustinCarter
I'm having an issue where specifying a channel_alias in condarc appears to be reordering channel priority during solving of specs.
So the solver creates a list of packages from different channels using the wrong priority order, and then the build process crashes when it tries to create an environment with that spec, because it runs afoul of the strict channel priority.
This is definitely a bug
I'm wondering if anyone has run into anything like this before.
To be clear, this issue is specific to conda-build. conda create collects the right set of specs regardless of channel_alias being specified.
Is there a reason conda-build doesn't use the same solver as conda create?
Faustin Carter
@FaustinCarter
I put some more details in this bug report
Isuru Fernando
@isuruf
@mingwandroid, when cross compiling, the various variant files are read first with build_platform specific selectors even if target_platform was set in a previous variant file. Any workarounds?
Ray Donnelly
@mingwandroid
@isuruf, hmm, no I guess someone will have to spend some time fixing it though.
If you file a bug or a PR with a xfail test to conda-build then it'd help get it on the radar.
1 reply