I guess I'm probably more interested in how
tarcheck.py works and what the inputs are
It seems like it's meant to be used with tox for installed packages, but there's significant overlapping functionality with conda build in that case...
@mattchan-tencent I have a preference for tests (provided they are not too expensive, size-wise and time-to-run-wise) to be added to the package itself via conda-build's test metadata. Sometimes this is easy, but sometimes not. Getting the tests into the package itself allows that package to be tested later in different environments which will help greatly with tracking down unexpected behaviours (for example when you update some of the deps)
It might be an idea to split the package into package and package-tests too. Then make the test for package depend on package-tests. That way people who do not care about the tests do not suffer the costs. This would also be a good way to handle tests that are larger (and maybe take longer) .. I would love to find the time to implement multi-level testing too or perhaps giving tests some metadata against which you (or tooling) can filter them (needs_network, needs_super_user, slow_test, heavy_test, etc).
Why does conda-build print warning messages twice? For eg:
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
sometimes 3 times
Multiple meta.yaml files found. The meta.yaml file in the base directory will be used.
WARNING:conda_build.utils:Multiple meta.yaml files found. The meta.yaml file in the base directory will be used.
WARNING conda_build.utils:find_recipe(1219): Multiple meta.yaml files found. The meta.yaml file in the base directory will be used.
"UserWarning: The environment variable 'CONDA_FORGE' is being passed through with value 'yes'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time."
reasons: 1. these vars are passed specifically to modify build behavior (as I use them at least!), and have no bearing on tests ..
- If we do want this warning, well, conda-build knows if we're using --no-test or not, and we could at least keep quiet when that isn't in play.
C:\opt\conda-win-32\lib\site-packages\conda_build\environ.py:447: UserWarning: The environment variable 'PY_INTERP_LINKAGE_NATURE' is being passed through with value ''. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
warnings.warn(
C:\opt\conda-win-32\lib\site-packages\conda_build\environ.py:447: UserWarning: The environment variable 'PY_INTERP_DEBUG' is being passed through with value 'no'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
warnings.warn(
C:\opt\conda-win-32\lib\site-packages\conda_build\environ.py:447: UserWarning: The environment variable 'OPENSSL_DIR' is being passed through with value '%PREFIX%\Library'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
warnings.warn(
C:\opt\conda-win-32\lib\site-packages\conda_build\environ.py:447: UserWarning: The environment variable 'SQLITE3_DIR' is being passed through with value '%PREFIX%\Library'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
warnings.warn(
C:\opt\conda-win-32\lib\site-packages\conda_build\environ.py:447: UserWarning: The environment variable 'CONDA_FORGE' is being passed through with value 'yes'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
warnings.warn(
INFO:conda_build.metadata:Attempting to finalize metadata for python
two ones, no two, oh well.
Hi all, hoping someone can help me, struggling a bit with conda-build. I am trying to package a closed-source c++ library which is distirbuted as tarball. I want to accomplish a (seemingly) simple task: produce a conda package from a the headers and static libraries from this tarball. I have the following meta.yaml file:
package:
name: my-package
version: 1.0
source:
url: http://path/to/tar.gz
sha256: here
build:
number: 0
outputs:
- name: my-package
script: dev-files.pyThe dev-files.py file script selects the header files from the current (ie work) directory and copies them into the corresponding $PREFIX directory (I assume this is correct - it's not documented). The copy task seems to work but then the packaging job fails with this error:
Failed to rename host env directory despite sleeping and retrying. This is some Windows file locking mis-bahaviour.Which is interesting because I'm not on Windows. Any help appreciated.
16 replies
With reference to the above issue I now believe this is an entirely spurious error message. From reading the code in question, it will always produce this error message even when the underlying move operation is entirely successful. I'll raise an issue/PR when I get some time. Thanks @nehaljwani for your help.
Hey folks. I am getting a strange build error when using conda build. It has not happened before and googling did not yield any results. I built my package, but when uploading to anaconda.org I get this error.
Uploading nyuki-0.0.3-py_0.tar.bz2 to anaconda.org
Traceback (most recent call last):
File "/media/krishnab/lakshmi/anaconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 474, in main
execute(sys.argv[1:])
File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 465, in execute
verify=args.verify, variants=args.variants)
File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/site-packages/conda_build/api.py", line 210, in build
notest=notest, variants=variants)
File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 3155, in build_tree
handle_anaconda_upload(tarballs, config=config)
File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 3236, in handle_anaconda_upload
subprocess.call(cmd + [package])
File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/subprocess.py", line 339, in call
with Popen(*popenargs, **kwargs) as p:
File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/subprocess.py", line 1465, in _execute_child
executable = os.fsencode(executable)
File "/media/krishnab/lakshmi/anaconda3/lib/python3.7/os.py", line 812, in fsencode
filename = fspath(filename) # Does type-checking of `filename`.
TypeError: expected str, bytes or os.PathLike object, not listSo it is execting a string or os.PathLike object, and not a list. I am not sure where this error could come coming from. Does it seem familiar to anyone. The only other thing I found in the entire build stdout is a reference to that ERROR:conda_build.utils:Failed to rename work directory despite sleeping and retrying. This is some Windows file locking mis-bahaviour. errro that someone mentioned yesterday or a few days ago.
I can share the
meta.yaml file if it would be useful. Seems like the build process completed, but this message arises when uploading to anaconda.org.
Any general advice for distributing nightly packages through conda? I'm leaning towards having a separate package named
foo-nightly with versions 1.0.0-nightly-AAAAMMDD, but maybe there is a better solution? Ideally, I'd like the -nightly package to be incompatible with the real releases, to avoid conflicts
An other option being having a separate channel
But you probably do want different filenames. You can see how we address that in the AD python-recipe here: https://github.com/AnacondaRecipes/python-feedstock/blob/master/recipe/meta.yaml
follow PY_INTERP_DEBUG
Check the build/string stuff too if you didn't already: https://github.com/AnacondaRecipes/python-feedstock/blob/master/recipe/meta.yaml#L120-L124
It serves to distinguish multiple builds of the same version of a package. These may be due to changes in recipes to fix build problems, to variants of the same base recipes (e.g., choosing between py36, py37, or py38 builds of numpy), or to whatever other purpose the package maintainer chooses.