@mingwandroid I have not tried using zip_keys yet, but from how I understand the documentations, zip_keys would cause only specific pairings of versions, e.g. in the example above, one boost version with one python version. And it would also not work with 4 boost versions, but only 3 python versions, as the lists would need to be identical lengths, or am I missunderstanding something? Thanks a lot for the help regardless!
conda-build 3.20.0 has been tagged, https://github.com/conda/conda-build/releases/tag/3.20.0. Expect packages in the
conda-canary channel tomorrow. This release supports Python 2.7 but support for running conda-build in a Python 2.7 or <3.6 environments will be dropped in a 4.0.0 release in the near future. See conda/conda-build#4024 for more discussion.
Hey all, can anyone help shed some light on whether channeldata.json is actually used for anything? The only mentions of it in the docs are here: https://docs.conda.io/projects/conda-build/en/latest/concepts/generating-index.html?highlight=channeldata.json#parts-of-a-channel, but no mention of actual utility.
I'm wondering what negative downstream consequences there are if that file does not exist in a channel (but all the repodata and current_repodata files are in place).
@mingwandroid, what does
target in https://github.com/conda/conda-build/blob/7dac1cbad5195f719bde1eaeb0d5795186dc0eb0/tests/test-recipes/metadata/_cross_libgreeting/meta.yaml do?
nvm, found it
Blog posts, docs, etc
I am not aware of docs but https://github.com/conda/conda-build/pull/4026/commits/6169878ef57c9f6b23e79e994bf1b05a7c300fb4 and conda/conda-build#4028 are the code that implements it
26 replies
or does it only pull the compressed ones?
Hi @scopatz , conda-build 3.20.x have support for the jinja2 function "ccache(method)" where method can be symlinks, env_vars or native.
Defaults has fairly robust ccache exes, but ccache is moving fast, esp. on Windows support so it might be worth making sure that we're up to date with our ccaches
To see that it's working, you want to do e.g. ls -lrt ~/.ccache during builds and see that the order changes around lots! or run
ccache -s to see some stats
probably I want to clear the stats at the start of a build and print them at the end of it, PRs and any testing/fixing very welcome!
Oops, missed Jonathan's comment. I'd like people to kick this feature around a bit for now, I don't think it's quite ready yet.
I have been trying to use it on windows, where everything is weird
I'll try to kick the tyres a bit later, for sure it'll work better on the other OSes and it is still a bit of an experiment. Isuru didn't want it integrated into conda-build really but I want conda-build's stats reporting to tie into it via calling
ccache -s. It can then report if using ccache actually did anything useful for your build or not so you can investigate that (maybe try the other methods, or modify bld.bat/build.sh etc)
I guess I'm probably more interested in how
tarcheck.py works and what the inputs are
It seems like it's meant to be used with tox for installed packages, but there's significant overlapping functionality with conda build in that case...
@mattchan-tencent I have a preference for tests (provided they are not too expensive, size-wise and time-to-run-wise) to be added to the package itself via conda-build's test metadata. Sometimes this is easy, but sometimes not. Getting the tests into the package itself allows that package to be tested later in different environments which will help greatly with tracking down unexpected behaviours (for example when you update some of the deps)
It might be an idea to split the package into package and package-tests too. Then make the test for package depend on package-tests. That way people who do not care about the tests do not suffer the costs. This would also be a good way to handle tests that are larger (and maybe take longer) .. I would love to find the time to implement multi-level testing too or perhaps giving tests some metadata against which you (or tooling) can filter them (needs_network, needs_super_user, slow_test, heavy_test, etc).
Why does conda-build print warning messages twice? For eg:
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
sometimes 3 times
Multiple meta.yaml files found. The meta.yaml file in the base directory will be used.
WARNING:conda_build.utils:Multiple meta.yaml files found. The meta.yaml file in the base directory will be used.
WARNING conda_build.utils:find_recipe(1219): Multiple meta.yaml files found. The meta.yaml file in the base directory will be used.
"UserWarning: The environment variable 'CONDA_FORGE' is being passed through with value 'yes'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time."
reasons: 1. these vars are passed specifically to modify build behavior (as I use them at least!), and have no bearing on tests ..
- If we do want this warning, well, conda-build knows if we're using --no-test or not, and we could at least keep quiet when that isn't in play.
C:\opt\conda-win-32\lib\site-packages\conda_build\environ.py:447: UserWarning: The environment variable 'PY_INTERP_LINKAGE_NATURE' is being passed through with value ''. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
warnings.warn(
C:\opt\conda-win-32\lib\site-packages\conda_build\environ.py:447: UserWarning: The environment variable 'PY_INTERP_DEBUG' is being passed through with value 'no'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
warnings.warn(
C:\opt\conda-win-32\lib\site-packages\conda_build\environ.py:447: UserWarning: The environment variable 'OPENSSL_DIR' is being passed through with value '%PREFIX%\Library'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
warnings.warn(
C:\opt\conda-win-32\lib\site-packages\conda_build\environ.py:447: UserWarning: The environment variable 'SQLITE3_DIR' is being passed through with value '%PREFIX%\Library'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
warnings.warn(
C:\opt\conda-win-32\lib\site-packages\conda_build\environ.py:447: UserWarning: The environment variable 'CONDA_FORGE' is being passed through with value 'yes'. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
warnings.warn(
INFO:conda_build.metadata:Attempting to finalize metadata for python
two ones, no two, oh well.
We patch stuff to be exe-relative