Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Pierre Augier
@paugier
In this case, there is a tiny Makefile, so (with make installed), just run make clean, make and then make bench.
To reproduce similar results shown in the webpage, I guess it would be even better to run pip install -U pythran (or the equivalent command with conda) and to use the clang compiler.
A standard way to choose the compiler for Pythran is to use the configuration file ~\.pythranrc (https://pythran.readthedocs.io/en/latest/MANUAL.html#customizing-your-pythranrc).
Pierre Augier
@paugier
For these benchmarks, I use
[pythran]
complex_hook = True

[compiler]
CXX=clang++-6.0
CC=clang-6.0
blas=openblas

It's puzzling.

We have to add warnings to explain this kind of unexpected behavior!

luk-f-a
@luk-f-a
the environment had both pythran and numba installed. why wasn't numba accelerated? When I removed the transonic decorator and put the numba njit, I got a speed up. why didn't the boost decorator call numba's jit?
I have some more feedback but I don't want to use numba's channel. do you have a channel?
Pierre Augier
@paugier
why didn't the boost decorator call numba's jit?
It is the expected behavior for the boost decorator which is for AOT compilation. If the file is not compiled, it falls back on Python-Numpy, which is not silly for AOT compilers. I agree that there should be a warning in this case. We'll add that.
do you have a channel?
We don't use Gitter anymore but actually yes, we have one channel : https://gitter.im/fluiddyn/dev
why didn't the boost decorator call numba's jit?
Pierre Augier
@paugier
There are of course many things that are not yet great, especially for the Numba backend. Pull requests are of course very welcome! :slight_smile:
For example, it would be really reasonable to have a fastmath argument for Transonic decorators, so that it would be possible to use fastmath=True for the Numba backend. It would be very simple to implement, but it is not our priority in terms of development. Again, PR welcome.
valorcurse
@valorcurse
@stuartarchibald I've been getting either one of these two errors: https://paste.ee/p/OEept
I have a list of 100 matrices, which start at (28, 28)
But at some point when one becomes bigger, it gives the that error
stuartarchibald
@stuartarchibald
@paugier Numba does AOT. I wonder what happens with Transonic AOT when you want to ship your code to another machine? What if the instruction set is different?
Pierre Augier
@paugier
With Cython and Pythran, we just ship the binaries (compiled without -march=native) and it works fine (same as all packages using Cython today). With Numba we don't use yet AOT compilation. Another thing to be improved in the Numba backend.
stuartarchibald
@stuartarchibald
but -march=native means you are just compiling to whatever chip you have on the build machine. If you have e.g. a SkyLake you might compile in AVX512 instructions which then won't run on e.g. a Core2 or a Nehalem.
And this is the problem with AOT compilation without a fall-back JIT. IIRC Numba's "vision" for it's AOT support is to be able to compile a de-tuned binary e.g. target "nocona" chips, so the binary will work pretty much anywhere, but then if at run time a better chip is discovered and Numba is installed the AOT dispatch could JIT compile a version optimal for the chip on which it's executing.
Pierre Augier
@paugier
@stuartarchibald Yes, I understand. It is a nice feature of JIT compilation. Note however, that for C and C++, there are ways to compile for multi-architectures. I think scikit-image does that for their Cython extensions in their wheels (which are very fast so I guess that they use advanced instructions on machines where they are available).
stuartarchibald
@stuartarchibald
Oh for sure there are ways to do this for traditionally compiled languages, compile for numerous architectures and then cpuid + dlopen at run time is common.
luk-f-a
@luk-f-a
@paugier , if boostis AOT, and is not calling numba's jit then is it trying to call numba's @cc.export?
stuartarchibald
@stuartarchibald
@valorcurse what's the call site that's generating that message look like? Numba seems to think you have an object dtype, which is probably more like "no idea what this is"-dtype, in this case.
francoislauger
@francoislauger
@stuartarchibald with @jitclassis there a way to pass some option of @jit. I'm thinking cached=True and something to enable parrallel in a method?
stuartarchibald
@stuartarchibald
@francoislauger at present there is not, I think there's an open issue about this but can't seem to find it.
Gaƫl Donval
@gdonval_gitlab
It's there I believe: numba/numba#3417
francoislauger
@francoislauger
@gdonval_gitlab yeah it's seems to be the one for parrallel. For cahched=Truethere this numba/numba#1681 that is related and this closed issue : numba/numba#1964
francoislauger
@francoislauger
@stuartarchibald any chance that there change on that before a refactoring of the `@jitclass. And on the refactoring any idea on timeline (here : https://numba.pydata.org/numba-doc/dev/developer/roadmap.html it's mention 2019H2). Thank's
Pierre Augier
@paugier
@luk-f-a No, currently the Numba backend is very simple. It only writes a Python file with all necessary functions (by the way, one does not need to add @boost to the functions used by the boosted functions), add @njit to all functions and use the corresponding jitted function for the boosted functions! At the end, you exactly get the Numba function so there is no overhead.
Of course, it would be simple to improve that to be able to use @cc.export.
Pierre Augier
@paugier
Note that what we need today for Transonic is more improvement of the Cython backend, so that Transonic would be able to recreate nearly the same Cython files as the ones found in packages like scikit-image or scikit-learn, so that they could use Transonic with Cython (a safe and not too big transition).
Then, it would be easy to use other accelerators (like Numba or Pythran) for some modules/functions.
stuartarchibald
@stuartarchibald
@francoislauger I don't think that's changed for jitclass, a work around might be to simply call a module level function from the jitclass method, where the module level function it jitted with whatever options you want.
luk-f-a
@luk-f-a
@paugier , I don't understand. is boost supposed to call njitor not? In my test it didn't call it, is that a bug or the intended behaviour? I think it really needs a warning about whether the compilation actually took place.
Pierre Augier
@paugier
boost is supposed to call njit if the (Transonic) compilation has been done. And yes, once again, "I [also] think it really needs a warning about whether the compilation actually took place" :slight_smile:
Benjamin Zaitlen
@quasiben
Is there a timeline for when numba 0.46 or 0.47 will be released on conda-forge ?
stuartarchibald
@stuartarchibald
0.46.0 was only released last week, perhaps open a ticket on the conda-forge numba feedstock for it ?
Benjamin Zaitlen
@quasiben
Good suggestion. Thanks!
stuartarchibald
@stuartarchibald
0.47.0 is a fair way away (we're working on it now)
Perhaps CC @jakirkham on the ticket
Benjamin Zaitlen
@quasiben
I found an automatically generated PR: conda-forge/numba-feedstock#32 It's currently failing. Sorry for the noise here
stuartarchibald
@stuartarchibald
no worries
I expect it's failing as llvmlite=0.30.0 needs to be built first
Benjamin Zaitlen
@quasiben
yup yup
jetxeberria
@jetxeberria
In https://numba.pydata.org/numba-doc/dev/user/jitclass.html it's presented the jitclass decorator. But in limitations says that: Support for jitclasses are available on CPU only. (Note: Support for GPU devices is planned for a future release.)
The way I'm using Numba is by defining normal CPU classes/functions and adding @jit decorator for those functions that I want to be executed on GPU. Why would I like to decorate the classes defined for CPU?
jakirkham
@jakirkham
@stuartarchibald, there's also a PR for llvmlite ( conda-forge/llvmlite-feedstock#21 ), but it is failing some tests atm. If you have a few mins to take a look and suggest next steps, that would be very helpful :slight_smile:
Scratch that this may be the consequence of some recent restructuring of LLVM packaging in conda-forge. Will reach out to some people there to figure out next steps
jetxeberria
@jetxeberria

In https://numba.pydata.org/numba-doc/dev/user/jitclass.html it's presented the jitclass decorator. But in limitations says that: Support for jitclasses are available on CPU only. (Note: Support for GPU devices is planned for a future release.)
The way I'm using Numba is by defining normal CPU classes/functions and adding @jit decorator for those functions that I want to be executed on GPU. Why would I like to decorate the classes defined for CPU?

excuse me:
@cuda.jit, not @jit
(it doesn't let me edit the comment)