by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
AidanReddy
@AidanReddy
(base) Aidans-MacBook-Pro:Calculations aidanreddy$ python3 energyTransferEfficiency
Traceback (most recent call last):
File "energyTransferEfficiency", line 6, in <module>
import numba
ModuleNotFoundError: No module named 'numba'
stuartarchibald
@stuartarchibald
yeah, it's probably because the python running your script isn't the one which has numba installed
Here you find instructions to ensure that you are using the correct python installation (i.e. your anaconda installation)
(Which will probably work fine if you already installed numba using conda)
Step 7 is probably the critical one for you
AidanReddy
@AidanReddy
okay, thanks a lot. I just tried running the script specifying 'python3.7' as opposed to 'python3' and it seemed to work.
Hannes Pahl
@HPLegion
Great :) I still recommend you set your terminal to use the anaconda installation by default (since that is what you want in like 99% of the cases I can imagine) The system python is mostly meant for OS built ins and things like widgets etc (if I am not mistaken)
AidanReddy
@AidanReddy
@HPLegion If I delete the copy of the numba folder in the folder than my script is in, I still get the error message ModuleNotFoundError: No module named 'numba'
stuartarchibald
@stuartarchibald
have you created a conda environment with numba and python in it and then activated that environment?
AidanReddy
@AidanReddy
yes, I just tried and it did not work
stuartarchibald
@stuartarchibald
what does conda list say ?
AidanReddy
@AidanReddy

packages in environment at /opt/anaconda3:

#

Name Version Build Channel

_ipyw_jlab_nb_ext_conf 0.1.0 py38_0
amqp 2.6.1 pypi_0 pypi
anaconda-clean 1.1.0 py38_1
anaconda-client 1.7.2 py38_0
anaconda-navigator 1.9.12 py38_0
appnope 0.1.0 py38_1001
attrs 19.3.0 py_0
backcall 0.2.0 py_0
backports 1.0 py_2
backports.functools_lru_cache 1.6.1 py_0
backports.tempfile 1.0 py_1
backports.weakref 1.0.post1 py_1
beautifulsoup4 4.9.1 py38_0
billiard 3.6.3.0 pypi_0 pypi
blas 1.0 mkl
bleach 3.1.5 py_0
brotlipy 0.7.0 py38haf1e3a3_1000
bzip2 1.0.8 h1de35cc_0
ca-certificates 2020.6.24 0
celery 4.4.7 pypi_0 pypi
certifi 2020.6.20 py38_0
cffi 1.14.0 py38hc512035_1
chardet 3.0.4 py38_1003
click 7.1.2 py_0
clyent 1.2.2 py38_1
conda 4.8.3 py38_0
conda-build 3.18.11 py38_0
conda-env 2.6.0 1
conda-package-handling 1.6.1 py38h1de35cc_0
conda-verify 3.4.2 py_1
cryptography 2.9.2 py38ha12b0ac_0
cudatoolkit 9.0 h41a26b3_0
dbus 1.13.16 h18a8e69_0
decorator 4.4.2 py_0
defusedxml 0.6.0 py_0
entrypoints 0.3 py38_0
expat 2.2.9 hb1e8313_2
filelock 3.0.12 py_0
freetype 2.10.2 ha233b18_0
future 0.18.2 py38_1
gettext 0.19.8.1 hb0f4f8b_2
glib 2.65.0 hc5f4afa_0
glob2 0.7 py_0
icu 58.2 h0a44026_3
idna 2.10 py_0
importlib-metadata 1.7.0 py38_0
importlib_metadata 1.7.0 0
intel-openmp 2019.4 233
ipykernel 5.3.4 py38h5ca1d4c_0
ipython 7.16.1 py38h5ca1d4c_0
ipython_genutils 0.2.0 py38_0
ipywidgets 7.5.1 py_0
jedi 0.17.2 py38_0
jinja2 2.11.2 py_0
jpeg 9b he5867d9_2
json5 0.9.5 py_0
jsonschema 3.2.0 py38_1
jupyter_client 6.1.6 py_0
jupyter_core 4.6.3 py38_0
jupyterlab 2.1.5 py_0
jupyterlab_server 1.2.0 py_0
kombu 4.6.11 pypi_0 pypi
lcms2 2.11 h92f6f08_0
libarchive 3.4.2 haa3ed63_0
libcxx 10.0.0 1
libedit 3.1.20191231 h1de3

stuartarchibald
@stuartarchibald
Perhaps use a pastebin :)
or put it into a threaded comment
Seems like it gets cut off at l in the alphabet, it also looks like that's possibly your base environment and not on that's been created and activated
AidanReddy
@AidanReddy
sorry, I'm not quite sure how to use paste bins, but let me know if that doesn't work
do you want me to run conda list within the conda environment with numba and python?
stuartarchibald
@stuartarchibald
you;ve installed numba via pip according to that?
and seemingly into the base anaconda environment
AidanReddy
@AidanReddy
yeah, I initially only had numba installed via conda, and then tried installing it via pip to see if that would solve my problem.
stuartarchibald
@stuartarchibald
I think that may well have compounded them unfortunately.
If you can get a new Anaconda installation, and then just create a new environment with Python and Numba and then go from there, I'd do that.
AidanReddy
@AidanReddy
okay, thanks for the help.
stuartarchibald
@stuartarchibald
no problem, good luck!
voegtlel
@voegtlel

Hey there, I'm currently using numba as recommended here (my code is much more complex, this is the simplified problem):

@jit(nopython=True, inline="always")
def jitted_g_function(x):
    return x + 1  # This is a simple function, which should be replaceable, and inlined in f below

def make_f(g):
    # Note: a new f() is created each time make_f() is called!
    @jit(nopython=True, cache=True)
    def f(x):
        # Assume a complex loop here
        return g(x) + g(-x)
    return f

f = make_f(jitted_g_function)
f(1)

but it cannot cache the inner f function, because NumbaWarning: Cannot cache compiled function "f" as it uses outer variables in a closure. Do you have any idea on how I could solve this? (essentially I have different jitted_g_functions which I'd like to pass to create a variant of f).
Thanks!

29 replies
Hameer Abbasi
@hameerabbasi
Here's a solution
In [1]: import numba as nb

In [2]: @nb.njit
   ...: def f(g, x):
   ...:     return g(x) + g(-x)
   ...:

In [3]: @nb.njit
   ...: def my_g(x):
   ...:     return x + 1
   ...:

In [4]: f(my_g, 1)
Out[6]: 2
code-master24
@code-master24
How can i debug my cuda kernel? Is there any option to print variables values?
40 replies
Hameer Abbasi
@hameerabbasi

Hello, I'm using the following kernel:

@njit(no_cfunc_wrapper=True, boundscheck=False)
def other_kernel(shape, pos, crd, m, v):
    out = np.zeros((shape[1],), dtype=np.float64)
    for i in range(shape[1]):
        p1 = i
        val = np.float64(0)
        p2begin, p2end = pos[p1], pos[p1 + 1]
        for jA in range(p2begin, p2end):
            j = crd[jA]
            val += m[jA] * v[j]
        out[i] = val

    return out

And compared to SciPy which has a far more complicated kernel for the same thing, it's about 60x slower. Am I doing something horribly wrong here? Everything being passed in is an ndarray other than a shape, which is a 2-tuple.

Graham Markall
@gmarkall
Is that an SpMV, and is the scipy kernel doing something to optimize memory access / caching with all its complexity?
Hameer Abbasi
@hameerabbasi
It is an SpMV, and no, the SciPy kernel is doing a SpMM.
I just looked, the last two were Numba typed lists.
Graham Markall
@gmarkall
how are you comparing the speed of kernels doing two different things?
ah
Hameer Abbasi
@hameerabbasi
Swapping those out for ndarrays bought the time down to 1.5x
But still... A list access shouldn't be THAT slow.
stuartarchibald
@stuartarchibald
typed list has a bug
it makes getitem slow, we know what it is and how to fix it, someone just needs to do it :)
Hameer Abbasi
@hameerabbasi
Graham, assume a 1-D array being a row vector. It performs the same computation.
Is it easy enough for me to do?
stuartarchibald
@stuartarchibald
it's a load of llvm builder stuff, basically writing optimised algs for things (based on the current reflected list impl) and then checking the machine code that comes out and it's performance to make sure it's ok, so if you are familiar with that then quite possibly something you could do?
Hameer Abbasi
@hameerabbasi
Never written LLVM, or even interpreted it too deeply, sadly. :(
What I could probably do is build a List type on top of an Array type. :)
luk-f-a
@luk-f-a
why not use the array type directly?
Hameer Abbasi
@hameerabbasi
The kernels become twice as complicated, sadly. :(
luk-f-a
@luk-f-a
with an array instead of a list?
because it's n-D?
Hameer Abbasi
@hameerabbasi
Yes, because I'd need to grow it.
Or perform a pass to determine the items.