numba/numba

Public channel for discussing Numba usage. Don't post confidential info here! Consider posting questions to: https://numba.discourse.group/ !

AidanReddy

@AidanReddy

(base) Aidans-MacBook-Pro:Calculations aidanreddy$ python3 energyTransferEfficiency
Traceback (most recent call last):
File "energyTransferEfficiency", line 6, in <module>
import numba
ModuleNotFoundError: No module named 'numba'

stuartarchibald

@stuartarchibald

yeah, it's probably because the python running your script isn't the one which has numba installed

Hannes Pahl

@HPLegion

https://docs.anaconda.com/anaconda/install/mac-os/#:~:text=In%20order%20to%20initialize%20after,and%20then%20run%20conda%20init%20.&text=If%20you%20are%20on%20macOS,followed%20by%20conda%20init%20zsh%20.

Here you find instructions to ensure that you are using the correct python installation (i.e. your anaconda installation)

(Which will probably work fine if you already installed numba using conda)

Step 7 is probably the critical one for you

AidanReddy

@AidanReddy

okay, thanks a lot. I just tried running the script specifying 'python3.7' as opposed to 'python3' and it seemed to work.

Hannes Pahl

@HPLegion

Great :) I still recommend you set your terminal to use the anaconda installation by default (since that is what you want in like 99% of the cases I can imagine) The system python is mostly meant for OS built ins and things like widgets etc (if I am not mistaken)

AidanReddy

@AidanReddy

@HPLegion If I delete the copy of the numba folder in the folder than my script is in, I still get the error message ModuleNotFoundError: No module named 'numba'

stuartarchibald

@stuartarchibald

have you created a conda environment with numba and python in it and then activated that environment?

AidanReddy

@AidanReddy

yes, I just tried and it did not work

stuartarchibald

@stuartarchibald

what does conda list say ?

AidanReddy

@AidanReddy

packages in environment at /opt/anaconda3:

stuartarchibald

@stuartarchibald

Perhaps use a pastebin :)

or put it into a threaded comment

Seems like it gets cut off at l in the alphabet, it also looks like that's possibly your base environment and not on that's been created and activated

AidanReddy

@AidanReddy

fYaDY0WB.txt

sorry, I'm not quite sure how to use paste bins, but let me know if that doesn't work

do you want me to run conda list within the conda environment with numba and python?

stuartarchibald

@stuartarchibald

you;ve installed numba via pip according to that?

and seemingly into the base anaconda environment

AidanReddy

@AidanReddy

yeah, I initially only had numba installed via conda, and then tried installing it via pip to see if that would solve my problem.

stuartarchibald

@stuartarchibald

I think that may well have compounded them unfortunately.

If you can get a new Anaconda installation, and then just create a new environment with Python and Numba and then go from there, I'd do that.

AidanReddy

@AidanReddy

okay, thanks for the help.

stuartarchibald

@stuartarchibald

no problem, good luck!

voegtlel

@voegtlel

Hey there, I'm currently using numba as recommended here (my code is much more complex, this is the simplified problem):

@jit(nopython=True, inline="always")
def jitted_g_function(x):
    return x + 1  # This is a simple function, which should be replaceable, and inlined in f below

def make_f(g):
    # Note: a new f() is created each time make_f() is called!
    @jit(nopython=True, cache=True)
    def f(x):
        # Assume a complex loop here
        return g(x) + g(-x)
    return f

f = make_f(jitted_g_function)
f(1)

but it cannot cache the inner f function, because NumbaWarning: Cannot cache compiled function "f" as it uses outer variables in a closure. Do you have any idea on how I could solve this? (essentially I have different jitted_g_functions which I'd like to pass to create a variant of f).
Thanks!

29 replies

Hameer Abbasi

@hameerabbasi

Here's a solution

In [1]: import numba as nb

In [2]: @nb.njit
   ...: def f(g, x):
   ...:     return g(x) + g(-x)
   ...:

In [3]: @nb.njit
   ...: def my_g(x):
   ...:     return x + 1
   ...:

In [4]: f(my_g, 1)
Out[6]: 2

code-master24

@code-master24

How can i debug my cuda kernel? Is there any option to print variables values?

40 replies

Hameer Abbasi

@hameerabbasi

Hello, I'm using the following kernel:

@njit(no_cfunc_wrapper=True, boundscheck=False)
def other_kernel(shape, pos, crd, m, v):
    out = np.zeros((shape[1],), dtype=np.float64)
    for i in range(shape[1]):
        p1 = i
        val = np.float64(0)
        p2begin, p2end = pos[p1], pos[p1 + 1]
        for jA in range(p2begin, p2end):
            j = crd[jA]
            val += m[jA] * v[j]
        out[i] = val

    return out

And compared to SciPy which has a far more complicated kernel for the same thing, it's about 60x slower. Am I doing something horribly wrong here? Everything being passed in is an ndarray other than a shape, which is a 2-tuple.

Graham Markall

@gmarkall

Is that an SpMV, and is the scipy kernel doing something to optimize memory access / caching with all its complexity?

Hameer Abbasi

@hameerabbasi

It is an SpMV, and no, the SciPy kernel is doing a SpMM.

I just looked, the last two were Numba typed lists.

Graham Markall

@gmarkall

how are you comparing the speed of kernels doing two different things?

Hameer Abbasi

@hameerabbasi

Swapping those out for ndarrays bought the time down to 1.5x

But still... A list access shouldn't be THAT slow.

stuartarchibald

@stuartarchibald

typed list has a bug

it makes getitem slow, we know what it is and how to fix it, someone just needs to do it :)

Hameer Abbasi

@hameerabbasi

Graham, assume a 1-D array being a row vector. It performs the same computation.

Is it easy enough for me to do?

stuartarchibald

@stuartarchibald

it's a load of llvm builder stuff, basically writing optimised algs for things (based on the current reflected list impl) and then checking the machine code that comes out and it's performance to make sure it's ok, so if you are familiar with that then quite possibly something you could do?

Hameer Abbasi

@hameerabbasi

Never written LLVM, or even interpreted it too deeply, sadly. :(

What I could probably do is build a List type on top of an Array type. :)

luk-f-a

@luk-f-a

why not use the array type directly?

Hameer Abbasi

@hameerabbasi

The kernels become twice as complicated, sadly. :(

luk-f-a

@luk-f-a

with an array instead of a list?

because it's n-D?

Hameer Abbasi

@hameerabbasi

Yes, because I'd need to grow it.

Or perform a pass to determine the items.

Where communities thrive

numba/numba

packages in environment at /opt/anaconda3:

Name Version Build Channel

Where communities thrive

People

Repo info

Activity

packages in environment at /opt/anaconda3:

Name Version Build Channel