And to that respect the separation numpy/scipy is sometimes (often) confusing
from aesara.tensor.special import softmax ?
from aesara.tensor.math import softmax
Well math objects are usually offered at the tensor level, so it would be
from aesara.tensor import softmax or more typically import aesara.tensor as at; at.softmax
if it’s only going to be visible to developers I’d suggest
special.py. Makes comparisons with what’s available in scipy easier
So we should move the equivalent tensor/math.py methods that are implemented scipy.special to there, and then what I wanted to do in the first place which was to take the softmax out of nnet
Yeah that's coming out. I just didn't want to put it in the already large math.py
I’m reading the code for linkers and I have 2 questions:
1- Is there any reason why the file structure is different for numba and jax linkers?
2- Is there any way to print the generated code without compiling for debugging ? That was very useful with MCX.
Regarding the file structure I guess the one of the reason we split numba's dispatch was because the file was getting too long.
The jax file structure is pretty much same except for dispatch being in same file. We should probably split it like Numba.
Thanks! Oh and 3- why is the code generation approach used with numba and not (that I saw) with jax?
Probably because Jax offers much more flexibility than Numba.
For instance take the case of Scan. Jax has an inbuilt scan like functionality, but in case of Numba we have to create the loops manually.
We can use code generation approach in Jax too, but i think we're yet to run into logic that cannot be implemented in Jax.
1 reply
Regarding debugging I don't know what's the usual way to go about it. For me I just put a break point in the compile_function_src. Which gives me source string for required function.
That function is located in link/utils.py I think
The problem is some of the functions aren't generated with strings while some of them are. So it's pretty hard to get a uniform debug functionality on them.
1 reply
and both are technically taking the exact same approach
the difference is the interface
at least in the case of
Scan
that has more to do with JAX's limitations (or scope/purpose), though
it's a tensor library
Numba is much more general
attempting to cover nearly all of Python
but, yeah, I see what you're saying
it's a limitation in terms of mapping simplicity and the like
or something of that nature
but Numba's generality is what will ultimately make it considerably easier to write high performance
Ops
e.g. like writing standard Python
Ops
I'm saying that
Op.perform implementations will ultimately be able to serve as the only necessary Op implementation code
we might need/want to refactor the interface a bit, and the Python needs to be Numba compatible
but that's the idea
like
Op.perform
what it takes as arguments and what it returns/does
the current design is built around Theano's old ideas
e.g. that
lists serve as "pointers" to memory that Theano manages manually
it's neat, but not particularly good
brandonwillard: I tried to follow your blogpost on DLMs, but I didn't succeed in getting auto_updates to work with this minimal graph:
rng = aesara.shared(np.random.default_rng(), borrow=True)
rng.tag.is_rng = True
rng.default_update = rng
def step(v_tm1, rng):
v_t = at.random.normal(v_tm1, 1.0, name='v', rng=rng)
return v_t
v, updates = aesara.scan(
fn=step,
outputs_info=[np.array(0.0)],
non_sequences=[rng],
n_steps=5,
strict=True,
)
v_draws = aesara.function([], v, updates=updates)()
assert len(np.unique(np.diff(v_draws))) > 1 # All values have the same offsetAny thing obvious I am missing?