[Sameer Deshmukh] @ralf.gommers I think this message states that the latest ndtypes does not need refcounting. Let me try updating this on the ruby side and then I can update you on what is going on as of now.
An update on reference counted types and the history:
The Numba team was concerned about having reference counted types in
libndtypes. However, the more I looked at it, I can't see any harm. For example, ...
t = ndt_string(ctx); ndt_del(t)
.... is now just:
t = ndt_string(ctx); ndt_decref(t);
So refcounting is all internal to
libndtypes and has nothing to do with Python refcounting.
It is all implemented in the latest
One implementation detail: An
ndt_t * is now always
const except for the
ndt_decref(const ndt_t *) follows the convention in the Linux kernel, where
kfree() takes a
const void *.
The convention in the Linux kernel has been the source of some controversy. I understand some of the counter arguments, but on the whole I think it's a good idea. More here:
const *, which has Linus' endorsement but might lead to raised eyebrows elsewhere.
[Stefan Krah] I've implemented flexible ND arrays, which are more versatile than the offset-based
These are needed for quirky formats like
The trade-off is that they are not in a single memory block.
You can now assign subarrays of different sizes:
>>> >>> x = xnd([, [1, 2]], type="array of array of int64") >>> x xnd([, [1, 2]], type='array of array of int64') >>> x = [1000, 2000, 3000] >>> x xnd([[1000, 2000, 3000], [1, 2]], type='array of array of int64') >>>
xnd-daskshould be possible for ndarray: dask/dask#4669
[Sameer Deshmukh] Can someone please explain the possible cause of this error:
util.c:140: internal error: invalid value?
It happens during the GC sweep phase in Ruby, and I'm not able to understand whether this is due to premature deallocation or something else, because line 140 in
util.c uses a switch-case statement utilizing a pointer to an ndt.
ndtype(float * float * float [m/s])or similar
>>> x = xnd([[, [2,3]], [, [5,6]]]) >>> y = x[:, 0] >>> y xnd([, ], type='var * var * int64') >>> print(y.type.ast_repr()) VarDim( VarDimElem( VarDim( Int64(access=Concrete, ndim=0, datasize=8, align=8, flags=), offsets=[0, 1, 3, 4, 6], slices=, itemsize=8, access=Concrete, ndim=1, datasize=48, align=8, flags= ), index=0, offsets=[0, 2, 4], slices=, itemsize=8, access=Concrete, ndim=2, datasize=48, align=8, flags= ), offsets=[0, 2], slices=[0:9223372036854775807:1], itemsize=8, access=Concrete, ndim=3, datasize=48, align=8, flags= )
[Stefan Krah] @wolfv
ndtypes two ways:
1) Named type constructors:
>>> ndt("Velocity(float32)") ndt("Velocity(float32)") >>> >>> ndt("Velocity(float32)") == ndt("float32") False
>>> typedef("velocity", "float32") >>> ndt("velocity") ndt("velocity") >>> ndt("velocity") == ndt("float32") False
[Stefan Krah] One could allow special characters in typedefs, this is currently not possible:
>>> typedef("m/s", "float32") Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: invalid character in typedef: '/'
But there's no fundamental reason.