Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Quansight
    @quansightai_twitter
    [XND]
    Event starting in 15 minutes:
    XND meeting
    June 21st, 2019 from 9:00 AM to 10:00 AM GMT-0400
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] xnd-dask should be possible for ndarray: dask/dask#4669

    [Stefan Krah] Nice, this also needs to be resolved, probably on the NumPy side:

    dask/dask#4632

    Quansight
    @quansightai_twitter
    [XND]
    Calendar event was cancelled
    XND meeting
    March 22nd, 2019 from 9:00 AM to 10:00 AM GMT-0400
    Recurring
    Quansight
    @quansightai_twitter
    [Stefan Krah] Meeting cancelled because of vacation time. Next is probably July 19th.
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter

    [Sameer Deshmukh] Can someone please explain the possible cause of this error: util.c:140: internal error: invalid value?

    It happens during the GC sweep phase in Ruby, and I'm not able to understand whether this is due to premature deallocation or something else, because line 140 in util.c uses a switch-case statement utilizing a pointer to an ndt.

    Quansight
    @quansightai_twitter
    [Hameer Abbasi] I think it’s May be because primitive ndts were made singletons.
    Wolf Vollprecht
    @wolfv
    hey, we had a discussion yesterday about types in scientific python at scipy
    partially about units
    i was wondering if one could extend ndtypes with unit support, so that one could write something like ndtype(float * float * float [m/s]) or similar
    could be incredibly powerful in a way
    but might be out of scope
    Quansight
    @quansightai_twitter
    [Sameer Deshmukh] @habbasi meaning? How does that change the behaviour of the front end wrappers? I could not see any major changes in the python wrappers for NDT.
    Quansight
    @quansightai_twitter
    [Sameer Deshmukh] @habbasi any update on this? @stefan.krah ?
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] I’m not sure. Unless you were incref/decref-ing the native types, it shouldn’t make a difference.
    [Hameer Abbasi] Any code that was correct then will be correct now.
    Quansight
    @quansightai_twitter

    [Stefan Krah] @sameer.deshmukh next_dim() is a static function that's only called in the dtype() functions.

    So it looks like the cause is that t->ndim > 0 but t is not a dimension type. This should not happen.

    Quansight
    @quansightai_twitter

    [Stefan Krah] Also, are you using the VarDimElem type? That is a bit trickier, since it introduces logical and physical dimensions.

    I'd leave that one out at first, it is only needed for eliminating dimensions in mixed indexing and slicing of var dimensions.

    Quansight
    @quansightai_twitter
    [Stefan Krah] Example:
    >>> x = xnd([[[1], [2,3]], [[4], [5,6]]])
    >>> y = x[:, 0]
    >>> y
    xnd([[1], [4]], type='var * var * int64')
    >>> print(y.type.ast_repr())
    VarDim(
      VarDimElem(
        VarDim(
          Int64(access=Concrete, ndim=0, datasize=8, align=8, flags=[]),
          offsets=[0, 1, 3, 4, 6],
          slices=[],
          itemsize=8,
          access=Concrete, ndim=1, datasize=48, align=8, flags=[]
        ),
        index=0,
        offsets=[0, 2, 4],
        slices=[],
        itemsize=8,
        access=Concrete, ndim=2, datasize=48, align=8, flags=[]
      ),
      offsets=[0, 2],
      slices=[0:9223372036854775807:1],
      itemsize=8,
      access=Concrete, ndim=3, datasize=48, align=8, flags=[]
    )
    Quansight
    @quansightai_twitter
    [Stefan Krah] Of course, if Ruby does not have this sort of slicing, you don't need it at all.

    [Stefan Krah] @wolfv ndtypes two ways:

    1) Named type constructors:

    >>> ndt("Velocity(float32)")
    ndt("Velocity(float32)")
    >>> 
    >>> ndt("Velocity(float32)") == ndt("float32")
    False

    2) Typedefs:

    >>> typedef("velocity", "float32")
    >>> ndt("velocity")
    ndt("velocity")
    >>> ndt("velocity") == ndt("float32")
    False
    Quansight
    @quansightai_twitter

    [Stefan Krah] One could allow special characters in typedefs, this is currently not possible:

    >>> typedef("m/s", "float32")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: invalid character in typedef: '/'

    But there's no fundamental reason.

    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] With the magic of uarray, xnd-dask is now possible.
    [Hameer Abbasi] Of course, it will break often, and in horrible ways sometimes.
    Quansight
    @quansightai_twitter
    [Travis Oliphant] This is an interesting use-case to pursue. As is xnd-xarray
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] This was part of the reason I wanted dispatching not just on arrays, but dtypes and ufuncs as well.
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] @stefan.krah Will you be available to meet sometime soon? The window for getting ndtypes/xnd/gumath into NumPy is really closing.
    Quansight
    @quansightai_twitter
    [Ralf Gommers] is it really? I'm looking forward to the first thing to be put on paper by Sebastian. that's a nice point for discussion - right now it's really hard to follow. A draft NEP is the point where there's a design to discuss / compare with.
    [Ralf Gommers] that said, +1 for having this conversation of course:)
    Quansight
    @quansightai_twitter
    [Stefan Krah] @habbasi Indeed, I should be focusing more on XND again. I'll start today. :)
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] I had a few ideas/suggestions.
    Quansight
    @quansightai_twitter
    [Sameer Deshmukh] @stefan.krah can you please explain why the XND#serialize function is an instance method and not a class method like that in NDT? For me its quite counter intuitive to see that it modifies self according to the value of the serialized string that it is passed. Might it not be better to implement it as a class method?
    Quansight
    @quansightai_twitter

    [Stefan Krah] Do you mean deserialize? It is a class method:

    "deserialize", (PyCFunction)pyxnd_deserialize, METH_O|METH_CLASS, NULL },

    In Python you can also call class methods from instances, but the instance is not modified:

    >>> x = Decimal("1.23")
    >>> y = x.from_float(100)
    >>> x
    Decimal('1.23')
    >>> y
    Decimal('100')
    Quansight
    @quansightai_twitter
    [Stefan Krah] The proper way (IMO) of course is to call via the type:
    >>> Decimal.from_float(100)
    Decimal('100')
    Quansight
    @quansightai_twitter
    [Stefan Krah] In the tests, of course, I use x.deserialize(b) out of laziness. :)
    Quansight
    @quansightai_twitter

    [Sameer Deshmukh] @stefan.krah question about NDT object deallocation:
    You said that reducing the refcount of a NDT object 0 via ndt_decref is supposed to be the only way of deallocating an NDT struct.

    However in the python code I can see that you simply point to the default PyObject_del for the deletion (without decreasing refcount). Can you please explain why this is so?

    [Sameer Deshmukh] How does the NDT object get deallocated safely even without reducing refcount?
    Quansight
    @quansightai_twitter

    [Stefan Krah] .tp_free = PyObject_Del is just the counterpart to self = (NdtObject *)type->tp_alloc(type, 0)

    It allocates and frees the *Python object structure*.

    .tp_dealloc = (destructor) ndtype_dealloc is the part that deallocates the ndt struct by calling ndt_decref().

    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [Ralf Gommers] Sebastian Berg just shared first rough draft of the designs for extensible dtypes in NumPy (https://hackmd.io/kxuh15QGSjueEKft5SaMug) and related changes in ufunc dispatching (https://hackmd.io/y7ghitUtRQaMyaHFGe-ueQ)
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [Travis Oliphant] The stdlib authors are going to be looking at xnd.io for ideas as they continue to develop the library. https://github.com/stdlib-js/stdlib
    [Travis Oliphant] Some of them may reach out to xnd devs