Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Quansight
    @quansightai_twitter
    [Ralf Gommers] thanks @sameer.deshmukh. A separate xnd-ruby repo seems like a good idea to me. makes it visible, and it can be tested together with libxnd in CI easily.
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] @sameer.deshmukh I think you may be wrong there.
    [Hameer Abbasi] It does ref count. It just says that a simple create and delete can be emulated by ref counting.
    Quansight
    @quansightai_twitter
    [Stefan Krah] Yes, there is ref-counting. That message is mostly concerned with the question in what way it could be harmful to JITs (I think it cannot) and to issue a preemptive rationalization for freeing a const *, which has Linus' endorsement but might lead to raised eyebrows elsewhere.
    Quansight
    @quansightai_twitter
    [XND] There is 1 event this week
    XND meeting
    June 7th, 2019 from 9:00 AM to 10:00 AM GMT-0400
    Quansight
    @quansightai_twitter

    [Stefan Krah] I've implemented flexible ND arrays, which are more versatile than the offset-based var dimensions.

    These are needed for quirky formats like GeoSON.

    The trade-off is that they are not in a single memory block.

    You can now assign subarrays of different sizes:

    >>> 
    >>> x = xnd([[0], [1, 2]], type="array of array of int64") 
    >>> x
    xnd([[0], [1, 2]], type='array of array of int64')
    >>> x[0] = [1000, 2000, 3000]
    >>> x
    xnd([[1000, 2000, 3000], [1, 2]], type='array of array of int64')
    >>>
    Quansight
    @quansightai_twitter

    [Stefan Krah] Together with tagged unions one arrives at a far more condensed GeoJSON representation:

    `\

    Quansight
    @quansightai_twitter
    [Hameer Abbasi] This is great! So are these arrays refcounted or always copies?
    Quansight
    @quansightai_twitter
    [Stefan Krah] It's a copy. I wonder if things get too tricky with sharing. Not actually implementation-wise, but for the user.
    [Hameer Abbasi] Well, NumPy currently only does copies. But with XND, I would assume people know what they’re doing.
    Quansight
    @quansightai_twitter
    [Pearu Peterson] sharing should be fine when read-only
    Quansight
    @quansightai_twitter
    [XND] There is 1 event this week
    XND meeting
    June 14th, 2019 from 9:00 AM to 10:00 AM GMT-0400
    Quansight
    @quansightai_twitter
    [XND]
    Calendar event was cancelled
    XND meeting
    June 14th, 2019 from 9:00 AM to 10:00 AM GMT-0400
    Quansight
    @quansightai_twitter
    [XND] There is 1 event this week
    XND meeting
    June 21st, 2019 from 9:00 AM to 10:00 AM GMT-0400
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] I think it should
    Quansight
    @quansightai_twitter
    [XND]
    Event starting in 15 minutes:
    XND meeting
    June 21st, 2019 from 9:00 AM to 10:00 AM GMT-0400
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] xnd-dask should be possible for ndarray: dask/dask#4669

    [Stefan Krah] Nice, this also needs to be resolved, probably on the NumPy side:

    dask/dask#4632

    Quansight
    @quansightai_twitter
    [XND]
    Calendar event was cancelled
    XND meeting
    March 22nd, 2019 from 9:00 AM to 10:00 AM GMT-0400
    Recurring
    Quansight
    @quansightai_twitter
    [Stefan Krah] Meeting cancelled because of vacation time. Next is probably July 19th.
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter

    [Sameer Deshmukh] Can someone please explain the possible cause of this error: util.c:140: internal error: invalid value?

    It happens during the GC sweep phase in Ruby, and I'm not able to understand whether this is due to premature deallocation or something else, because line 140 in util.c uses a switch-case statement utilizing a pointer to an ndt.

    Quansight
    @quansightai_twitter
    [Hameer Abbasi] I think it’s May be because primitive ndts were made singletons.
    Wolf Vollprecht
    @wolfv
    hey, we had a discussion yesterday about types in scientific python at scipy
    partially about units
    i was wondering if one could extend ndtypes with unit support, so that one could write something like ndtype(float * float * float [m/s]) or similar
    could be incredibly powerful in a way
    but might be out of scope
    Quansight
    @quansightai_twitter
    [Sameer Deshmukh] @habbasi meaning? How does that change the behaviour of the front end wrappers? I could not see any major changes in the python wrappers for NDT.
    Quansight
    @quansightai_twitter
    [Sameer Deshmukh] @habbasi any update on this? @stefan.krah ?
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] I’m not sure. Unless you were incref/decref-ing the native types, it shouldn’t make a difference.
    [Hameer Abbasi] Any code that was correct then will be correct now.
    Quansight
    @quansightai_twitter

    [Stefan Krah] @sameer.deshmukh next_dim() is a static function that's only called in the dtype() functions.

    So it looks like the cause is that t->ndim > 0 but t is not a dimension type. This should not happen.

    Quansight
    @quansightai_twitter

    [Stefan Krah] Also, are you using the VarDimElem type? That is a bit trickier, since it introduces logical and physical dimensions.

    I'd leave that one out at first, it is only needed for eliminating dimensions in mixed indexing and slicing of var dimensions.

    Quansight
    @quansightai_twitter
    [Stefan Krah] Example:
    >>> x = xnd([[[1], [2,3]], [[4], [5,6]]])
    >>> y = x[:, 0]
    >>> y
    xnd([[1], [4]], type='var * var * int64')
    >>> print(y.type.ast_repr())
    VarDim(
      VarDimElem(
        VarDim(
          Int64(access=Concrete, ndim=0, datasize=8, align=8, flags=[]),
          offsets=[0, 1, 3, 4, 6],
          slices=[],
          itemsize=8,
          access=Concrete, ndim=1, datasize=48, align=8, flags=[]
        ),
        index=0,
        offsets=[0, 2, 4],
        slices=[],
        itemsize=8,
        access=Concrete, ndim=2, datasize=48, align=8, flags=[]
      ),
      offsets=[0, 2],
      slices=[0:9223372036854775807:1],
      itemsize=8,
      access=Concrete, ndim=3, datasize=48, align=8, flags=[]
    )
    Quansight
    @quansightai_twitter
    [Stefan Krah] Of course, if Ruby does not have this sort of slicing, you don't need it at all.

    [Stefan Krah] @wolfv ndtypes two ways:

    1) Named type constructors:

    >>> ndt("Velocity(float32)")
    ndt("Velocity(float32)")
    >>> 
    >>> ndt("Velocity(float32)") == ndt("float32")
    False

    2) Typedefs:

    >>> typedef("velocity", "float32")
    >>> ndt("velocity")
    ndt("velocity")
    >>> ndt("velocity") == ndt("float32")
    False
    Quansight
    @quansightai_twitter

    [Stefan Krah] One could allow special characters in typedefs, this is currently not possible:

    >>> typedef("m/s", "float32")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: invalid character in typedef: '/'

    But there's no fundamental reason.

    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] With the magic of uarray, xnd-dask is now possible.
    [Hameer Abbasi] Of course, it will break often, and in horrible ways sometimes.
    Quansight
    @quansightai_twitter
    [Travis Oliphant] This is an interesting use-case to pursue. As is xnd-xarray
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] This was part of the reason I wanted dispatching not just on arrays, but dtypes and ufuncs as well.
    Quansight
    @quansightai_twitter
    [XND] There are no events this week
    Quansight
    @quansightai_twitter
    [Hameer Abbasi] @stefan.krah Will you be available to meet sometime soon? The window for getting ndtypes/xnd/gumath into NumPy is really closing.
    Quansight
    @quansightai_twitter
    [Ralf Gommers] is it really? I'm looking forward to the first thing to be put on paper by Sebastian. that's a nice point for discussion - right now it's really hard to follow. A draft NEP is the point where there's a design to discuss / compare with.
    [Ralf Gommers] that said, +1 for having this conversation of course:)