## Where communities thrive

• Join over 1.5M+ people
• Join over 100K+ communities
• Free without limits
##### Activity
Dave Hirschfeld
@dhirschfeld
As to whether or not sin should be a function or a method - this is an issue I see also in numpy in that the boundary for whats a method vs what's a function is somewhat blurred and opaque.
For the most part I like the method-call version as it can be clearer if you're chaining several operations together
I often get bitten by the fact that there is no abs method - only the function version
I quite like pandas' concept of function namespaces - e.g. df.dt, df.str which hold functions for datetime and string operations respectively
Dave Hirschfeld
@dhirschfeld
xnd.array could possibly use a similar concept where the xnd.array.ufunc "namespace" exposed the gumath.functions namespace with the first argument substituted for self
Along with Pearu's suggestion that cls=type(args[0]) by default, this would ensure that the method-call versions returned the type of the instance
Dave Hirschfeld
@dhirschfeld
e.g.
class A(xnd):
pass

class B(xnd):
pass

a = A([1])
b = B([2])

assert type(c) == type(a)

assert type(c) == type(a)

assert type(c) == type(b)

assert type(c) == type(b)
Dave Hirschfeld
@dhirschfeld
...but cls can still be overridden:
c = gumath.functions.add(a, b, cls=type(b))
assert type(c) == type(b)

assert type(c) == type(b)
Anyway, just a thought. On the downside xnd.array.ufunc is a bit of a mouthful ¯_(ツ)_/¯
Quansight
[Travis Oliphant] It’s true that you can’t just quote a principle nor is it always obvious whether inheritance or composition is a good idea. But, but, I wish I (or anyone else on the NumPy list at the time) had known about the Liskov substitution principle because it would have definitely led to np.matrix not being a sub-class of ndarray
[Travis Oliphant] Because of the fundamental changes to the interface that were made (which were not merely additive). '
[Travis Oliphant] Adding units (reasonable use of subclass)
[Travis Oliphant] Masked arrays are a gray area for me. They can reasonably be subclasses.
Quansight
[Hameer Abbasi] If we define Liskov in the following way: For a deterministic Operation on the parent class, data produced could have a child class, but the parent’s attributes and operations on that child should be identical to if the parent was passed in; or themselves satisfy Liskov.
Quansight

[Hameer Abbasi] Then xnd.array is a valid subclass of xnd.xnd, but this doesn’t hold for MaskedArrays or matrix objects.

astropy.Quantity is valid sometimes and sometimes errors, assuming consistency of units and no conversion. If inconsistent units are supplied, it violates Liskov in this form.

If we have a much more relaxed definition, such that the dtype and the shape have to be consistent (type in xnd-speak), and errors are allowed, then MaskedArray and unit become valid.

Quansight
[XND]
Event starting in 15 minutes:
XND meeting
May 10th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[Travis Oliphant] Good illustration that Liskov alone does not resolve the question -- but it does provide useful scaffolding for the conversation that will help you make a decision you will be happy with later.
Quansight
[XND] There is 1 event this week
XND meeting
May 17th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[XND]
Calendar event was cancelled
XND meeting
May 17th, 2019 from 9:00 AM to 10:00 AM GMT-0400
[Hameer Abbasi] @stefan.krah Is it possible to keep the meeting open by any chance? I had some productive discussion at BIDS I’d like everyone to hear.
[Stefan Krah] I'd rather not, I also have another meeting tomorrow. Can it wait until next week?
[Hameer Abbasi] Sure, if you prefer. I’ll try to make a brain dump so that it isn’t lost.
Quansight
[Stefan Krah] Thanks, it will be interesting to hear about the discussion.
Quansight
[XND] There is 1 event this week
XND meeting
May 24th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[XND]
Calendar event was cancelled
XND meeting
May 24th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[XND] There is 1 event this week
XND meeting
May 31st, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[Sameer Deshmukh] I'll moving all the ruby wrappers for the xnd project into this repo: https://github.com/xnd-project/xnd-ruby
Quansight

[Sameer Deshmukh] @ralf.gommers I think this message states that the latest ndtypes does not need refcounting. Let me try updating this on the ruby side and then I can update you on what is going on as of now.
An update on reference counted types and the history:

The Numba team was concerned about having reference counted types in libndtypes. However, the more I looked at it, I can't see any harm. For example, ...

t = ndt_string(ctx);
ndt_del(t)

.... is now just:

t = ndt_string(ctx);
ndt_decref(t);

So refcounting is all internal to libndtypes and has nothing to do with Python refcounting.

It is all implemented in the latest libndtypes.

One implementation detail: An ndt_t * is now always const except for the refcnt field. ndt_decref(const ndt_t *) follows the convention in the Linux kernel, where kfree() takes a const void *.

The convention in the Linux kernel has been the source of some controversy. I understand some of the counter arguments, but on the whole I think it's a good idea. More here:

https://yarchive.net/comp/const.html

Quansight
[Ralf Gommers] thanks @sameer.deshmukh. A separate xnd-ruby repo seems like a good idea to me. makes it visible, and it can be tested together with libxnd in CI easily.
Quansight
[Hameer Abbasi] @sameer.deshmukh I think you may be wrong there.
[Hameer Abbasi] It does ref count. It just says that a simple create and delete can be emulated by ref counting.
Quansight
[Stefan Krah] Yes, there is ref-counting. That message is mostly concerned with the question in what way it could be harmful to JITs (I think it cannot) and to issue a preemptive rationalization for freeing a const *, which has Linus' endorsement but might lead to raised eyebrows elsewhere.
Quansight
[XND] There is 1 event this week
XND meeting
June 7th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight

[Stefan Krah] I've implemented flexible ND arrays, which are more versatile than the offset-based var dimensions.

These are needed for quirky formats like GeoSON.

The trade-off is that they are not in a single memory block.

You can now assign subarrays of different sizes:

>>>
>>> x = xnd([[0], [1, 2]], type="array of array of int64")
>>> x
xnd([[0], [1, 2]], type='array of array of int64')
>>> x[0] = [1000, 2000, 3000]
>>> x
xnd([[1000, 2000, 3000], [1, 2]], type='array of array of int64')
>>>
Quansight

[Stefan Krah] Together with tagged unions one arrives at a far more condensed GeoJSON representation:

\

Quansight
[Hameer Abbasi] This is great! So are these arrays refcounted or always copies?
Quansight
[Stefan Krah] It's a copy. I wonder if things get too tricky with sharing. Not actually implementation-wise, but for the user.
[Hameer Abbasi] Well, NumPy currently only does copies. But with XND, I would assume people know what they’re doing.
Quansight
[Pearu Peterson] sharing should be fine when read-only
Quansight
[XND] There is 1 event this week
XND meeting
June 14th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[XND]
Calendar event was cancelled
XND meeting
June 14th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[XND] There is 1 event this week
XND meeting
June 21st, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[Hameer Abbasi] I think it should
Quansight
[XND]
Event starting in 15 minutes:
XND meeting
June 21st, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[Hameer Abbasi] xnd-dask` should be possible for ndarray: dask/dask#4669

[Stefan Krah] Nice, this also needs to be resolved, probably on the NumPy side:

Quansight
[XND]
Calendar event was cancelled
XND meeting
March 22nd, 2019 from 9:00 AM to 10:00 AM GMT-0400
Recurring
Quansight