## Where communities thrive

• Join over 1.5M+ people
• Join over 100K+ communities
• Free without limits
• Create your own community
##### Activity
Quansight
[Aaron Meurer] I'm curious where does np.matrix break? Doesn't broadcasting take care of 0-D vs. 2-D for the most part?
[Aaron Meurer] of course broadcasting isn't great semantics for matrix in the first place. That's why I said "only for multiplication". np.matrix([1]) + np.matrix([[1, 2], [3, 4]]) works, which is not good
[Ralf Gommers] definitely not, it breaks in all sorts of ways (indexing to start with). anyway, np.matrix is terrible, that's not the interesting discussion. the interesting one is which subclasses are and aren't a good idea. the units one is easy, but there's always a gray area. making fixed rules for those is not trivial. certainly just quoting a principle isn't enough
[Aaron Meurer] maybe not interesting to you :)
[Aaron Meurer] I disagree that np.matrix is terrible. I've had code where I had problems with np.array that went away when I switched to np.matrix, because matrix has better semantics (always being 2-D being one of them).
[Aaron Meurer] So I am interested in the other side of that
Quansight
[Aaron Meurer] my biggest thing is that I want proper type checking—for things to fail when shapes mismatch. The above behavior for 1x1 + 2x2 is one issue with that. The way numpy handles 1D @ 2D is another (which is solved by np.matrix).
[Ralf Gommers] terrible as a subclass I meant. if you do linalg, by all means create a matrix object. just don't make it an ndarray subclass. Liskov does apply there
Quansight
[Aaron Meurer] a matrix type that doesn't attempt to be a subclass could be good actually. Then it could be much more strict about semantics, rather than automatically inheriting everything from array
[Aaron Meurer] I don't know if something like that already exists
[Stefan Krah] Type checking in xnd isn't done on the object level. You can typedef "matrix" as N * M * Scalar and get type-checking in both xnd.xnd and xnd.array.
Quansight
[Stefan Krah] You could define xnd.matrix with a constructor that only allows construction of values that match this pattern.
[Ralf Gommers] not sure about a non-subclass matrix, but are you aware of PyLops?
[Aaron Meurer] no. I'll take a look
[Stefan Krah] No, I bookmarked it.
Dave Hirschfeld
@dhirschfeld

[Stefan Krah] We could be fancy and check for the narrowest type among the input arguments and return that, but this sort of thing slows down small array operations.

I'm pretty interested in keeping the overhead as minimal as possible so if that means having to specify the type (cls) then so be it however perhaps we can get the best of both worlds by skipping the type guessing heuristics in the case that cls is not None?

[Pearu Peterson] Similarly for multiple argument cases, the first argument type defines the type of the output objects (default behavior).

I'd be fine with this choice - so long as it's consistent it can be accomodated. I guess the other option would be return the type of the first common base but I assume that would have a much larger performance impact.

Dave Hirschfeld
@dhirschfeld
As to whether or not sin should be a function or a method - this is an issue I see also in numpy in that the boundary for whats a method vs what's a function is somewhat blurred and opaque.
For the most part I like the method-call version as it can be clearer if you're chaining several operations together
I often get bitten by the fact that there is no abs method - only the function version
I quite like pandas' concept of function namespaces - e.g. df.dt, df.str which hold functions for datetime and string operations respectively
Dave Hirschfeld
@dhirschfeld
xnd.array could possibly use a similar concept where the xnd.array.ufunc "namespace" exposed the gumath.functions namespace with the first argument substituted for self
Along with Pearu's suggestion that cls=type(args[0]) by default, this would ensure that the method-call versions returned the type of the instance
Dave Hirschfeld
@dhirschfeld
e.g.
class A(xnd):
pass

class B(xnd):
pass

a = A([1])
b = B([2])

c = gumath.functions.add(a, b)
assert type(c) == type(a)

assert type(c) == type(a)

c = gumath.functions.add(b, a)
assert type(c) == type(b)

assert type(c) == type(b)
Dave Hirschfeld
@dhirschfeld
...but cls can still be overridden:
c = gumath.functions.add(a, b, cls=type(b))
assert type(c) == type(b)

c = a.ufunc.add(b, cls=type(b))
assert type(c) == type(b)
Anyway, just a thought. On the downside xnd.array.ufunc is a bit of a mouthful ¯_(ツ)_/¯
Quansight
[Travis Oliphant] It’s true that you can’t just quote a principle nor is it always obvious whether inheritance or composition is a good idea. But, but, I wish I (or anyone else on the NumPy list at the time) had known about the Liskov substitution principle because it would have definitely led to np.matrix not being a sub-class of ndarray
[Travis Oliphant] Because of the fundamental changes to the interface that were made (which were not merely additive). '
[Travis Oliphant] Adding units (reasonable use of subclass)
[Travis Oliphant] Masked arrays are a gray area for me. They can reasonably be subclasses.
Quansight
[Hameer Abbasi] If we define Liskov in the following way: For a deterministic Operation on the parent class, data produced could have a child class, but the parent’s attributes and operations on that child should be identical to if the parent was passed in; or themselves satisfy Liskov.
Quansight

[Hameer Abbasi] Then xnd.array is a valid subclass of xnd.xnd, but this doesn’t hold for MaskedArrays or matrix objects.

astropy.Quantity is valid sometimes and sometimes errors, assuming consistency of units and no conversion. If inconsistent units are supplied, it violates Liskov in this form.

If we have a much more relaxed definition, such that the dtype and the shape have to be consistent (type in xnd-speak), and errors are allowed, then MaskedArray and unit become valid.

Quansight
[XND]
Event starting in 15 minutes:
XND meeting
May 10th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[Travis Oliphant] Good illustration that Liskov alone does not resolve the question -- but it does provide useful scaffolding for the conversation that will help you make a decision you will be happy with later.
Quansight
[XND] There is 1 event this week
XND meeting
May 17th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[XND]
Calendar event was cancelled
XND meeting
May 17th, 2019 from 9:00 AM to 10:00 AM GMT-0400
[Hameer Abbasi] @stefan.krah Is it possible to keep the meeting open by any chance? I had some productive discussion at BIDS I’d like everyone to hear.
[Stefan Krah] I'd rather not, I also have another meeting tomorrow. Can it wait until next week?
[Hameer Abbasi] Sure, if you prefer. I’ll try to make a brain dump so that it isn’t lost.
Quansight
[Stefan Krah] Thanks, it will be interesting to hear about the discussion.
Quansight
[XND] There is 1 event this week
XND meeting
May 24th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[XND]
Calendar event was cancelled
XND meeting
May 24th, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[XND] There is 1 event this week
XND meeting
May 31st, 2019 from 9:00 AM to 10:00 AM GMT-0400
Quansight
[Sameer Deshmukh] I'll moving all the ruby wrappers for the xnd project into this repo: https://github.com/xnd-project/xnd-ruby
Quansight

[Sameer Deshmukh] @ralf.gommers I think this message states that the latest ndtypes does not need refcounting. Let me try updating this on the ruby side and then I can update you on what is going on as of now.
An update on reference counted types and the history:

The Numba team was concerned about having reference counted types in libndtypes. However, the more I looked at it, I can't see any harm. For example, ...

t = ndt_string(ctx);
ndt_del(t)

.... is now just:

t = ndt_string(ctx);
ndt_decref(t);

So refcounting is all internal to libndtypes and has nothing to do with Python refcounting.

It is all implemented in the latest libndtypes.

One implementation detail: An ndt_t * is now always const except for the refcnt field. ndt_decref(const ndt_t *) follows the convention in the Linux kernel, where kfree() takes a const void *.

The convention in the Linux kernel has been the source of some controversy. I understand some of the counter arguments, but on the whole I think it's a good idea. More here:

https://yarchive.net/comp/const.html

Quansight
[Ralf Gommers] thanks @sameer.deshmukh. A separate xnd-ruby repo seems like a good idea to me. makes it visible, and it can be tested together with libxnd in CI easily.
Quansight
[Hameer Abbasi] @sameer.deshmukh I think you may be wrong there.
[Hameer Abbasi] It does ref count. It just says that a simple create and delete can be emulated by ref counting.
Quansight
[Stefan Krah] Yes, there is ref-counting. That message is mostly concerned with the question in what way it could be harmful to JITs (I think it cannot) and to issue a preemptive rationalization for freeing a const *, which has Linus' endorsement but might lead to raised eyebrows elsewhere.
Quansight