[Saul Shanabrook] IMG_20180712_134331.jpg
[Saul Shanabrook] I have enough lower level primitives built up now in the numba support to jit compile a matrix multiply function: https://github.com/Quansight/numba-xnd/blob/8cdd89362aedde4516afedc44b21c5fe7aa9bf3c/tests/test_examples.py
As you can see, it's very low level, similar to how you would write it in C, but in Python. Now I can start building the higher level APIs on top of that that have some more compile time typing knowledge.
[Ivan Ogasawara] very nice!
[Saul Shanabrook] @anderson.banihirwe Do you think https://xndframes.readthedocs.io/en/latest/index.html is usable enough to add a link to it from xnd.io?
[Anderson] @saul, not yet... I still need to do extensive testing to make sure that the pandas API integration works as expected. Once this is taken care of, we can add it to xnd.io earlier next week hopefully.
[Travis Oliphant] We need to create a generic array object and a multiple-dispatch mechanism that can seamlessly replace scikit-learn usage of NumPy
[Travis Oliphant] Best guess is next year if funding for Quansight Labs proceeds
[Travis Oliphant] More in the immediate future would be looking to see how projects like dask could use xnd (and then dask-ml)
Who would be willing to look at that.
Also, someone could look at how XND could be used to augment the type-system of Tensorflow.
I suspect Chainer might be the easiest, but Torch and Tensorflow the most useful long-term.
[Travis Oliphant] RAPIDS roadmap --- https://docs.google.com/spreadsheets/d/1TwgYtydaqIf-sHAQhGtCxEHDOTj2nyB_cPJRqtZQZ48/edit#gid=0
[Travis Oliphant] One way to get gufunctions is to use things like RAPIDS and other C++ libraries to build the functionality
[Travis Oliphant] We just need to wrap it with the gufunc concept.
[Travis Oliphant] I was talking to Frederic Bastien and he was commenting how C++ compilers are much better now and C++ makes it easier to write low-level code well.
[Travis Oliphant] XND should not limit gufunc implementations --- but only require a C-ABI to call the resulting code.
[Travis Oliphant] RAPIDS is producing a lot of good c++ code that can be turned into XND gufuncs
[Stefan Krah] XND uses C++ CUDA kernels.
[Stefan Krah] So as long as the kernels have headers with
extern "C" for the C++ functions, all is well.
[Travis Oliphant] Yeah, that’s what I was thinking.
[Stefan Krah] I'll look at RAPIDS, it sounds interesting.
[Pearu Peterson] RAPIDS cuml seems to use C++ interface
[Travis Oliphant] This is a great use-case for gumath --- to provide RAPIDS functionality to xnd containers
[Travis Oliphant] if all of these are gumath functions, we can add these to uscipy
[Travis Oliphant] with the xndarray backend.
[Stefan Krah] I'm essentially using e.g.
thrust::complex (also C++ obviously) by writing C wrapper macros.
[Stefan Krah] So I guess the same would be possible for RAPIDS.
[Travis Oliphant] And very valuable because RAPIDS is getting a lot of attention and 2) we are going to be teaching RAPIDS and can start adding xnd to our teaching
[Stefan Krah] That sounds great.
[Travis Oliphant] I think adding RAPIDS as gufuncs callable by gumath would give us demoable work that we can leverage today
[Travis Oliphant] and get people interested in xnd --- including potential RAPIDS people.
[Travis Oliphant] @stefan.krah if you could wrap a couple of these functions (maybe one from cuML and one from cuGraph) it would help show the way that others can follow-
[Travis Oliphant] This looks interesting: https://notamonadtutorial.com/weld-accelerating-numpy-scikit-and-pandas-as-much-as-100x-with-rust-and-llvm-12ec1c630a1
[Travis Oliphant] Interesting there is no reference to Structured Arrays in NumPy
[Travis Oliphant] which solves exactly this same problem.
[Travis Oliphant] and has a well-established API
[Travis Oliphant] or XND for that matter.