Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Quansight
    @quansightai_twitter
    [Pearu Peterson] I'll push the code in a few minutes.
    [Saul Shanabrook] IMG_20180712_134331.jpg
    Quansight
    @quansightai_twitter
    [Pearu Peterson] The code is pushed in. Few notes:
    1. to add other C functions that can be used from LLVM IR, these must be added to structinfo_config.c_source string.
    2. I didn't remove pycapsule related code but I believe this can be cleaned out
    Quansight
    @quansightai_twitter

    [Saul Shanabrook] I have enough lower level primitives built up now in the numba support to jit compile a matrix multiply function: https://github.com/Quansight/numba-xnd/blob/8cdd89362aedde4516afedc44b21c5fe7aa9bf3c/tests/test_examples.py

    As you can see, it's very low level, similar to how you would write it in C, but in Python. Now I can start building the higher level APIs on top of that that have some more compile time typing knowledge.

    [Ivan Ogasawara] very nice!
    Quansight
    @quansightai_twitter
    [Saul Shanabrook] @anderson.banihirwe Do you think https://xndframes.readthedocs.io/en/latest/index.html is usable enough to add a link to it from xnd.io?
    Quansight
    @quansightai_twitter
    [Anderson] @saul, not yet... I still need to do extensive testing to make sure that the pandas API integration works as expected. Once this is taken care of, we can add it to xnd.io earlier next week hopefully.
    Raul Maldonado
    @Cloud_Chaoszero_twitter
    Hello, I was wondering what are the latest efforts of embedding or using ML with XND. (e.g. use XND within sci-kit learn)
    Quansight
    @quansightai_twitter
    [Travis Oliphant] We are still a ways off from that. The recent uarray efforts in Quansight-Labs are about creating an interface that could enable scikit-learn to use xnd
    [Travis Oliphant] We need to create a generic array object and a multiple-dispatch mechanism that can seamlessly replace scikit-learn usage of NumPy
    [Travis Oliphant] Best guess is next year if funding for Quansight Labs proceeds
    [Travis Oliphant] More in the immediate future would be looking to see how projects like dask could use xnd (and then dask-ml)
    Travis E. Oliphant
    @teoliphant
    Hey all. A good way forward, I think would be to look at how xnd could be used to live underneath the new C++ tensors PyTorch uses as well as the C++ library that Chainer uses.
    Who would be willing to look at that.
    Also, someone could look at how XND could be used to augment the type-system of Tensorflow.
    I suspect Chainer might be the easiest, but Torch and Tensorflow the most useful long-term.
    Quansight
    @quansightai_twitter
    [Travis Oliphant] Another really good use-case for xnd is xarray --- you could have an xnd backend to xarray.
    Quansight
    @quansightai_twitter
    [Travis Oliphant] One way to get gufunctions is to use things like RAPIDS and other C++ libraries to build the functionality
    [Travis Oliphant] We just need to wrap it with the gufunc concept.
    [Travis Oliphant] I was talking to Frederic Bastien and he was commenting how C++ compilers are much better now and C++ makes it easier to write low-level code well.
    [Travis Oliphant] XND should not limit gufunc implementations --- but only require a C-ABI to call the resulting code.
    [Travis Oliphant] RAPIDS is producing a lot of good c++ code that can be turned into XND gufuncs
    [Stefan Krah] XND uses C++ CUDA kernels.
    Quansight
    @quansightai_twitter
    [Travis Oliphant] We should put an XND interface to RAPIDS
    [Stefan Krah] So as long as the kernels have headers with extern "C" for the C++ functions, all is well.
    [Travis Oliphant] Yeah, that’s what I was thinking.
    [Stefan Krah] I'll look at RAPIDS, it sounds interesting.
    [Pearu Peterson] RAPIDS cuml seems to use C++ interface
    Quansight
    @quansightai_twitter
    [Travis Oliphant] https://github.com/RAPIDSai
    [Travis Oliphant] This is a great use-case for gumath --- to provide RAPIDS functionality to xnd containers
    [Travis Oliphant] if all of these are gumath functions, we can add these to uscipy
    [Travis Oliphant] with the xndarray backend.
    [Stefan Krah] I'm essentially using e.g. thrust::complex (also C++ obviously) by writing C wrapper macros.
    [Stefan Krah] So I guess the same would be possible for RAPIDS.
    Quansight
    @quansightai_twitter
    [Travis Oliphant] Yes.
    [Travis Oliphant] And very valuable because RAPIDS is getting a lot of attention and 2) we are going to be teaching RAPIDS and can start adding xnd to our teaching
    [Stefan Krah] That sounds great.
    [Travis Oliphant] I think adding RAPIDS as gufuncs callable by gumath would give us demoable work that we can leverage today
    [Travis Oliphant] and get people interested in xnd --- including potential RAPIDS people.
    [Travis Oliphant] @stefan.krah if you could wrap a couple of these functions (maybe one from cuML and one from cuGraph) it would help show the way that others can follow-
    Quansight
    @quansightai_twitter
    [Stefan Krah] Ok, I can do that.
    Quansight
    @quansightai_twitter
    [Travis Oliphant] We can show off these interfaces to RAPIDS at the SciPy tutorial Anthony is giving
    Quansight
    @quansightai_twitter
    [Travis Oliphant] Interesting there is no reference to Structured Arrays in NumPy
    [Travis Oliphant] which solves exactly this same problem.
    [Travis Oliphant] and has a well-established API
    [Travis Oliphant] or XND for that matter.