Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Raul Bocanegra Algarra
    @raulbocanegra
    anyway let me share the script
    apt-get update && apt-get install -y --no-install-recommends \
        software-properties-common \
        build-essential \
        cmake \
        git \
        libboost-all-dev \
        libfftw3-dev \
        libfreeimage-dev \
        liblapack-dev \
        liblapacke-dev \
        libopenblas-dev
    Yuhong Sun
    @yuhongsun96
    Traceback (most recent call last):
    File "test.py", line 23, in <module>
    profile, index = kv.stomp_self_join(a, m)
    File "/usr/local/lib/python3.6/dist-packages/khiva-0.5.0-py3.6.egg/khiva/matrix.py", line 208, in stomp_self_join
    Exception: stomp_self_join: ArrayFire Exception (Internal error:998):
    In function std::vector<char> cuda::compileToPTX(const char*, std::string)
    In file src/backend/cuda/jit.cpp:
    In function af::array af::join(int, const af::array&, const af::array&)
    In file src/api
    removed the plot, now I see this
    but this is the same as what the notebook says
    so I ran the script, now I should remove opt arrayfire and remove it from ldconfig?
    Raul Bocanegra Algarra
    @raulbocanegra
    git clone --recursive https://github.com/arrayfire/arrayfire.git
    cd arrayfire && \
    git checkout tags/v3.6.2 && \ 
    git apply fix_queue.patch && \
    git apply fix_sort.patch && \
    git apply fix_sparse.patch && \
    mkdir build && cd build && \
    cmake .. -DCMAKE_INSTALL_PREFIX=/opt/arrayfire \
             -DCMAKE_BUILD_TYPE=Release \
             -DAF_BUILD_CPU=ON \
             -DAF_BUILD_CUDA=ON \
             -DAF_BUILD_OPENCL=OFF \
             -DCUDA_cublas_device_LIBRARY=/usr/local/cuda/lib64 \
             -DAF_BUILD_UNIFIED=ON \
             -DAF_WITH_GRAPHICS=OFF \
             -DAF_WITH_NONFREE=OFF \
             -DAF_BUILD_EXAMPLES=OFF \
             -DBUILD_TESTING=ON \
             -DAF_BUILD_DOCS=OFF \
             -DAF_WITH_LOGGING=OFF \
             -DAF_WITH_STATIC_FREEIMAGE=OFF && \
    cmake --build . --target install -- -j8
    yep you should remove /etc/ld.so.conf.d/arrayfire.conf
    and after that remove /opt/arrayfire
    then run the script above that clones arrayfire repo
    there you have the patches you need to apply
    It will take some time, so let's see on Monday
    have a nice weekend
    Yuhong Sun
    @yuhongsun96
    :D have a nice weekend, thanks for spending all this time
    I'll go through those steps and let you know
    Yuhong Sun
    @yuhongsun96
    from khiva.matrix import *
    from khiva.array import *
    from khiva.library import set_backend, KHIVABackend, get_backend
    
    print(get_backend())
    
    stomp_self_join_result = stomp_self_join(
        Array(data=[[10, 10, 11, 11, 10, 11, 10, 10, 11, 11, 10, 11, 10, 10], [
        11, 10, 10, 11, 10, 11, 11, 10, 11, 11, 10, 10, 11, 10]]), 3)
    
    
    print(stomp_self_join_result)

    With this, I see:

    yuhong@yuhong-lnx:~/tmp$ python khiva_self_join.py
    KHIVABackend.KHIVA_BACKEND_CUDA
    Traceback (most recent call last):
    File "khiva_self_join.py", line 9, in <module>
    11, 10, 10, 11, 10, 11, 11, 10, 11, 11, 10, 10, 11, 10]]), 3)
    File "/usr/local/lib/python3.6/dist-packages/khiva-0.5.0-py3.6.egg/khiva/matrix.py", line 208, in stomp_self_join
    Exception: stomp_self_join: ArrayFire Exception (Internal error:998):
    In function std::vector<char> cuda::compileToPTX(const char*, std::string)
    In file src/backend/cuda/jit.cpp:
    In function af::array af::join(int, const af::array&, const af::array&)
    In file src/api

        def test_stomp_self_join(self):
            stomp_self_join_result = stomp_self_join(
                Array(data=[[10, 10, 11, 11, 10, 11, 10, 10, 11, 11, 10, 11, 10, 10], [
                    11, 10, 10, 11, 10, 11, 11, 10, 11, 11, 10, 10, 11, 10]]), 3)
            expected_index = [6, 7, 8, 9, 10, 11, 0, 1, 2, 3, 4, 5, 9, 10, 11, 6, 7, 8, 3, 4, 5, 0, 1, 2]
    
            for i in range(6):
                self.assertAlmostEqual(stomp_self_join_result[0].to_numpy()[0][i], 0.0, delta=1e-2)
                self.assertEqual(stomp_self_join_result[1].to_numpy()[0][i], expected_index[i])

    The above is from matrix_unit_tests.py from the khiva-python bindings unit tests, and this works as shown by a snippet of the output below:

    test_stomp_self_join (main.MatrixTest) ... ok

    Raul Bocanegra Algarra
    @raulbocanegra
    Hi Yuhong, I know you are using Cuda, but could you use the CPU backend for this example while we investigate further?

    just do

    import khiva as kv
    kv.set_backend(kv.KHIVABackend.KHIVA_BACKEND_CPU)

    before the print statement

    Yuhong Sun
    @yuhongsun96
    Ok cool cool, CPU backend works
    I'll just work with this until the docker is released
    :D thanks man
    Antonio Vilches
    @avilchess
    if it works for CPU backend then the Arrayfire and Khiva are properly installed. However, it seems there is a problem between your CUDA installation and Arrayfire that is stopping you from using the most performant implementation of stomp.
    Hint, there is a matrix_profile method that does the same computation than stomp but at higher speed. Hope you enjoy khiva from now on. If you think the library is missing some functionality, please provide us some feedback.
    Yuhong Sun
    @yuhongsun96
    Is it an issue with the Cuda version? Is mine too old?
    yuhong@yuhong-lnx:~/Projects/khiva-use-cases$ nvcc --version
    nvcc: NVIDIA (R) Cuda compiler driver
    Copyright (c) 2005-2019 NVIDIA Corporation
    Built on Wed_Apr_24_19:10:27_PDT_2019
    Cuda compilation tools, release 10.1, V10.1.168
    Is there any other cuda related versions that I should check?
    yuhong@yuhong-lnx:~/Projects/khiva-use-cases$ cat /proc/driver/nvidia/version
    NVRM version: NVIDIA UNIX x86_64 Kernel Module 430.50 Thu Sep 5 22:36:31 CDT 2019
    GCC version: gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
    Antonio Vilches
    @avilchess
    I’m not an expert in Arrayfire-CUDA integration, but if you want a starting point, Arrayfire has a python library. Thus, you could create a notebook, importing arrayfire and try to execute some operations. If it fails, you may prefer to ask Arrayfire team directly. Probably you have found an issue.
    Yuhong Sun
    @yuhongsun96
    Ah ok, very neat :D thanks for the tips
    Antonio Vilches
    @avilchess
    if I’m right please let us know, just in case some new ideas pop our minds.
    Yuhong Sun
    @yuhongsun96
    Hey guys, I'm working on a project with Khiva just to get familiar with it
    I'm not understanding why the clustering is sometimes wrong, perhaps I am using the wrong functionality
    Please just skip to inputs 13-15, in 13 we see that there should be 3 groups, the timeseries within a group are exact translations of one another. In 15, we see an example of a class that isn't grouped correctly
    Thanks ahead of time for the help!
    Antonio Vilches
    @avilchess
    Hi Yuhong, the implementation of k-shape randomly select k timeseries as the centers, the problem with k-shape is that the final clustering depends on that first election, it is something we have to work it out, please try using k-means. K-means doesn’t have that issue.
    Yuhong Sun
    @yuhongsun96
    Oh I see, what other approach is there in khiva that I can make use of for this? doesn't have to be a grouping. For example, if I extract the frequency of peaks etc.
    Do you guys have a wiki of the available functions/API?
    Yuhong Sun
    @yuhongsun96
    I'm thinking of trying by trying to match motifs, would those work if the timeseries was shifted? like if all the values for one timeseries were larger by some amount or some factor
    Yuhong Sun
    @yuhongsun96
    If I wanted to learn the top motif from one sequence and find the best match and distance on another sequence, how would I do that?
    Yuhong Sun
    @yuhongsun96
    Anyway, :D open to ideas, I'm trying to match sequences that have the same period length but that don't necessarily align in time or in scale (their shapes though match exactly). Also any resources for learning Khiva aside from reading source code would be good as well
    Antonio Vilches
    @avilchess
    you ahve the following documentation available: https://khiva-python.readthedocs.io/en/latest/
    let me think a little bit in your use case
    Yuhong Sun
    @yuhongsun96
    Awesome :D It's always fastest to learn from the experts, let me know if you have ideas, I'll code them up