cupy/community

Leo Fang

@leofang

Hello all! CuPy's latest development branch has changed the website theme. If you have trouble browsing the "latest" docs at https://docs.cupy.dev/en/latest/, @kmaehashi shared a great tip: hold the shift key and then click reload to force cleaning cache. It works for me on Safari, not sure about other browsers.

3 replies

Christopher Mayes

@ChristopherMayes

We had a request a while back for some elliptic integral (special) functions: cupy/cupy#4266 . It turns out that scipy just calls a C function, so I’m wondering how hard it is to port to cupy.

meetaig

@meetaig

Hi! I want to implement skimage's structural_similarity using cupy and I am running into some problems. I am running Linux Mint 18.1 using CUDA 9.0 and anaconda python 3.6. My installation using conda install -c conda-forge cupy seems to have succeeded, as I can execute cupy code just fine. However, I cannot import any filters from cupyx.scipy.ndimage.

from cupyx.scipy.ndimage import uniform_filter, gaussian_filter

gives:

from cupyx.scipy.ndimage import uniform_filter, gaussian_filter
ImportError: cannot import name 'uniform_filter'

I am assuming the version is wrong, because when I execute conda list | grep cupy, I get:
cupy 6.0.0 py36h686fdb1_0

How can I force it to version 8.5 or higher?

Devansh Agarwal

@devanshkv

You can try conda install -c conda-forge cupy=8.5.0 cudatoolkit=9.0

Leo Fang

@leofang

it doesn't make much sense, conda install -c conda-forge cupy should just work to get the latest version. Could you post the output (up to where it prompts for y/n) when you run it?

Devansh Agarwal

@devanshkv

There might be other things in the conda env that are forcing the solver to chose an older version of cupy

Leo Fang

@leofang

that could be possible, then please create a new nev and retry: conda create -n my_new_env -c conda-forge python cupy ...

Kenichi Maehashi

@kmaehashi

:tada: Released CuPy v9.0.0rc1 & v8.6.0!
v9.0.0rc1 includes improved JIT support, generalized universal functions and more!
Also note that v8.6.0 is the final release for v8.x series. Please start testing your workload with v9 pre-releases.
Release notes for v9.0.0rc1: https://github.com/cupy/cupy/releases/tag/v9.0.0rc1
Release notes for v8.6.0: https://github.com/cupy/cupy/releases/tag/v8.6.0

meetaig

@meetaig

Thanks for the suggestions!
I tried installing into a new environment, which did install version 8.5 but got another error:

So I assume there is indeed a conflict in my conda env. The fresh environment defaults to cudatoolkit 11.0. Forcing 9.0 already generates conflicts in the fresh env.

Package cudatoolkit conflicts for:
cudatoolkit=9.0
cupy -> cudatoolkit[version='10.0|10.0.*|10.1|10.1.*|11.0|11.0.*|11.1|11.1.*|11.2|11.2.*|10.2|10.2.*|9.2|9.2.*|>=10.0.130,<10.1.0a0|>=9.2,<9.3.0a0|>=9.0,<9.1.0a0|>=8.0,<9.0a0|8.0.*|9.0.*']
cupy -> cudnn[version='>=8.1.0.77,<9.0a0'] -> cudatoolkit[version='10.0.*|10.1.*|10.2.*|11.*|>=10.1,<10.2|>=9.0,<9.1|>=9.2,<9.3|>=10.0,<10.1|>=10.2,<10.3|9.2.*']The following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.23=0
  - cupy -> __glibc[version='>=2.17|>=2.17,<3.0.a0']

Your installed version is: 2.23

This looks to me that there is a configuration problem. If I read this right it requires glibc >2.17, which I have, but it does not recognize it? I dont know enough about that to tell the difference between linux-64 and cupy here. But it looks like this should not be an issue

I uninstalled cupy again from my own env and then reinstalled. This is the printout before installing:

conda install -c conda-forge cupy
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/user/anaconda3/envs/py36

  added / updated specs:
    - cupy

The following NEW packages will be INSTALLED:

  cudnn              pkgs/main/linux-64::cudnn-7.6.5-cuda9.0_0
  cupy               pkgs/main/linux-64::cupy-6.0.0-py36h686fdb1_0
  fastrlock          conda-forge/linux-64::fastrlock-0.5-py36h831f99a_1
  nccl               pkgs/main/linux-64::nccl-1.3.5-cuda9.0_0

The following packages will be UPDATED:

  certifi            pkgs/main::certifi-2020.12.5-py36h06a~ --> conda-forge::certifi-2020.12.5-py36h5fab9bb_1

The following packages will be SUPERSEDED by a higher-priority channel:

  ca-certificates    pkgs/main::ca-certificates-2021.1.19-~ --> conda-forge::ca-certificates-2020.12.5-ha878542_0
  openssl              pkgs/main::openssl-1.1.1j-h27cfd23_0 --> conda-forge::openssl-1.1.1j-h7f98852_0

If I force to version 8.5 the terminal locks without any printout (most likely because there are too many conflicts)

meetaig

@meetaig

I also tried the new release from pip:

$ python -m pip install cupy-cuda90
Collecting cupy-cuda90
  Downloading cupy_cuda90-8.6.0-cp36-cp36m-manylinux1_x86_64.whl (319.1 MB)
     |████████████████████████████████| 319.1 MB 102 kB/s 
Requirement already satisfied: fastrlock>=0.3 in /home/user/anaconda3/envs/py36/lib/python3.6/site-packages (from cupy-cuda90) (0.6)
Requirement already satisfied: numpy>=1.15 in /home/user/anaconda3/envs/py36/lib/python3.6/site-packages (from cupy-cuda90) (1.18.5)
Installing collected packages: cupy-cuda90
Successfully installed cupy-cuda90-8.6.0

It seems we're getting somewhere, but now I get the error:

Traceback (most recent call last):
  File "/home/user/anaconda3/envs/py36/lib/python3.6/site-packages/cupy/__init__.py", line 20, in <module>
    from cupy import core  # NOQA
  File "/home/user/anaconda3/envs/py36/lib/python3.6/site-packages/cupy/core/__init__.py", line 1, in <module>
    from cupy.core import core  # NOQA
  File "cupy/core/core.pyx", line 1, in init cupy.core.core
  File "/home/user/anaconda3/envs/py36/lib/python3.6/site-packages/cupy/cuda/__init__.py", line 5, in <module>
    from cupy.cuda import compiler  # NOQA
  File "/home/user/anaconda3/envs/py36/lib/python3.6/site-packages/cupy/cuda/compiler.py", line 12, in <module>
    from cupy.cuda import function
  File "cupy/cuda/function.pyx", line 1, in init cupy.cuda.function
  File "cupy/core/_carray.pyx", line 1, in init cupy.core._carray
  File "cupy/core/internal.pyx", line 1, in init cupy.core.internal
  File "cupy/cuda/memory.pyx", line 1, in init cupy.cuda.memory
ModuleNotFoundError: No module named 'fastrlock'

but

$ python -m pip install fastrlock
Requirement already satisfied: fastrlock in /home/user/anaconda3/envs/py36/lib/python3.6/site-packages (0.6)

Praveen Sahu

@povinsahu1909

Hi! I am looking to apply for GSoC'21. I got pretty familiar with contributing guidelines and workflow. This summer, I want to add and improve methods related to the sparse matrix, i.e. sparse matrices and sparse linear algebra module. I would like to know if this work can be done during this GSoC period. And also want to know what details are required to be in the proposal.

Leo Fang

@leofang

@meetaig If you're installing from conda-forge, cudatoolkit=9.0 was never supported. The lowest CUDA version we support is 9.2, but you should be free to choose any version as long as your driver is compatible

so this explains the first part of the errors.

next, in your reinstalling attempt with cupy-cuda90, I think your conda env got corrupted again, so try again with a fresh conda env, and then do pip install cupy-cuda90 see what happens

conda should work fine with wheels from PyPI in general

Masayuki Takagi

@takagi

@povinsahu1909 I’ll contact you later!

ameyachawla99

@ameyachawla99

espected sir/madam

Hi , I am Ameya Chawla currently pursuing my B.Tech in Computer Science and Engineering .
I read about your organization in GSOC 2021 and looking forward to work with your organization .

I have also made a pull request "Implemented the Hyperbolic Sine Function for matrix #4944"

Please can you guide further how should start working .

Please can you share a sample template for the proposal to be made for Google Summer of Code Application..

Thank You

I would love to work on linear algebra functions

Kenichi Maehashi

@kmaehashi

Hi @ameyachawla99, thank you for the interest in participating GSoC! I'd suggest you to read through the Student Guide first: https://google.github.io/gsocguides/student/ . Other resoureces are also available at https://summerofcode.withgoogle.com/help/ .

ameyachawla99

@ameyachawla99

Thank You

Masayuki Takagi

@takagi

@povinsahu1909 As you can see on CuPy's comparison table, many SciPy sparse features are not implemented in CuPy yet. You can choose the ones you think would be most helpful and can be done during the period. We can discuss the candidates you choose in the draft proposal.

As for the proposal's contents, we don't have our proposal template so following the one prescribed by GSoC is fine. We're especially interested in the "Deliverables" part of it.

Praveen Sahu

@povinsahu1909

Thanks for your response. I'll post my draft proposal here in one or two days.

Masayuki Takagi

@takagi

Maybe better to directly send it to me than to post it here. Then I'll share it with dev members!

Praveen Sahu

@povinsahu1909

Okay

Leo Fang

@leofang

The latest CuPy releases v8.6.0 and v9.0.0rc1 are also available on Conda-Forge now, for both Linux and Windows!

meetaig

@meetaig

@leofang Thanks for your feedback! I was able to verify what causes the conflict. Apparently the combination of cudatoolkit==9.0 and python==3.6 does not allow for cupy versions higher than 6.0. When just using python 3.6 I can get the install working with cudatoolkit=10.1, but then the driver makes a problem. I am running on cuda driver 384.130.
Steps to reproduce:

conda create -n cupy_env python=3.6.8 cudatoolkit=9.0
conda activate cupy_env
conda install -c conda-forge cupy

Unfortunately that means I cannot use cupy, as I have the limitations of my system still. I currently cannot upgrade to a newer driver nor cuda version due to other software relying on this configuration.

meetaig

@meetaig

I also tried installing from source, but then I get the message that CUDA 9.0 is too old and CUDA 9.2 is the minimum required version. If that is indeed the case, the documentation probably needs to be updated, as it lists 9.0 as possible. According to https://docs.nvidia.com/deploy/cuda-compatibility/index.html this means I cannot install cupy currently, as the min driver version for 9.2 is 396.26.

    **************************************************
    *** WARNING: CUDA version is too old: 9000
    *** WARNING: CUDA 9.2 or newer is required
 ************************************************************
    * CuPy Configuration Summary                               *
    ************************************************************

    Build Environment:
      Include directories: ['/tmp/pip-req-build-kzlcpuh5/install/../cupy/_core/include/cupy/cub', '/tmp/pip-req-build-kzlcpuh5/install/../cupy/_core/include', '/usr/local/cuda-9.0/include']
      Library directories: ['/usr/local/cuda-9.0/lib64']
      nvcc command       : ['/usr/local/cuda-9.0/bin/nvcc']
      hipcc command      : (not found)

    Environment Variables:
      CFLAGS          : (none)
      LDFLAGS         : (none)
      LIBRARY_PATH    : (none)
      CUDA_PATH       : /usr/local/cuda-9.0
      NVTOOLSEXT_PATH : (none)
      NVCC            : (none)
      HIPCC           : (none)
      ROCM_HOME       : (none)

    Modules:
      cuda      : No (version 9000)
        -> The library is installed but not supported.

    ERROR: CUDA could not be found on your system.
    Please refer to the Installation Guide for details:
    https://docs.cupy.dev/en/stable/install.html

Leo Fang

@leofang

@meetaig If you installed from github master branch, CUDA 9.0 is no longer supported in the upcoming v9 and the v10 dev branches. I would recommend you to investigate the possibility of updating your driver. Also, the reason you can't install cudatoolkit=9.0 from conda-forge is because the lowest cuda we provide is 9.2

so cudatoolkit=9.0 you got is from the default (Anaconda's) channel, which we have no control over

for now, the only solution is for you to build from source: pip install "cupy==8.6.0". This works even if you're using a conda env. But remember to clean up all prior installations.

Leo Fang

@leofang

btw it's legit to always use the latest driver and old cuda (like 9.0), which is how the cuda stuff is expected to work on conda. In fact, on conda-forge it's decided to default to cuda 10.2+ to reduce resource usage, so really we build cupy for cuda 9.2+ just to follow cupy's policy on PyPI, and conda users are encouraged to use a newer cuda if possible: You just need to update your driver and nothing else

Masayuki Takagi

@takagi

To students going to apply to CuPy GSOC: On the student registration form, please prefix your proposal title with "CuPy:" and choose N/A for NumFOCUS proposal tag, which has only ten slots, and CuPy is not on the list. It's there just for filtering proposals easier. Thanks!

ameyachawla99

@ameyachawla99

Thank You for instructions
I have uploaded my draft proposal please can you review it and guide me too improve my proposal

Ryota Yamamura

@oishigyunyu

Hello! I'm considering to perticipate CuPy projects in GSoC2021. But I unconfidence for my programming skils. Is it able contribute CuPy project such me?
And I have a question everyone. How kind of functions is especially needed to implemented in CuPy?Let me know some functions as example.

Masayuki Takagi

@takagi

@ameyachawla99 Thanks for uploading! Your draft is surely on the mentors' dashboard. I'll check it again!

@oishigyunyu Thanks for posting on gitter and your email to me! I'm going to make a reply to you. Also, does anyone has a suggestion for his proposal?

Masayuki Takagi

@takagi

GSoC: I added a note about the capacity for our project on the wiki. As the last year, we will have a student because of our limited resources. Thanks for your understanding.

Ryota Yamamura

@oishigyunyu

@takagi Thank you for your cheking. And does it mean that the student who will be accepted GSoC is only one?

Masayuki Takagi

@takagi

Yes, it's also because of GSoC scheme where students are paid stipends. Of course, if you are interested in contributing to opensources (as opposed to GSoC itsef), it's highly welcome. We can discuss your idea deeper than as usual with your proposal.

Ryota Yamamura

@oishigyunyu

Hello, I'm considering to deal with implementing elliptic function in CuPy routine through discussing with Mr. Takagi.
In the meantime, I have set up sharing of the draft on the GSoC page. The Proposal is not completely written yet, but I will write the details of the elliptic function in chapter 2 and the specific schedule in chapter 3. So, I would like to know what kind of process is required for the implementation of the function. (Code review, testing, etc.)

ameyachawla99

@ameyachawla99

First have a complete understanding of the function
Then try to make pseudo code/algorithm for the function and test it in rough does it work
then implement the function with maximum gpu utilization and test does the output matches the output by the other SciPy/NumPy function then check time taken by your implementation it should be faster than SciPy/NumPy implementaion . If your code doesnt run faster then try to optimize it.

Ryota Yamamura

@oishigyunyu

@ameyachawla99 Thank you for advising me some imformations of developping processes! I will improve my proposal more better!

Ryota Yamamura

@oishigyunyu

Hello. I've written up a whole proposal and was wondering if anyone could give me some advice. Particularly, I'm annoyed by how I plan the project schedule.So I planned the schedule by referencing some proposals in the past years. Thank you.

Masayuki Takagi

@takagi

We understand that it is, in general, hard in software development to estimate the time a task takes. Especially when you're new to the problem you're going to solve. In such a case, I often try to find axes that can be used to divide the entire period smaller. This time, you will have multiple functions, which is an axis. Also, you will implement, test, write documents, make a performance comparison and take a review, which is another axis. With these axes, you can equally divide the task into smaller ones with the same length without thinking of how long the task actually takes.

I will also take a look at your draft soon! Thanks for your effort! (The deadline is approaching...)

Masayuki Takagi

@takagi

@oishigyunyu I've sent you feedback via email!

Ryota Yamamura

@oishigyunyu

@takagi Thanks for the reply! I'll revise my proposal right away!

Masayuki Takagi

@takagi

:thumbsup:

Ryota Yamamura

@oishigyunyu

Hello. I've submitted my proposal in GSoC register page. Thank you for supporting me to write a proposal. I'm looking forward to participating in GSoC 2021 in CuPy project! Thank you.

Masayuki Takagi

@takagi

Thanks for submitting your proposal! @oishigyunyu

Where communities thrive

People

Repo info

Activity