set_allocatorline, you get the OOM error (array too big for gpu): https://pastebin.com/LXp1qrf1
i suppose i can try to split my ndarray into sizes that can fit in my gpu memory manually and do several fft calls
This would work, yes. Another hack off top of my head is to create a
cupy.cuda.cufft.PlanNd object yourself, and then call
PlanNd.fft() with your managed-memory-backed array. Warning: this is internal API, it's not stable, and we do not provide documents, so you might have to play around with it to figure out how it works. The simplest way is for you to run a
ifft2 call with a small array, and print the plan cache:
cupy.fft.config.get_plan_cache(). It'd tell you the plan you just created with all arguments you need. Then, you will have a clue for how to create a larger plan.
gpu_array = cupy.array(big_array) plan = cupyx.scipy.fftpack.get_fft_plan(gpu_array) # perform giant 2D FFT in-place plan.fft(gpu_array, gpu_array, 1) # perform giant 2D IFFT in-place plan.fft(gpu_array, gpu_array, -1) back_to_host = cupy.asnumpy(gpu_array)
:tada: Just Released CuPy v9.0.0!
v9.0.0 includes JIT APIs, cuSPARSELt, improved ROCm support and more!
Read the blog for highlights: https://medium.com/cupy-team/cupy-v9-is-here-27e9cbfbf7e5
Or release notes for the full changes: https://github.com/cupy/cupy/releases/tag/v9.0.0
Dear @/all, in a few hours CuPy v9.0.0 will appear on Conda-Forge as well! Note that starting CuPy v9, all satellite packages (cuDNN, cuTENSOR, etc) become optional. If you need to use them, you should do an explicit
conda install. See the installation guide for more information.
btw we are solving an issue on the (optional) cuGraph support for the Conda-Forge package. Once it's resolved I will update here.
Dear @/all, the support for cuGraph is enabled. For Conda-Forge users, it means you can just reinstall CuPy v9.0.0 by
conda install -c conda-forge cupy libcugraph
and the new
cupyx.scipy.sparse.csgraph.connected_components() API will work now!
That's right, unfortunately the infrastructure for aarch64/ppc64le is not ready yet.
If you are interested in helping, please take look at https://conda-forge.org/docs/maintainer/knowledge_base.html?#adding-support-for-a-new-cuda-version, which also applies to supporting a new architecture. Then, in the referred feedstocks you'll find incomplete works that you might be able to pick up. I'd say we're closer to get ppc64le done than it is for arrch64. For this support, though, it's better to continue the discussion in https://gitter.im/conda-forge/conda-forge.github.io instead of here 🙂
da.onesarray on the GPU, and it takes a long time, much longer than just using the CPU)