Hey @drahnr I'm blocked on not having a gpu that supports cuda, but I was lucky to get a 1080ti for cheap and it's supposed to arrive tuesday. So I won't be active until then, but should probably be pretty active after that :)
Just keep that in mind.
Once you did a few Para we also have a sharable GPU which also runs the CI checks.
*PRs
But it's mostly there for compat + low end tests
Thanks for letting me know! And no problem! I expect this to be a somewhat high latency process and to be slow going at least at first. Iām used to working with people on the other side of the world from me and people having limited internet :)
Iām planning on trying to implement one of the super resolution GAN models from https://github.com/cszn/KAIR
BatchNormalization2d is the first layer that it uses
So once I follow the process you outlined for adding a new layer to juice and go through the PR and review process I plan on moving through the remainder of the layers that that model uses from pytorch and adding them from rcudnn-sys in juice
Obviously I will do each as a separate PR and follow proper git commit practice
Would you also like me to raise a corresponding issue for each layer before submitting the PR? Or is just the PR sufficient?
I ended up having quite a hectic week with IRL appointments and commitments. So Iāll finally be setting up my desktop with the 1080ti and new power supply tomorrow (Saturday) morning and should be on my way on the code shortly thereafter if all the cuDNN installation etc goes well
Everything is installed and the examples compile! So Iām all set to start now... well Iāll start working on juice tomorrow actually :)
The spearow.io link on the github repo errors with 502 bad gateway. Are you already aware of this?
I guess the default should be the book, or use a subdomain, I have to figure if github can handle this - before I had custom infra which I started to scale down a bit.
What is spearow?
It's a mix of spearou (a Belgian comic) and sparrow (bird)
Cool! Is it meant to be the name of an organization for machine learning?
I like the name a lot!
I started the task of implementing BatchNormalization in rcudnn last night and a thought occurred to me. Might it be easier to auto generate all of the ParamaterNotSupported error strings by parsing them per function from the cuDNN API reference pdf?
Whoops I meant āCUDNN_STATUS_BAD_PARAMā
Hi @drahnr, Iām about halfway through the process you laid out for implementing BatchNormalization in juice but have become stuck with regard to implementing the ādescriptorā step. It seems like the non-LRN/non-divisive normalization cuDNN functions are quite different in their usage API than functions like LRN and Activation functions. Despite spending a solid half an hour reading the cuDNN API reference trying to decipher the correct way to implement this in juice Iām quite confused and stuck. Can we arrange to have a synchronous chat in the next few days so I can hopefully figure this out and proceed with implementing this set of cuDNN functions in juice? Cheers :)
Preferably in the evening hours
2 replies
Could create a draft PR upfront with your current changes?
Sure will do when I get home today!
Iām PST/PDT
Are you available at some point this afternoon or evening? Itās about 14:30 your time right now correct?
Also sorry I didnāt create the PR yet, Iāll do that today