Hey @drahnr I'm blocked on not having a gpu that supports cuda, but I was lucky to get a 1080ti for cheap and it's supposed to arrive tuesday. So I won't be active until then, but should probably be pretty active after that :)
Sounds good, I'll have limited internet until Friday
Just keep that in mind.
Once you did a few Para we also have a sharable GPU which also runs the CI checks.
Thanks for letting me know! And no problem! I expect this to be a somewhat high latency process and to be slow going at least at first. I’m used to working with people on the other side of the world from me and people having limited internet :)
BatchNormalization2d is the first layer that it uses
So once I follow the process you outlined for adding a new layer to juice and go through the PR and review process I plan on moving through the remainder of the layers that that model uses from pytorch and adding them from rcudnn-sys in juice
Obviously I will do each as a separate PR and follow proper git commit practice
Would you also like me to raise a corresponding issue for each layer before submitting the PR? Or is just the PR sufficient?
Yeah, adding individual issues would be much appreciated! Thank you I'll try to review your PRs promptly
I ended up having quite a hectic week with IRL appointments and commitments. So I’ll finally be setting up my desktop with the 1080ti and new power supply tomorrow (Saturday) morning and should be on my way on the code shortly thereafter if all the cuDNN installation etc goes well
I started the task of implementing BatchNormalization in rcudnn last night and a thought occurred to me. Might it be easier to auto generate all of the ParamaterNotSupported error strings by parsing them per function from the cuDNN API reference pdf?
Whoops I meant “CUDNN_STATUS_BAD_PARAM”
Technically yes, but I think the effort required to impl such a tool would be very high compared to doing 10 to 100 manual error strings
Hi @drahnr, I’m about halfway through the process you laid out for implementing BatchNormalization in juice but have become stuck with regard to implementing the ‘descriptor’ step. It seems like the non-LRN/non-divisive normalization cuDNN functions are quite different in their usage API than functions like LRN and Activation functions. Despite spending a solid half an hour reading the cuDNN API reference trying to decipher the correct way to implement this in juice I’m quite confused and stuck. Can we arrange to have a synchronous chat in the next few days so I can hopefully figure this out and proceed with implementing this set of cuDNN functions in juice? Cheers :)
Sure, how about Friday or Saturday?
Preferably in the evening hours
Could create a draft PR upfront with your current changes?