Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Dec 28 2019 21:53

    drahnr on master

    chore/archive: add archive note (compare)

  • Dec 28 2019 21:52

    drahnr on master

    chore/archive: archive this rep… (compare)

  • Dec 28 2019 21:51

    drahnr on master

    chore/archive: closing this repo (compare)

  • Jun 18 2019 16:15
    drahnr commented #16
  • Jun 18 2019 09:44
    andreytkachenko commented #16
  • Apr 04 2019 01:41
    vadixidav opened #18
  • Dec 16 2018 13:44
    Xoronic closed #17
  • Dec 16 2018 13:44
    Xoronic commented #17
  • Dec 11 2018 17:32

    drahnr on master

    fix/README.md: remove link to s… (compare)

  • Dec 11 2018 17:16
    Xoronic edited #17
  • Dec 11 2018 17:16
    Xoronic edited #17
  • Dec 11 2018 17:15
    Xoronic opened #17
  • Aug 31 2018 16:45

    drahnr on master

    chore/version: release 0.2.1 (compare)

  • Apr 07 2018 06:19

    drahnr on master

    doc/badge: adjust path again (compare)

  • Apr 07 2018 06:17

    drahnr on master

    doc/badge: adjust path (compare)

  • Apr 06 2018 22:06

    sirmergealot on gh-pages

    doc/automatic: update (compare)

  • Apr 06 2018 22:04

    sirmergealot on gh-pages

    doc/automatic: update (compare)

  • Apr 06 2018 21:56
    drahnr commented #5
  • Apr 06 2018 21:56
    drahnr commented #5
  • Apr 06 2018 21:53

    drahnr on master

    fix/backends: prevent impl of :… (compare)

subversive-owl
@subversive-owl
worth mentioning, if you have debian, you might run into this: https://bugs.launchpad.net/ubuntu/+source/nvidia-graphics-drivers-390/+bug/1753796
may want to avoid upgrading for a long time
Bernhard Schuster
@drahnr
sigh, why don't they use libglvnd?
I strongly recommend fedora for predictabilities sake
or centos for that sake
Mamy Ratsimbazafy
@mratsim
For pure Debian I build the nvidia driver from SVN: https://wiki.debian.org/NvidiaGraphicsDrivers#Building_newer_releases_from_SVN
I have a setup with Debian as a host and Archlinux as a LXC container with GPU passthrough. So Debian has the nvidia driver and Archlinux has nvidia-smi/nvidia-utils, Cuda and Cudnn drivers and Debian must be kept in sync with Arch.
Recently released
Could someone explain how does it compare to coaster / coaster cnn ?
Coaster NN
Bernhard Schuster
@drahnr
It's more juice related
jonysy
@jonysy
@drahnr How far along are you?
Bernhard Schuster
@drahnr
@jonysy regarding what? I am about to change jobs so things are little more busy than planned
John Karasev
@j.karasev_gitlab
Are there more examples of how to use coaster?
Bernhard Schuster
@drahnr
Hi @j.karasev_gitlab well, depends at what you are aiming
do you want to implement an algorithm for an existing backend?
give more feat to an existing backend
John Karasev
@j.karasev_gitlab
I was thinking of implementing a RNN layer on a existing back end probably on the CPU for now
John Karasev
@j.karasev_gitlab
nevermind, I will just look into the examples in juice
Bernhard Schuster
@drahnr
I think that is the easiest way, try to pick something with as few macros as possible, with the cudnn backend there are quite a few macros involved for implementing over multiple datatypes
Thomas Havlik
@thavlik
Ping
Bernhard Schuster
@drahnr
Pong
John
@onFireForGod_gitlab
64 bytes from coaster: icmp_seq=0 ttl=54 time=20.941 ms
John
@onFireForGod_gitlab
What is the difference between ComputeInputGradient and ComputeParametersGradient traits
John
@onFireForGod_gitlab

for example, the linear layer has:

impl<B: IBackend + LayerOps<f32>> ComputeInputGradient<f32, B> for Linear {
    fn compute_input_gradient(&self,
                              backend: &B,
                              weights_data: &[&SharedTensor<f32>],
                              output_data: &[&SharedTensor<f32>],
                              output_gradients: &[&SharedTensor<f32>],
                              input_data: &[&SharedTensor<f32>],
                              input_gradients: &mut [&mut SharedTensor<f32>]) {
        // Gradient with respect to input data
        backend.gemm(&self.one,
                  Transpose::NoTrans,
                  output_gradients[0],
                  Transpose::NoTrans,
                  weights_data[0],
                  &self.zero,
                  input_gradients[0])
            .unwrap();
    }
}

impl<B: IBackend + LayerOps<f32>> ComputeParametersGradient<f32, B> for Linear {
    fn compute_parameters_gradient(&self,
                                   backend: &B,
                                   output_data: &[&SharedTensor<f32>],
                                   output_gradients: &[&SharedTensor<f32>],
                                   input_data: &[&SharedTensor<f32>],
                                   parameters_gradients: &mut [&mut SharedTensor<f32>]) {
        // gradient w.r.t. weights
        backend.gemm(&self.one,
                  Transpose::Trans,
                  output_gradients[0],
                  Transpose::NoTrans,
                  input_data[0],
                  &self.zero,
                  parameters_gradients[0])
            .unwrap();
    }
}

I am confused on computing gradients here since there is no derivatives of activation functions

Bernhard Schuster
@drahnr
one is with respect to the parameters or weights of layer itself
it other is regarding the input sample given
iirc
@onFireForGod_gitlab
the activation functions are modeled as separate layers in juice
as such there is no point in having them interleaved with this level
(it mmight make sense for performance reasons, but that's yet to be benched and decided)
Geordon Worley
@vadixidav
@drahnr would you be interested in me refactoring coaster to use const generics once const generics are in nightly?
specifically, if tensors could have typed dimensions
like Tensor2<Const<2>, Dynamic>
so each dimension could be a dynamic or constant dimension
then all operations which reshape tensors in in ways where the output size can be determined statically we could have statically verified
Bernhard Schuster
@drahnr
Sounds pretty sweet to me :)
I'd wait another week before you start so I can actually get the one-repo integration done
Do you have a link to the const generics RFC?
Bernhard Schuster
@drahnr
@vadixidav ^^
The only issue that might pop up, is the fact that the backends tend to work with fixed dimensions, as such it might be nice from the interface pov, but not necessarily make it any easier to implement any layer on top of an existing backend
Geordon Worley
@vadixidav
@drahnr I think that anything that builds on top of an underlying interface that uses const generics will be better from a type system perspective
I am more interested in the user improvements than the backend improvements
from that perspective, wrapping coaster in an a const generics api might be sufficient as well
tracking RFC for const generics: rust-lang/rust#44580
unfortunately, it still doesnt look ready enough for something like this, even for initial implementation
when its ready I might also have other interests at the time, but I definitely put this as a high priority because having a pure-rust data-parallelism abstraction over CUDA, opencl, and native is important
and extending the API to support things like knowing the output dimensions of matrix multiplications at compile time is critical
Bernhard Schuster
@drahnr
@vadixidav I understand your point and I for sure would prefer it from an end user API perspective :) I am just trying to convey that this is some heavy lifting and that the internal representation on a per backend basis is and will not be statically determined - CPU based execution might be the exception here