nobody working on it now?
maybe after I am familiar with the api ill look into it
Yeah, we haven't had the time yet
I think it's a bit tricky to get it right, since you can't really just "test" if a backend works by just using it, since that will lead to a crash.
And I don't know quite enough about cautious runtime checking for availability of libraries.
Hi did you guys resolve the performance issue #13 ? I cant find the rblas overhead test in trunk only the old one https://github.com/autumnai/collenchyma/tree/8b7a7aeeaf67482031da0fd712328f747be09e72/benches
I keep thinking of using Collenchma in my own project ( https://github.com/bklooste/ANNe) , i tried initially but have diverged , we seem on different paths . The key difference is my focus on robotics .. I need 1) Feed back loops / directed graphs with interacting traditional code . 2) Looking at large apps with many interacting nets not single output 3) No GPU but will use multi core SIMD 4) Use different weights including f64 and bytes eg AVX512 can do 64 weights multiply and add in a single op per core ( and even common cpus have >12 cores) . At present it looks like a poor fit. I may use it within layers as its useful to have Cuda for backprop training,
1/2: That should become much easier with Leaf 0.2. There layers become much more composable, and it should be much easier to build dynamic, interacting nets.
3: I don't think that's a big concern (at the moment). Generally SIMD should be handled a layer below Collenchyma by e.g. OpenBLAS. It should only really be a concern when we provide Native implementations ourself, but there I am not sure if the current state of SIMD support in Rust is there yet.
4: Primary concern ATM was getting the overall structure right, and just sticking with f32 throughout made that a lot simpler. In Collenchyma we made at least f64 available where the underlying libraries supported it. In Leaf the datatype of the Layers should become more generic in the future.
I think it maybe best if i use Collenchyma from my layers , i use my own buffer memory manager at the moment . With regard to the other points 3) .. yes and no , SIMD allows very fine grained operations think 100 layers in a graph many very small but has some interesting requirements to achive good performance especially , interweaving / loading of data into simds registers.. And yes at present SIMD is pretty much a c intrinsic call .
My biggest issue is how back propagation needs such wide access that it just goes through so many structures , especially with back ends etc.
Hi all ! I've just learnt about Autumn/Collenchyma. Really nice ! I do have a couple questions of general character (getting lost among all those Framework/Plugins/Hardware/Backend terminology in the guide): how exacly do you abstract over hardware? I mean, there has to be a separate implementation for each computation backend, amirite?
at first I thought there'd be some sort of virtualization but I must have misunderstood
However we try to offload as much of the frameworks specifics as possible into the core of collenchyma so that plugins are as easy to write as possible and should look very similar between different backends
I've also created n-dimensional conv code for native. Finished it last weekend, but the tests incorrectly expected different results, so I had to borrow a friends desktop (which has an nVidia GPU) check what cuDNN generates. I've thus discovered my code was correct. Anyway, I decided to refactor the tests, and thus need to retest on my friends computer, which I can't do till Monday.
Really looking forward to see how you did it :clap:
Thanks! W.r.t. slicing: such an interface would be a good idea anyway, having to manually handle strides was kinda annoying. I've haven't used ndarray, but taking a quick look, I'd say that would be a good direction to take as it provides the features I would have liked to have when I wrote it.
you did some pretty great work there with the conv fixup
(Sorry just saw this!)
I would like to have a quick chat with you about collemchyma this week if you are interested :)
mostly about design decisions that were made and if you are interested in further pushing it
If not, I am interested in putting one together
I see in https://github.com/autumnai/collenchyma-nn/blob/master/README.md#provided-operations none of the OpenCL operations are not implemented. Would that be a good place to start?
actually https://github.com/ratpoison-io/collenchyma-nn and https://github.com/ratpoison-io/collenchyma are a good place to start