(since backprop works well with lenses, I thought I might share)
this space is really awesome
I went to a few of the FHPC talks too, they seem awesome
NPFL is also sort of a data-haskell project
Numerical programming in functional languages
and I think it's going to need submissions! Dominic (idontgetoutmuch) said it got a bit stressful trying to rally people to submit
Primarily, I am realizing that a mutable API is nessecary and I'm wondering if there are good options which exist. I think backprop uses ST to mutate gradients inplace, is that right?
hm, do you mean other libraries that manage mutation and stuff?
yeah, backprop uses ST
but most of the code is polymorphic over PrimMonad
interestingly enough vector-sized, which a lot of my math packages rely on, is blocked on haskell/hackage-server#788
I decided to give backprop a try, but I am really struggling with a basic matrix multiplication test.
Hi! I have tried to better understand how backprop works and have written a short post http://penkovsky.com/post/neural-networks-3/
testMul :: H.L 2 2
testMul = evalBP2 (BP.<>) mk (H.tr mk)
mk :: H.L 2 2
mk = H.build (\a b -> a + b)
getDim :: H.Sized Double s LA.Matrix => s -> (Int, Int)
getDim s =
let m = H.extract s :: LA.Matrix Double
in (LA.rows m, LA.cols m)
main = do
print $ getDim testMul
if not, feel free to ask them again
- To be more precise, I can express all neural network (fully-connected) layers as a list.
Then, the network is:
network :: (Reifies s W) =>
=> BVar s (Acc (Matrix Float)) -- ^ Inputs
-> BVar s [Acc (Matrix Float)] -- ^ Weights
-> BVar s (Acc (Matrix Float)) -- ^ OutputsgradBP network is expected to produce [Acc (Matrix Float)], a list of gradients. Now, in gradient descent-like scenario, one would iteratively subtract that gradients from the initial weights thus obtaining a new list of weights (or equivalent). In terms of Accelerate, a GPU algorithm is constructed. To obtain the result Acc a a function run :: Arrays a => Acc a -> a is applied. How do I represent [Acc (Matrix Float)] as Arrays a => a so that I can take advantage of the backprop library?
i'll look at this deeper, but i think roughly the idea is to give Acc an appropriate
Backprop instance
where add would be "GPU add", to provide another Acc
As for running, you might have to figure out a way to put an [Acc (Matrix Float)] as an instance of Arrays
i feel like this should be a part of the GPU library
sorry, wrong window :)