A place for discussing OgmaNeo https://github.com/ogmacorp/OgmaNeo2
People
Repo info
Activity
Adam
@alienatorZ
the segfault happens at h.setAVLR(1, 0.005)
222464
@222464
the index should be 2, not 1 I think
0 and 1 are both predictions, 2 is actions
Adam
@alienatorZ
Ahhh Yes! That’s what the 1 is for🤦♂️ Thanks!
222464
@222464
no problem!
Gershom
@llucid-97
Hey there, quick question: is it possible to make the agent take actions without the boltman exploration policy (say for a kind of "evaluation mode")
So greedy actions I mean
222464
@222464
@llucid-97 not at the moment, we can add it though!
Gershom
@llucid-97
cool :) I think that'd be really useful
222464
@222464
New release for AOgmaNeo/PyAOgmaNeo is now available (1.5.0). It includes a new function setAExplore(index, true/false) as suggested by @llucid-97 , many many run-time checks in the Python bindings, the ability to set IO layers to type "none" again, and a new ImageEncoder that can be optionally hierarchical itself
sorry that it took so long!
Thanh-Binh
@Thanh-Binh
@222464 congratulation!
Gershom
@llucid-97
Awesome :) can't wait to try it
Gershom
@llucid-97
I upgraded to the current master on AOgmaNeo and pyAOgmaNeo, and when trying either the setAExplore or getAExplore methods, the program crashes with: Process finished with exit code -1073741819 (0xC0000005)
222464
@222464
@llucid-97 thanks for the report, looking into it!
222464
@222464
@llucid-97 should be fixed now - there is a new IO layer indexing scheme internally that broke it
Gershom
@llucid-97
Just tried it out 👍 works great
Thanh-Binh
@Thanh-Binh
@222464 is the current version as goal oriented RL?
222464
@222464
@Thanh-Binh the current master is still regular RL, if you want to try goal oriented learning you can try the "goal_program" branch
Thanh-Binh
@Thanh-Binh
Thank. I will look at this branch
Thanh-Binh
@Thanh-Binh
@222464 do you have any demo for goal oriented? Thx
222464
@222464
@Thanh-Binh Since it is still in development, the demos I have are a bit messy still. I can take one and clean it up a bit though, if you like. I can for instance make Car_Racing work with UBL (goal-oriented) learning by using the RLAdapter
Most tests are in Python though. Car_Racing is still C++
Also, the concept has changed a little but since last time: Instead of "goal states", it has been re-conceptualized as a "program state". A "Program" can be a sequence of 1 or more program states. In order to perform goal-state matching as before, I have included a StateAdapter alongside the RLAdapter. These adapters generate program state sequences for the UBL hierarchy
StateAdapter: Go to a goal state RLAdapter: Perform regular reinforcement learning
Thanh-Binh
@Thanh-Binh
@222464 yes, pls do it, so that i can clean it further if needed! Thanks
Thanh-Binh
@Thanh-Binh
@222464 could you pls send car racing demo to me per email if you finished ! Thanks
Gershom
@llucid-97
Hi there, I've been playing with this library off and on for some toy projects on world modelling. I'm amazed at how well it works given very little data and compute power, but when I try to run it on slightly larger toy problems, it just doesn't perform as well as deep learning.
Clockwork vaes are the closest deep learning based parallel to this, and comparing them on the moving mnist dataset, I can't get Ogma to achieve similar performance
I suspect it's because the deep learning based methods are being given orders of magnitude more compute power, but it got me thinking about scaling Ogma to GPUs. I notice this is something you'd tried before with a previous version. Why did you discontinue that and switch to CPU only?
222464
@222464
@llucid-97 we used to have a GPU version, but we stopped using it mostly because experimentation was annoying when using it. I would be interested in revisiting the moving MNIST dataset again (we used it a long time ago). It should work, if you like I can take a look at your code to see what might improve performance!
222464
@222464
To add to the GPU thing: A GPU version of OgmaNeo requires custom OpenCL/CUDA kernels for just about everything, which is what we used to do, and it slowed down experimentation a bunch. We may revisit it though.
Gershom
@llucid-97
I'm using the same code as in your Gan theft auto example but just applying it to the moving mnist dataset
Ok I thought there would be some hidden caveats to trying to scale things up on GPUs; like maybe since OgmaNeo is not designed to run it batches like deep learning, I thought there'd be some growing pains in trying to get good GPU utilisation of training throughput or something of the sort
222464
@222464
@llucid-97 that's kind of a secondary reason, but still important. GPUs don't play well with sparsity, and a lot of compute is wasted. It does work, though
I will try the moving MNIST digits with the latest version, I am interested to see how it does nowadays (with 1.8.1). I'll let you know of the result in an hour or so!
222464
@222464
@llucid-97 I have implemented the moving MNIST digit example, it seems to work, especially when using a large image encoder
will share after a few more tweaks
The image encoding is the main bottleneck I think
Gershom
@llucid-97
That's awesome, thanks :)
222464
@222464
@llucid-97 here is a paste of the code: https://pastebin.com/tPL3ZSfs It indeed seems to be mostly the ImageEncoder that is limiting it, as unless a very large (and slow) ImageEncoder is made, the digits do not have their identities preserved, only their motions.
222464
@222464
since the data is so short (in terms of sequence length), no hierarchy is needed
This is with a small image encoder. The image encoder doesn't benefit from being sparse itself, as it is the dense->sparse converter basically. This makes it quite slow at sizes > 16x16x16
Guess we need to work on a GPU version again after all :)
Gershom
@llucid-97
That's Interesting. I appreciate you taking the time looking into this. So if I wanted to test that out before diving into trying a full GPU port of this, I could try setting that up with a larger encoder then just running it for a few days or weeks on CPU only just to see if that works
Looking through the code, I don't understand what the sparsity issue would be in running these on GPUs; I mean the sparsity is in activation space only and you're essentially then treating it as a dense array of integers no?
222464
@222464
@llucid-97 the lookup pattern of the weight values is sparse, as they are accessed at out-of-order indices. Hurts cache hits on the CPU, but is still very much worth it there. Now on GPUs I don't know - in the past the GPU versions always felt way slower than they should be. It might be time to try again, though!
I would likely not write it in C++ this time however. Probably going to use Odin and bind OpenCL to it. Python can by bound to that with Cython then.
Odin is a C-replacement language that is very nice to work with. It won't affect the user experience from python anyways, which is how most people use it.