Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Adam
    @alienatorZ
    the segfault happens at h.setAVLR(1, 0.005)
    222464
    @222464
    the index should be 2, not 1 I think
    0 and 1 are both predictions, 2 is actions
    Adam
    @alienatorZ
    Ahhh Yes! That’s what the 1 is for🤦‍♂️ Thanks!
    222464
    @222464
    no problem!
    Gershom
    @llucid-97
    Hey there, quick question: is it possible to make the agent take actions without the boltman exploration policy (say for a kind of "evaluation mode")
    So greedy actions I mean
    222464
    @222464
    @llucid-97 not at the moment, we can add it though!
    Gershom
    @llucid-97
    cool :) I think that'd be really useful
    222464
    @222464
    New release for AOgmaNeo/PyAOgmaNeo is now available (1.5.0). It includes a new function setAExplore(index, true/false) as suggested by @llucid-97 , many many run-time checks in the Python bindings, the ability to set IO layers to type "none" again, and a new ImageEncoder that can be optionally hierarchical itself
    sorry that it took so long!
    Thanh-Binh
    @Thanh-Binh
    @222464 congratulation!
    Gershom
    @llucid-97
    Awesome :) can't wait to try it
    Gershom
    @llucid-97
    I upgraded to the current master on AOgmaNeo and pyAOgmaNeo, and when trying either the setAExplore or getAExplore methods, the program crashes with:
    Process finished with exit code -1073741819 (0xC0000005)
    222464
    @222464
    @llucid-97 thanks for the report, looking into it!
    222464
    @222464
    @llucid-97 should be fixed now - there is a new IO layer indexing scheme internally that broke it
    Gershom
    @llucid-97
    Just tried it out 👍 works great
    Thanh-Binh
    @Thanh-Binh
    @222464 is the current version as goal oriented RL?
    222464
    @222464
    @Thanh-Binh the current master is still regular RL, if you want to try goal oriented learning you can try the "goal_program" branch
    Thanh-Binh
    @Thanh-Binh
    Thank. I will look at this branch
    Thanh-Binh
    @Thanh-Binh
    @222464 do you have any demo for goal oriented? Thx
    222464
    @222464
    @Thanh-Binh Since it is still in development, the demos I have are a bit messy still. I can take one and clean it up a bit though, if you like. I can for instance make Car_Racing work with UBL (goal-oriented) learning by using the RLAdapter
    Most tests are in Python though. Car_Racing is still C++
    Also, the concept has changed a little but since last time: Instead of "goal states", it has been re-conceptualized as a "program state". A "Program" can be a sequence of 1 or more program states. In order to perform goal-state matching as before, I have included a StateAdapter alongside the RLAdapter. These adapters generate program state sequences for the UBL hierarchy
    StateAdapter: Go to a goal state
    RLAdapter: Perform regular reinforcement learning
    Thanh-Binh
    @Thanh-Binh
    @222464 yes, pls do it, so that i can clean it further if needed! Thanks
    Thanh-Binh
    @Thanh-Binh
    @222464 could you pls send car racing demo to me per email if you finished ! Thanks
    Gershom
    @llucid-97
    Hi there, I've been playing with this library off and on for some toy projects on world modelling. I'm amazed at how well it works given very little data and compute power, but when I try to run it on slightly larger toy problems, it just doesn't perform as well as deep learning.
    Clockwork vaes are the closest deep learning based parallel to this, and comparing them on the moving mnist dataset, I can't get Ogma to achieve similar performance
    I suspect it's because the deep learning based methods are being given orders of magnitude more compute power, but it got me thinking about scaling Ogma to GPUs. I notice this is something you'd tried before with a previous version. Why did you discontinue that and switch to CPU only?
    222464
    @222464
    @llucid-97 we used to have a GPU version, but we stopped using it mostly because experimentation was annoying when using it. I would be interested in revisiting the moving MNIST dataset again (we used it a long time ago). It should work, if you like I can take a look at your code to see what might improve performance!
    222464
    @222464
    To add to the GPU thing: A GPU version of OgmaNeo requires custom OpenCL/CUDA kernels for just about everything, which is what we used to do, and it slowed down experimentation a bunch. We may revisit it though.
    Gershom
    @llucid-97
    I'm using the same code as in your Gan theft auto example but just applying it to the moving mnist dataset
    Ok I thought there would be some hidden caveats to trying to scale things up on GPUs; like maybe since OgmaNeo is not designed to run it batches like deep learning, I thought there'd be some growing pains in trying to get good GPU utilisation of training throughput or something of the sort
    222464
    @222464
    @llucid-97 that's kind of a secondary reason, but still important. GPUs don't play well with sparsity, and a lot of compute is wasted. It does work, though
    I will try the moving MNIST digits with the latest version, I am interested to see how it does nowadays (with 1.8.1). I'll let you know of the result in an hour or so!
    222464
    @222464
    @llucid-97 I have implemented the moving MNIST digit example, it seems to work, especially when using a large image encoder
    will share after a few more tweaks
    The image encoding is the main bottleneck I think
    Gershom
    @llucid-97
    That's awesome, thanks :)
    222464
    @222464
    @llucid-97 here is a paste of the code: https://pastebin.com/tPL3ZSfs It indeed seems to be mostly the ImageEncoder that is limiting it, as unless a very large (and slow) ImageEncoder is made, the digits do not have their identities preserved, only their motions.
    222464
    @222464
    since the data is so short (in terms of sequence length), no hierarchy is needed
    moving_mnist.gif
    This is with a small image encoder. The image encoder doesn't benefit from being sparse itself, as it is the dense->sparse converter basically. This makes it quite slow at sizes > 16x16x16
    Guess we need to work on a GPU version again after all :)
    Gershom
    @llucid-97
    That's Interesting. I appreciate you taking the time looking into this. So if I wanted to test that out before diving into trying a full GPU port of this, I could try setting that up with a larger encoder then just running it for a few days or weeks on CPU only just to see if that works
    Looking through the code, I don't understand what the sparsity issue would be in running these on GPUs; I mean the sparsity is in activation space only and you're essentially then treating it as a dense array of integers no?
    222464
    @222464
    @llucid-97 the lookup pattern of the weight values is sparse, as they are accessed at out-of-order indices. Hurts cache hits on the CPU, but is still very much worth it there. Now on GPUs I don't know - in the past the GPU versions always felt way slower than they should be. It might be time to try again, though!
    I would likely not write it in C++ this time however. Probably going to use Odin and bind OpenCL to it. Python can by bound to that with Cython then.
    Odin is a C-replacement language that is very nice to work with. It won't affect the user experience from python anyways, which is how most people use it.