Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Jinay Jain
    @JinayJain
    Oh, ok! I had noticed that earlier but just thought it was something on my end.
    Anyhow, I'll start working on contributions to the documentation using the GitHub docs folder
    Ali Kheyrollahi
    @aliostad
    @mirraaj would you think you will be able to look into PPO tf => keras?
    Tejasri Nampally
    @tejasri19
    how to solve the error when i insert Flatten() layer to get features from InceptionV3 model. Please help. Traceback (most recent call last): File "extract_features.py", line 73, in <module> model = Model(input=base_model.input, output=base_model.get_layer('custom').output) File "/home/isemes/anaconda3/envs/tensorflow-venv/lib/python3.6/site-packages/keras/engine/network.py", line 364, in get_layer raise ValueError('No such layer: ' + name) ValueError: No such layer: custom
    Ali Kheyrollahi
    @aliostad
    @tejasri19 can you please send a github gist or open an issue with more information? Hard to know what you are trying to do.
    Rohan Raj
    @mirraaj
    Hey @aliostad, you can define keras placeholders using keras.backend.placeholder and continue in the same fashion as OpenAI. While defining the model you can define your input from keras.layer.input. Use this input to make a Keras model from keras.models . You can proceed further to define your function in the defined manner. You may wish to have a look at ACER I implemented few days back. This is under review and will be merged soon.
    Rohan Raj
    @mirraaj
    @JinayJain , You should focus on contributing to the docs available with the project.Please have a look at https://github.com/keras-rl/keras-rl/tree/master/docs.
    The documentation is available at https://keras-rl.readthedocs.io/en/latest/
    Please open a PR once it is ready. We will be happy to merge your contribution with the project.
    @tejasri19 Please elaborate what you have been trying to do. I believe this has something to do with layer naming.
    Tejasri Nampally
    @tejasri19
    @mirraaj @aliostad https://github.com/Gogul09/flower-recognition/blob/master/extract_features.py In this i have added custom layer in Inceptionv3 model. So, i have added custom layer in the inceptionv3 core layer of keras applications folder. Even though i have included in the core layers i get error as no such layer: custom
    Traceback (most recent call last): File "extract_features.py", line 73, in <module> model = Model(input=base_model.input, output=base_model.get_layer('custom').output) File "/home/isemes/anaconda3/envs/tensorflow-venv/lib/python3.6/site-packages/keras/engine/network.py", line 364, in get_layer raise ValueError('No such layer: ' + name) ValueError: No such layer: custom
    Rohan Raj
    @mirraaj
    @tejasri19 , I think it has something to do with your naming. I am not sure what should be the correct way to tackle this. I think you should post your issues under the same repository. The contributors of flower-recognition will definitely help you out. https://github.com/Gogul09/flower-recognition
    Tejasri Nampally
    @tejasri19
    @mirraaj I have done that but no response :(
    gvgramazio
    @gvgramazio
    I don't know if this is the right places for feature requests. I've join this lobby after looking at the contributing guidelines on keras-rl. I want to request a new feature: the possibility to load and store memory. I think that I could implement myself the feature but I don't know your policy about merge requests and the layout of the code in general.
    Rohan Raj
    @mirraaj
    The layout of code is acceptable as long as you are utilising the functions available with the repo. In case you feel that certain code should be reviewed and enhanced, feel free to open a PR.
    gvgramazio
    @gvgramazio
    @mirraaj In this case I enhanced the code by simply adding two new methods to memory class to load/save memory. I used pickle to store the two deque but I'm not sure if it's the better thing to do. I'm not sure of what is the best thing to do: if only store the two deque or to store the whole object (in the latter case pikle doesn't seem the best choice).
    Tejasri Nampally
    @tejasri19
    Hello all! Anyone worked on Resnet50 for classification? How to prevent overfitting and also how to decrease validation loss? My date size is 1000 samples of 10 classes(100 each)
    Rohan Raj
    @mirraaj
    @tejasri19 , I think resnet 50 has high model complexity for your dataset. It lies in the high variance region of figure 6 in the following link (http://scott.fortmann-roe.com/docs/BiasVariance.html) . You should probably decrease your model complexity (model size and parameters).
    Tejasri Nampally
    @tejasri19
    @mirraaj What about inceptionv-3 and vgg19 ? In general, how to decrease validation loss for any model?
    Rohan Raj
    @mirraaj
    As I said, you are in the high variance region for bias/variance trade-off. You better reduce the model complexity. I believe even alexnet might be very complex for your dataset. However, I am not sure about it. One thing you can try is freezing the layers of these models. This will help in reducing total number of learn-able model weights.
    kazhoyan
    @kazhoyan
    Hey guys, sorry if it's a stupid question. I'm new to rl, and especially keras-rl.
    The thing is, I don't have a real Env - it's difficult / impossible to simulate. But I do have a "replay" - consecutive states with their approximate actions and rewards.
    Can this replay be used to train the agent?
    It seems Agent.fit cannot be used directly, because it will try to take exploratory actions, which require a real Environment.
    gvgramazio
    @gvgramazio
    Hi guys, it's possible to easily change the implementation of the DQNAgent in order to use it with multiple actions? Right now it could have only one action in a discrete set of n action. What I want is to have m actions in a discrete set of actions.
    jheffez
    @jheffez
    Hey all, how can we dynamically change (i.e., restrict) the action space available to the keras-rl agent? Let's say that at the beginning there are 4 possible actions (up/down/left/right). Then, at some stage in the simulation (game), there are only two possible actions (left/right). A link/example is appreciated.
    gvgramazio
    @gvgramazio
    @jheffez I don't think there is any problem in the environment. However, I think that with the current agents available, you should use a different NN for each stage. Another possibility is to have different NN for each stage but with a common first half of NN. However it depends on how the stages are different from each other. If the action space is reduced only due to some constraints on the state (e.g. your player could not go left any further because it reached the end of the map) this should be handled by the environment or by a processor of the action.
    jheffez
    @jheffez
    I tried running on colab with gpu as well as a cuda/gpu ubuntu. On both cases the performance compared to cpu was not much. I read someplace that one way to speed up on gpu is by batches. Can someone shed light how to speed up keras-rl on gpu?
    Mahmut Bulut
    @vertexclique
    Hey guys, I am training my own env with DDQN and after training has finished it gives always the same actions continuously.
    but during training I printed out what actions it took and how it's progressing, it is all ok but during execution of test it doesn't apply what it experienced.
    gvgramazio
    @gvgramazio
    @vertexclique I'm not sure to have really understood. Do you mean that during training the agent explore the environment (i.e. tries different actions) but after the training (i.e. in the testing) it always repeat the same action? If this is the case, I suppose that in the testing phase you correctly set the agent (i.e. it is not learning) but that you do not retrieve correctly the weights after the training. Could you ensure that the weights after the training phase are the same of the ones at the beginning and at the end of testing phase?
    Of course, I'm assuming that you are training and testing your agent in two different runs. If you are testing right after training in the same script is very unlikely that my comment is relevant to you.
    Mahmut Bulut
    @vertexclique
    hey @gvgramazio , thanks for the reply. Yes, I am testing in the same script, right after training with a different dataset. Also tried with the same dataset I used to train, but also repeated the same actions.
    gvgramazio
    @gvgramazio
    @vertexclique Except for the new environment, are you using keras-rl without substantial modifications? Could you link the script you're using? If it would have performed bad both in training and testing a lot of things could have caused that but it is very strange that it performs bad only in the testing phase.
    @vertexclique I noticed now that you mentioned the word dataset. What you mean by that? DDQN is an algorithm for RL, not for supervised learning.
    Mahmut Bulut
    @vertexclique
    I am putting some data to set env properly. Different data means different env. Interestingly setting up test_policy to BoltzmannQPolicy helped me to throw not repetitive actions.
    I will send the notebook to you if you want.
    gvgramazio
    @gvgramazio
    @vertexclique What were you using before instead of BoltzmannQPolicy? Anyway, if you have a github/gitlab repository you could post here the link and I can take a look at your code as well as continue the discussion there.
    jheffez
    @jheffez
    Has anyone tried AutoML tools (e.x. Auto-Keras) to improve model/parameters? Is it possible to do?
    yujia21
    @yujia21
    Hi, quick question: in lines 141-146 of core.py (random start steps at the start of each episode for fit function of agents), the action sampled from the environment is processed by the processor, then fed into the step function of the environment. From what I understand the processor should not be applied to the action sampled from the environment as it should already belong to its action space, but applied to the action that the agent outputs (as in lines 169-171). Is this a bug? @RaphaelMeudec @mirraaj
    Stuart Farmer
    @StuartFarmer
    Question- training and testing a DQN in a single run works fine and visualizes properly
    loading weights into a DQN and testing causes no output and action = 0 always
    how do I get around this?
    Stuart Farmer
    @StuartFarmer
    Hey everyone I figured it out
    DQN Agent takes a policy object. If you do not pass the same policy as the test_policy object as well, the model won't work when you load the weights
    gionic
    @giorgionicola
    Hi, I am trying to load the weights of a DDPGAgent previously trained. Although when I test the model does not behave correctly, it seems like the weights are random. What am I missing?
    JaMesLiMers
    @JaMesLiMers
    Hi everyone, I think I found a little bug when I was trying to use ddpg with my custom Processor class. The way that ddpg add metrics and metric_names in Processor is actually adding the function itself into a list, which will cause "TypeError: 'method' object is not subscriptable", hope you guys will see this note and have a check, you may just add "()" at the 228 line "names += self.processor.metrics_names[:]" and 299 line "metrics += self.processor.metrics"
    jamesm-vt
    @jamesm-vt

    Hi all. I just forked this repo to work on some contributions. I just ran the tests to verify my environment before starting and one test consistently fails:

    [gw0] [ 83%] FAILED tests/rl/agents/test_dqn.py::test_naf_layer_full
    Replacing crashed worker gw0

    Thought I'd check if anyone else was seeing the same before I dive into this.

    Yu Jia Cheong
    @yjcheong_gitlab
    @JaMesLiMers if the base class of your processor is the Processor defined in rl/core.py, then metrics_names and metrics have the @property decorator, so self.processor.metrics returns a list and not a function (idem for metrics_names). If you overwrite metrics_names and metrics in your custom class add in the @property decorator and it should work
    jamesm-vt
    @jamesm-vt
    @jamesm-vt It was an environment issue. Resolved now.
    Yahya
    @John-Almardeny
    Hi All, just a quick question please
    What is the purpose of the new episode in the while loop in fit()? it just pass a zero reward to the agent
    so in other words, what is the intuition of passing a zero reward and performing a back-propagation on a zero reward?