by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    SP Mohanty
    @spMohanty
    @SeungmoonS_twitter : Was there a doc describing the observations somewhere ?
    Seungmoon Song
    @SeungmoonS_twitter
    @Scitator Hi, Sergey. f, l and v are muscle force, length and velocity.
    Seungmoon Song
    @SeungmoonS_twitter
    Hopefully, these pages/notes give some explanation.
    Ung Hee Lee
    @unghee_lee_twitter
    I tried to use ddpg train_arm.py example to run the walking task; however, the reward is not increasing (stays at ~6). Has anyone tried this approach and achieved better results?
    Ung Hee Lee
    @unghee_lee_twitter
    I’m new to opensim and trying to model active prosthesis with two joints. @SeungmoonS_twitter could you provide me some pointers where to start from ?
    Seungmoon Song
    @SeungmoonS_twitter
    @unghee_lee_twitter You can check out our last year's osim-rl repository (https://github.com/stanfordnmbl/osim-rl/tree/ver2.1). It is a human model with a passive below-knee prosthesis.
    Ung Hee Lee
    @unghee_lee_twitter
    @SeungmoonS_twitter thank you! I’ll look into it.
    Sergey Kolesnikov
    @Scitator
    Hi, @SeungmoonS_twitter @spMohanty @kidzik
    Is it possible to use osim dict observation, rather than proposed projection?
    Seungmoon Song
    @SeungmoonS_twitter
    Hi @Scitator. The evaluations will be done with the dict observation you get with the project=True and obs_as_dict=True setting, I think is what you are calling the proposed projection. Please let us know if you have any concern with it.
    Sergey Kolesnikov
    @Scitator
    @SeungmoonS_twitter I am just wondering if it is possible not to use project=True during evaluation?
    I have an intuition, that full raw observation could be better for agent training.
    Seungmoon Song
    @SeungmoonS_twitter
    @Scitator Thanks for sharing your thoughts. You are probably right that it is better for training. However, we designed the current observation dict as it seems closer to biological sensory data used in humans and previous studies (e.g. https://youtu.be/ZkOrRcc4dWg) show that it is possible to control locomotion with those data. We will most likely keep the current setting unless there is some biology-based concern or fundamental limitation in training.
    brokenBrain
    @brokenBrain
    I'm having trouble understanding the second submission option. The one without the docker container. Can we submit as many times as we want, and will our score be the max of the submissions? Or will it be an average over a certain number of episodes?
    Seungmoon Song
    @SeungmoonS_twitter
    @brokenBrain The final score is the maximum of all submissions.
    Hans Zeng
    @hanszeng
    It looks like docker submission way and server submission way return different results?, related issue: stanfordnmbl/osim-rl#199
    Sergey Kolesnikov
    @Scitator
    Hi, @SeungmoonS_twitter , as the Round 2 is quite close, I have some concern about current reward function ;)
    based on my experiments,
    https://www.youtube.com/watch?v=8UmrjVa_Wk
    https://www.youtube.com/watch?v=eIMqz4WURj4
    you can get ~200+ reward (current 1st place – 166) with agent doing nothing
    Seungmoon Song
    @SeungmoonS_twitter
    @Scitator Thanks for sharing your concerns. It will be easier to track the discussion if we have it at this thread: https://discourse.aicrowd.com/t/reward-function-for-round-2/2032
    At that thread, you will find some links about the new reward for Round 2. Have you checked the new reward function out? It is designed in that the best possible reward is much higher than just standing still. Let's continue the discussion at the thread.
    Sergey Kolesnikov
    @Scitator
    @SeungmoonS_twitter yup, second Round 2 – https://www.youtube.com/watch?v=eIMqz4WURj4
    still the same
    Seungmoon Song
    @SeungmoonS_twitter
    @Scitator Yes, you still will be able to get high scores by just standing for a while, but will get higher scores by following target velocities for the same time, and much higher scores (in Round 2) by fulfilling the tasks. For example, you will get +500 by reaching and staying at the first target location. I guess the concern is nobody being able to achieve this task?
    Sergey Kolesnikov
    @Scitator
    yup, current results looks a bit confusing :)
    btw, @SeungmoonS_twitter is there any opportunity to set init_pose from the obs_dict? for curriculum learning
    Sergey Kolesnikov
    @Scitator
    or good bounds for init_pose? I seem to be lacking osim-specific knowledge :)
    Seungmoon Song
    @SeungmoonS_twitter
    @Scitator You can refer to https://github.com/stanfordnmbl/osim-rl/blob/master/examples/sim_L2M2019_controller1.py for setting initial pose.
    Also check out this: https://discourse.aicrowd.com/t/reset-simulator-to-specific-state-e-g-state-previously-seen-during-an-episode/1611/6
    As far as the best types of solutions get the highest rewards, we want to leave it up to the participants on how they achieve those solutions, rather than us shaping the reward to guarantee smooth training. I will think of giving higher weights on the velocity-deviation penalty so that walking vs. standing get more different rewards.
    Sergey Kolesnikov
    @Scitator
    @SeungmoonS_twitter
    Could you please say the boundaries for the initial pose? Currently there are lack of documentation about it
    The main problem is that current environment use some projection from the original osim_state. So... my question is, how can I reconstruct the osim state based on this projection?
    Seungmoon Song
    @SeungmoonS_twitter
    You probably won't be able to reconstruct the exact osim_state from the observation you get. To get an estimate you could inverse the state->obs process, which you can find here: https://github.com/stanfordnmbl/osim-rl/blob/7b1629c1289d008ef64c542f1c2d4ffaefb82375/osim/env/osim.py#L599-L659
    The observation space is defined here: https://github.com/stanfordnmbl/osim-rl/blob/7b1629c1289d008ef64c542f1c2d4ffaefb82375/osim/env/osim.py#L420-L443
    The boundary of osim_state (or initial pose) are defined in the .osim file: https://github.com/stanfordnmbl/osim-rl/blob/7b1629c1289d008ef64c542f1c2d4ffaefb82375/osim/models/gait14dof22musc_20170320.osim#L149-L150
    For training, you can read the osim state directly as discussed here: stanfordnmbl/osim-rl#125
    znake77
    @znake77
    @SeungmoonS_twitter we have an error when trying to submit a solution for this new round. The code provided here : https://github.com/stanfordnmbl/osim-rl/blob/master/examples/submission.py returns a 502 Server Error: Bad Gateway for url: http://osim-rl-grader.aicrowd.com/v1/envs/. Does anybody experience the same issue ? We were able to submit solution for the first round though.
    Sergey Kolesnikov
    @Scitator
    @SeungmoonS_twitter Hi,
    Is there any starter kit for round 2 submission?
    Seungmoon Song
    @SeungmoonS_twitter
    @znake77 @Scitator For Round 2, we are planning to receive solutions using only docker containers. More details, including the starter kit will be shared soon on our aicrowd page.
    znake77
    @znake77

    @SeungmoonS_twitter Thank you for the answer ! Concerning submission, I have a little question: when donig a submission are the init_pose pre-defined, random or to be set manually ?

    When checking the first obs_dict from the server (when doing submission for round1) we noticed that the character is perfectly straight, while for training we can specify an init pose, and the one provided in the example spinal controller has an initial velocity and non-zero initial joint angles.

    znake77
    @znake77
    @znake77 up
    Seungmoon Song
    @SeungmoonS_twitter
    @znake77 sorry that I missed your first note. init_pose is set to be the default standing pose. However, each of the five simulations will be initiated with env.reset(...) meaning that init muscle state will be slightly different. The settings will be the same across evaluations, so the score for one controller will be the same across multiple evaluations.
    znake77
    @znake77
    Thank you very much @SeungmoonS_twitter
    Hans Zeng
    @hanszeng
    @SeungmoonS_twitter Hi, I cannot upload to the gitlab with error "remote: fatal: pack exceeds maximum allowed size"
    luckeciano
    @luckeciano_gitlab
    Hello guys! Well, I am trying to submit using the docker container but I am facing several problems to build the image... Basically, I can't install mpi4py (a dependency of my repository). Is there anyone else that use MPI and submitted a solution?
    Shivam Khandelwal
    @skbly7

    @hanszeng The recommended way to checkin large model weights etc into your submission repository, is to use git-lfs. More on that here : https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/ 20

    If you want to try to remove the large offending files from your git history, you can try this: https://help.github.com/en/articles/moving-a-file-in-your-repository-to-git-large-file-storage 3

    Hi @luckeciano_gitlab, are you still facing problem with mpi4py? Seems like you managed to install it.
    Ah, looks like you made submission by removing it and issue still exist. Let me try getting it installed and update you on the same.
    Hans Zeng
    @hanszeng
    @skbly7 Thanks for the reply, it works now :)
    Sergey Kolesnikov
    @Scitator
    Dear @SeungmoonS_twitter @skbly7 and @spMohanty , could you please check https://gitlab.aicrowd.com/scitator/neurips2019-learn-to-move/issues/10?
    "success" became "failed" somehow
    Shivam Khandelwal
    @skbly7
    Hey, that’s weird. Let me check it, otherwise will simply requeue at least.
    Valentin Khrulkov
    @KhrulkovV
    Dear @SeungmoonS_twitter, is it possible to extend the deadline for paper submission? Since we are focused on the experiments, there is very little time left to write it.
    Seungmoon Song
    @SeungmoonS_twitter
    Yes, we extended the paper submission due date by one week. Please check the webpage: https://www.aicrowd.com/organizers/stanford-neuromuscular-biomechanics-laboratory/challenges/neurips-2019-learn-to-move-walk-around
    SP Mohanty
    @spMohanty
    Do we have a big crowd from here at NeurIPS this year ? O:-)
    Seungmoon Song
    @SeungmoonS_twitter
    I am :) Are you around, Mohanty?