The official gitter channel for the participants of the NIPS Learning to Run Challenge
@unghee_lee_twitter You can check out our last year's osim-rl repository (https://github.com/stanfordnmbl/osim-rl/tree/ver2.1). It is a human model with a passive below-knee prosthesis.
I have an intuition, that full raw observation could be better for agent training.
@Scitator Thanks for sharing your thoughts. You are probably right that it is better for training. However, we designed the current observation dict as it seems closer to biological sensory data used in humans and previous studies (e.g. https://youtu.be/ZkOrRcc4dWg) show that it is possible to control locomotion with those data. We will most likely keep the current setting unless there is some biology-based concern or fundamental limitation in training.
It looks like docker submission way and server submission way return different results?, related issue: stanfordnmbl/osim-rl#199
based on my experiments,
https://www.youtube.com/watch?v=8UmrjVa_Wk
https://www.youtube.com/watch?v=eIMqz4WURj4
you can get ~200+ reward (current 1st place – 166) with agent doing nothing
https://www.youtube.com/watch?v=8UmrjVa_Wk
https://www.youtube.com/watch?v=eIMqz4WURj4
you can get ~200+ reward (current 1st place – 166) with agent doing nothing
@Scitator Thanks for sharing your concerns. It will be easier to track the discussion if we have it at this thread: https://discourse.aicrowd.com/t/reward-function-for-round-2/2032
At that thread, you will find some links about the new reward for Round 2. Have you checked the new reward function out? It is designed in that the best possible reward is much higher than just standing still. Let's continue the discussion at the thread.
At that thread, you will find some links about the new reward for Round 2. Have you checked the new reward function out? It is designed in that the best possible reward is much higher than just standing still. Let's continue the discussion at the thread.
still the same
@Scitator Yes, you still will be able to get high scores by just standing for a while, but will get higher scores by following target velocities for the same time, and much higher scores (in Round 2) by fulfilling the tasks. For example, you will get +500 by reaching and staying at the first target location. I guess the concern is nobody being able to achieve this task?
btw, @SeungmoonS_twitter is there any opportunity to set init_pose from the obs_dict? for curriculum learning
@Scitator You can refer to https://github.com/stanfordnmbl/osim-rl/blob/master/examples/sim_L2M2019_controller1.py for setting initial pose.
Also check out this: https://discourse.aicrowd.com/t/reset-simulator-to-specific-state-e-g-state-previously-seen-during-an-episode/1611/6
As far as the best types of solutions get the highest rewards, we want to leave it up to the participants on how they achieve those solutions, rather than us shaping the reward to guarantee smooth training. I will think of giving higher weights on the velocity-deviation penalty so that walking vs. standing get more different rewards.
Also check out this: https://discourse.aicrowd.com/t/reset-simulator-to-specific-state-e-g-state-previously-seen-during-an-episode/1611/6
As far as the best types of solutions get the highest rewards, we want to leave it up to the participants on how they achieve those solutions, rather than us shaping the reward to guarantee smooth training. I will think of giving higher weights on the velocity-deviation penalty so that walking vs. standing get more different rewards.
The main problem is that current environment use some
projection from the original osim_state. So... my question is, how can I reconstruct the osim state based on this projection?
You probably won't be able to reconstruct the exact osim_state from the observation you get. To get an estimate you could inverse the state->obs process, which you can find here: https://github.com/stanfordnmbl/osim-rl/blob/7b1629c1289d008ef64c542f1c2d4ffaefb82375/osim/env/osim.py#L599-L659
The observation space is defined here: https://github.com/stanfordnmbl/osim-rl/blob/7b1629c1289d008ef64c542f1c2d4ffaefb82375/osim/env/osim.py#L420-L443
The boundary of osim_state (or initial pose) are defined in the .osim file: https://github.com/stanfordnmbl/osim-rl/blob/7b1629c1289d008ef64c542f1c2d4ffaefb82375/osim/models/gait14dof22musc_20170320.osim#L149-L150
The observation space is defined here: https://github.com/stanfordnmbl/osim-rl/blob/7b1629c1289d008ef64c542f1c2d4ffaefb82375/osim/env/osim.py#L420-L443
The boundary of osim_state (or initial pose) are defined in the .osim file: https://github.com/stanfordnmbl/osim-rl/blob/7b1629c1289d008ef64c542f1c2d4ffaefb82375/osim/models/gait14dof22musc_20170320.osim#L149-L150
For training, you can read the osim state directly as discussed here: stanfordnmbl/osim-rl#125
@SeungmoonS_twitter we have an error when trying to submit a solution for this new round. The code provided here : https://github.com/stanfordnmbl/osim-rl/blob/master/examples/submission.py returns a 502 Server Error: Bad Gateway for url: http://osim-rl-grader.aicrowd.com/v1/envs/. Does anybody experience the same issue ? We were able to submit solution for the first round though.
@SeungmoonS_twitter Thank you for the answer ! Concerning submission, I have a little question: when donig a submission are the init_pose pre-defined, random or to be set manually ?
When checking the first obs_dict from the server (when doing submission for round1) we noticed that the character is perfectly straight, while for training we can specify an init pose, and the one provided in the example spinal controller has an initial velocity and non-zero initial joint angles.
@znake77 sorry that I missed your first note. init_pose is set to be the default standing pose. However, each of the five simulations will be initiated with env.reset(...) meaning that init muscle state will be slightly different. The settings will be the same across evaluations, so the score for one controller will be the same across multiple evaluations.
@hanszeng The recommended way to checkin large model weights etc into your submission repository, is to use git-lfs. More on that here : https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/ 20
If you want to try to remove the large offending files from your git history, you can try this: https://help.github.com/en/articles/moving-a-file-in-your-repository-to-git-large-file-storage 3
Hi @luckeciano_gitlab, are you still facing problem with mpi4py? Seems like you managed to install it.
Ah, looks like you made submission by removing it and issue still exist. Let me try getting it installed and update you on the same.
Dear @SeungmoonS_twitter @skbly7 and @spMohanty , could you please check https://gitlab.aicrowd.com/scitator/neurips2019-learn-to-move/issues/10?
"success" became "failed" somehow
"success" became "failed" somehow
Yes, we extended the paper submission due date by one week. Please check the webpage: https://www.aicrowd.com/organizers/stanford-neuromuscular-biomechanics-laboratory/challenges/neurips-2019-learn-to-move-walk-around