A testing pipeline that allows us to run a behavioural phenotyping of our virtual worm running the same test statistics the Schafer Lab used on their worm data.
Hi @cheelee, to answer your question about where to find Rex's 75 MiB WCON file, it is on the google drive as Jim says. The Travis-CI test actually downloads the file, so you can see it here as "from_kerr.wcon"
The Google Drive folder to look at if you want to view it in a file browser is: https://drive.google.com/drive/u/0/folders/0B9dU7zPD0s_LWWlKeGcwT3N4VWc
And @cheelee sorry we didn't get to your question at the meeting about "Is there a kind of context framework to anchor worm features? (e.g. Context = {"food","mating"}.". @JimHokanson can answer this with more detail perhaps but basically we do have a system for "extending" the feature set with dimensions like looking at the worm moving forward or backward, but not for "food" or "mating" specifically.
@MichaelCurrie It's cool, I found the 75Mb file quite soon after the meeting thanks! My question on context was really in response to @Eviatar 's comment about the growing feature space in worm movement analysis in general. I was wondering if there was a way to hierarchically layer features in some kind of a context-based framework. Some of the complexity might go away if we are able to construct some kind of broad temporal transition graph based on upper layer contexts ... for example (and I'm pulling this out of my magic hat by pure imagination) "Near food" transitions to "At food" so there is no need to tag each frame of worm movement features with entries like [ Frame 1: Near food x dorsal turn x 45 degrees ] [Frame2: Near food x ventral turn x 30 degrees] but use hierarchy to store the sequence as:
[Context: Near food]
[Frame 1: dorsal turn x 45 degrees]
[Frame 2: ventral turn x 30 degrees]
[End Context: Near food]
[Context: At food]
...
Anyway I don't know if I understood the problem as described, and I'm not sure if I'm making any sense here. Feel free to ignore the thought or question if it does not make sense :)
[Context: Near food]
[Frame 1: dorsal turn x 45 degrees]
[Frame 2: ventral turn x 30 degrees]
[End Context: Near food]
[Context: At food]
...
Anyway I don't know if I understood the problem as described, and I'm not sure if I'm making any sense here. Feel free to ignore the thought or question if it does not make sense :)
I have to review my code. I think I store the worm's proximity to food. As I recall, if the food lawn is modeled as an ellipse, there is no closed form solution to computing distance of the worm (modeled as a single point) to the food ellipse boundary. But, Newtonian methods iterate fast to solve the problem.
A question of my own. Apologies, it's been ages since I last really used version control (over a decade now). I have multiple updates to the SegWorm code. It's hard for me to discover what's changed since the master branch on GitHub. Should I be forking the code or branching? Once I do either of these, should I just copy my new code onto my clone of the present GitHub code and then commit to see a nice diff of all changes. Or is there a better way? Finally, do I create a pull request to merge this commit to the main GitHub branch?
@eviatar I would 1. clone to your local machine 2. create a branch "ev_revisions" or something 3. delete all existing files 4. copy and paste in all your new files 5. publish this branch 6. browse this branch on github and click "create pull request" 7. Then we can review the changes made, as git will make them visible.
A fork works too, the workflow is similar.
I just granted you membership to OpenWorm and made you a member of the Movement Analysis team so you should now have rights to push the branch at step 5
Trying this using GitHub desktop for Mac. For some reason I can't see my branch nor my commits on the GitHub website. I'm trying to Publish but received an error that one of the files is too big (I had a test.avi at ~250MB). I removed the file, committed the revision but still receive the error when trying to Publish. What should I do? Thanks.
Also, apparently I forked SegWorm. How do I unfork so I can just use branches of the main?
@eviatar if you want to save test.avi somewhere, please feel free to add it to our Google Drive example folder: https://drive.google.com/drive/u/0/folders/0B7to9gBdZEyGNWtWUElWVzVxc0E
@Eviatar on your question about unforking, I would just delete your account's copy of SegWorm as it is now behind openworm:master
I see it here: https://github.com/Eviatar/SegWorm
You can go to settings and in the "danger zone" delete the repo.
Thanks again for the updates! Feel free to use github to push more updates to segworm if it's being actively worked on. Next step is probably to identify areas where the Open Worm Analysis Toolbox needs to be brought into sync with your ~3 years of changes...
@Eviatar I'd appreciate any help you can offer here in identifying what needs to be ported.
thanks!
Necessary* Skillset. I have been told to go he in the main Chat... So how can I be of help, do you have any beginner project/tasks?
For #113, I agree with your suggestion that converting from WCON to BDML will be easier than the reverse, so if you start with that it would be awesome!
You'll probably still have questions - if you do please feel free to set up a quick meeting by google hangouts with me using http://www.meetme.so/MichaelCurrie
Or just chat away on here if you prefer
Cheers! Thanks for your interest!
@MichaelCurrie I have a question about outputing the data as an ordered dictionary... From what I understand the dic cotains subdics, each representing a worm, my question is how those subdictionarys arranged, are the Indexed by timestep (Like BDML)? Or is there some other arrangement? Any way the way to convert the data to BDML is probably by two nested loops, the outer loop loops over the worms and the inner loop loops over the data and generates the XML file.
The biggest issue is the fact that BDML requires a sort of graphical representaition for each object, which means we must create one for each worm, proceduraly. Cronin, Et al. create such a thing by patching together a bunch of lines, I hope Dr. Ken Ho can help us with such a thing.
Hi @ido77778! The format is discussed in detail here https://github.com/openworm/tracker-commons/blob/master/WCON_format.md and the schema is here. The part of the schema relevant to your question is the "data" part.
However, rather than writing a tool to parse WCON files yourself, I highly recommend you use https://github.com/openworm/tracker-commons/tree/master/src/Python or one of the other languages available here: https://github.com/openworm/tracker-commons/tree/master/src.
I wrote the Python version and I can say that you would do something like
import wcon; w = wcon.WCONWorms.load_from_file('test_file.wcon')
then, you can access the worm's data as a pandas dataframe.
just play with the
w object in your python shell to see what's possible
or look at the API here: https://github.com/openworm/tracker-commons/tree/master/src/Python
Your task to complete the issue openworm/tracker-commons#113 would be to write a function that can take such a
w object and output a valid BDML object.
So by doing this, I've saved you half the effort (loading WCON).
So you won't have to puzzle over the nested format of WCON at all.
Hope this helps! Let me know if you need anything else.
Hey @aexbrown @ichoran @ver228 @Eviatar here is OpenWorm senior contributor Balázs Szigeti's omega turns survey: http://groups.inf.ed.ac.uk/worms/index.html
Please consider participating!
- Worm literature is notoriously poor in defining behavior. We have a strong definition for pirouettes at http://www.wormbook.org/chapters/www_behavior/behavior.html#sec7_6
But it's a weird and forced definition.
Worse yet
From Wormbook "Some researchers include omega turns as reversals. And, some researchers count increased backward locomotion as a reversal or response; most do not. A precise definition of the behavior to be scored is critical for analysis."