A testing pipeline that allows us to run a behavioural phenotyping of our virtual worm running the same test statistics the Schafer Lab used on their worm data.
:). I also just realized that because Octave has no direct wrapper mechanisms around Python, it forced me to write an intermediate C wrapper for WCON. As a result I think we may have actually gained a C-interface (easily extendable to C++ with name-mangling caveats) to WCON for free!
Hello guys. I have question about the units of the output features. Is the info in open-worm-analysis-toolbox/documentation/database schema/Features Specifications.xlsx correct? I notice that the units of path.curvature while the rest of the features are in degrees? Is this on purpose? Best
Hi Ev!
We have made no changes to the https://github.com/openworm/segworm repository. It is there as a record of the work you did at the Schafer Lab. If you want to make commits to that, please feel free, it is your codebase :)
Since August 2013 Jim and I and others have ported the SegWorm code to Python and made numerous structural changes and improvements. That is now called the "OpenWorm Analysis Toolbox", and it's being used by OpenWorm and André Brown in their respective analysis pipelines. So depending on the kind of changes you want to commit, it may make sense to also create an issue at the https://github.com/openworm/open-worm-analysis-toolbox repository requesting to port your new code.
Hope this helps!
I see in that list they are also listed here as being in Radians/Microns. I'm not sure why the difference between the units used for this feature, and the units used for others like locomotion.crawling_bends.midbody.amplitute, which as you say are in degrees.
I believe we inherited this from the original SegWorm code so I don't have a good reason for you.
Sorry for the delay in responding
To clarify, I think we should aim to separate reversals and regular turns under the curvature measurement.
I've built a lot of code in the intervening years. There is a Worm Features 2 Figures program that takes groups of feature files, delineated as a control group vs. 1 or more groups of experiments and, using the control vs. experiment group(s), outputs all the statistics from the paper, visual representations of these stats , & publication-quality figures. This has been in circulation for the last 2 years and has been used for several papers from the Hobert Lab. I'd like to give this code peer-reviewed representation in a paper. I think the discussion was to incorporate this with work from Andre.
I also built code for food-based features (although it's unclear how these become fully incorporated within the previous statistical framework), enhancing existing experiments of small worms so that they segment appropriately, & a better omega-turn detector. Personally, I'd like to contribute most of these towards a single publication that Andre helms as I'm primarily focused on Clownbow now & see my immediate future work as being less tool-based in general after the development of Clownbow completes. I'd like to switch to several behavioral questions that have been on my mind. As such, it would be great to hand over the code at some point in time to contribute to both OpenWorm & Andre's efforts. Let me know what you think.
@Eviatar that's fantastic you've added those capabilities; is it possible to structure them as a commit to the OpenWorm/SegWorm repository? Then we can clearly see what the additions were. In my opinion that would be the way to proceed, but @JimHokanson might have a different idea.
@Eviatar I've also sent you an invite by email to our monthly Movement Analysis meeting; André usually attends as does Jim. That might be a good way to hash things out as well. It's at 17:00 UTC on 6 May 2016.
Hi @cheelee, to answer your question about where to find Rex's 75 MiB WCON file, it is on the google drive as Jim says. The Travis-CI test actually downloads the file, so you can see it here as "from_kerr.wcon"
The Google Drive folder to look at if you want to view it in a file browser is: https://drive.google.com/drive/u/0/folders/0B9dU7zPD0s_LWWlKeGcwT3N4VWc
And @cheelee sorry we didn't get to your question at the meeting about "Is there a kind of context framework to anchor worm features? (e.g. Context = {"food","mating"}.". @JimHokanson can answer this with more detail perhaps but basically we do have a system for "extending" the feature set with dimensions like looking at the worm moving forward or backward, but not for "food" or "mating" specifically.
@MichaelCurrie It's cool, I found the 75Mb file quite soon after the meeting thanks! My question on context was really in response to @Eviatar 's comment about the growing feature space in worm movement analysis in general. I was wondering if there was a way to hierarchically layer features in some kind of a context-based framework. Some of the complexity might go away if we are able to construct some kind of broad temporal transition graph based on upper layer contexts ... for example (and I'm pulling this out of my magic hat by pure imagination) "Near food" transitions to "At food" so there is no need to tag each frame of worm movement features with entries like [ Frame 1: Near food x dorsal turn x 45 degrees ] [Frame2: Near food x ventral turn x 30 degrees] but use hierarchy to store the sequence as:
[Context: Near food]
[Frame 1: dorsal turn x 45 degrees]
[Frame 2: ventral turn x 30 degrees]
[End Context: Near food]
[Context: At food]
...
Anyway I don't know if I understood the problem as described, and I'm not sure if I'm making any sense here. Feel free to ignore the thought or question if it does not make sense :)
[Context: Near food]
[Frame 1: dorsal turn x 45 degrees]
[Frame 2: ventral turn x 30 degrees]
[End Context: Near food]
[Context: At food]
...
Anyway I don't know if I understood the problem as described, and I'm not sure if I'm making any sense here. Feel free to ignore the thought or question if it does not make sense :)
I have to review my code. I think I store the worm's proximity to food. As I recall, if the food lawn is modeled as an ellipse, there is no closed form solution to computing distance of the worm (modeled as a single point) to the food ellipse boundary. But, Newtonian methods iterate fast to solve the problem.
A question of my own. Apologies, it's been ages since I last really used version control (over a decade now). I have multiple updates to the SegWorm code. It's hard for me to discover what's changed since the master branch on GitHub. Should I be forking the code or branching? Once I do either of these, should I just copy my new code onto my clone of the present GitHub code and then commit to see a nice diff of all changes. Or is there a better way? Finally, do I create a pull request to merge this commit to the main GitHub branch?
@eviatar I would 1. clone to your local machine 2. create a branch "ev_revisions" or something 3. delete all existing files 4. copy and paste in all your new files 5. publish this branch 6. browse this branch on github and click "create pull request" 7. Then we can review the changes made, as git will make them visible.
I just granted you membership to OpenWorm and made you a member of the Movement Analysis team so you should now have rights to push the branch at step 5
Trying this using GitHub desktop for Mac. For some reason I can't see my branch nor my commits on the GitHub website. I'm trying to Publish but received an error that one of the files is too big (I had a test.avi at ~250MB). I removed the file, committed the revision but still receive the error when trying to Publish. What should I do? Thanks.
Also, apparently I forked SegWorm. How do I unfork so I can just use branches of the main?
@eviatar if you want to save test.avi somewhere, please feel free to add it to our Google Drive example folder: https://drive.google.com/drive/u/0/folders/0B7to9gBdZEyGNWtWUElWVzVxc0E
@Eviatar on your question about unforking, I would just delete your account's copy of SegWorm as it is now behind openworm:master
I see it here: https://github.com/Eviatar/SegWorm
You can go to settings and in the "danger zone" delete the repo.
Thanks again for the updates! Feel free to use github to push more updates to segworm if it's being actively worked on. Next step is probably to identify areas where the Open Worm Analysis Toolbox needs to be brought into sync with your ~3 years of changes...
@Eviatar I'd appreciate any help you can offer here in identifying what needs to be ported.
thanks!
Necessary* Skillset. I have been told to go he in the main Chat... So how can I be of help, do you have any beginner project/tasks?
For #113, I agree with your suggestion that converting from WCON to BDML will be easier than the reverse, so if you start with that it would be awesome!
You'll probably still have questions - if you do please feel free to set up a quick meeting by google hangouts with me using http://www.meetme.so/MichaelCurrie
Or just chat away on here if you prefer
Cheers! Thanks for your interest!
@MichaelCurrie I have a question about outputing the data as an ordered dictionary... From what I understand the dic cotains subdics, each representing a worm, my question is how those subdictionarys arranged, are the Indexed by timestep (Like BDML)? Or is there some other arrangement? Any way the way to convert the data to BDML is probably by two nested loops, the outer loop loops over the worms and the inner loop loops over the data and generates the XML file.
The biggest issue is the fact that BDML requires a sort of graphical representaition for each object, which means we must create one for each worm, proceduraly. Cronin, Et al. create such a thing by patching together a bunch of lines, I hope Dr. Ken Ho can help us with such a thing.
Hi @ido77778! The format is discussed in detail here https://github.com/openworm/tracker-commons/blob/master/WCON_format.md and the schema is here. The part of the schema relevant to your question is the "data" part.
However, rather than writing a tool to parse WCON files yourself, I highly recommend you use https://github.com/openworm/tracker-commons/tree/master/src/Python or one of the other languages available here: https://github.com/openworm/tracker-commons/tree/master/src.