Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Jacob Filik
    @jacobfilik
    where as the hdf5 has the scanned variable and number of points flatterned
    Ok cool, I can play with this, I will pass on what I have learned to the others
    Thanks for you help, I should probably stop now
    Thomas A Caswell
    @tacaswell
    better out of the box AD support has been in our middle-priority to-do list for a while now, but it keeps getting pushed off for other things (and it is hard to push to the top given that we have fixed all of these issues on our beamlines...)
    Jacob Filik
    @jacobfilik
    Yeah, having working beamlines with new shiny features vs better ease of use and docs for external people is not an easy fight
    but you have good support for externals, and that goes a long way
    anyway, really better go before I get in trouble
    Thomas A Caswell
    @tacaswell
    night!
    Jacob Filik
    @jacobfilik
    thanks again!
    Thomas A Caswell
    @tacaswell
    np
    Garrett Bischof
    @gwbischof
    Sorry, I should of mentioned that today is a holiday in the US, I was busy with family stuff. Glad to see that it is working!
    Janine Wezelenburg
    @janinew_be_gitlab
    Hi, anyone who has experience using the publisher-proxy-remotedispatcher ?
    Thomas A Caswell
    @tacaswell
    yes, what is going wrong?
    Janine Wezelenburg
    @janinew_be_gitlab
    :-) I cannot get it to work, and being new to it, I though best to first ask if my jupiter script can actually work
    Thomas A Caswell
    @tacaswell
    I would start trying to get it to work outside of jupyter. jupyter also uses ZMQ to communicate so it is possible that we are not integrating correctly.
    Janine Wezelenburg
    @janinew_be_gitlab
    Ah, that could be it
    I naively thought to use a simple tutorial case and then i.s.o. of subscribing the best effort callback to the RE, to subscribe a publisher and the BEC to the remote
    N.B. as you may guess , I am for the moment more interested in figuring out how to create an event based plan, then to actually capture data
    Thomas A Caswell
    @tacaswell
    that is a good thought, but not one I am 100% sure will work. sorry, :coffee: slowly hitting brain, the other way that this can go sideways is that newer versions of tornado register them selves as ascyio event loops which means you can't start another asyncio event loop in one of its tasks
    Janine Wezelenburg
    @janinew_be_gitlab
    From your reply jupyter may not be the best test environment
    Thomas A Caswell
    @tacaswell
    we did the refactor to make the RE work, but we may not have done the work to refactor the zmq publisher
    What exactly do you mean by "event based plans"?
    Janine Wezelenburg
    @janinew_be_gitlab
    I would see if Bluesky allows to set up a batch run with online adaptive calibration
    Thomas A Caswell
    @tacaswell
    https://github.com/bluesky/bluesky-queueserver <- we are rather rapidly building exactly that right now
    Janine Wezelenburg
    @janinew_be_gitlab
    so I would program the batch parameters, start collectning data, suorcing that data into some compute, which returns a result that would adapt the programming of successive data capture
    Wow, you answer faster than I can type :-)
    The only worry I might have is the event throughput rate of 35K/second
    Thomas A Caswell
    @tacaswell
    What is the context you are doing?
    Bluesky is best thought of as an orchestration / management layer rather than a DAQ layer. This is, if you need to push 35k+ events/s, you probably want to be running that either in a dedicated process or hardware. What bluesky can do for you is to make it easier to manage how you set that other system up, "push the go button", and the collect the whole set of results at the end
    Janine Wezelenburg
    @janinew_be_gitlab
    More industrial than scientific. Automated sampling of (lots of) data with the need to automatically associate the data with all relevant machine and environment parameters
    Thomas A Caswell
    @tacaswell
    it is primarily a "monitoring" rather than a "driving" application?
    Janine Wezelenburg
    @janinew_be_gitlab
    not monitoring in the bluesky sense (lossy), it has both aspects, the objective is monitoring, but to do that correctly there is the driving aspect as well
    if not too much realtime, I thought the bluesky callback and remote compute facility would swat all the flies in one go
    Thomas A Caswell
    @tacaswell
    (side note, the lossy monitoring in due to EPICS (the control system we started against) being maybe lossy on monitors (for very good reason), not bluesky)
    Janine Wezelenburg
    @janinew_be_gitlab
    You can see the advantage if all that can be expressed in bluesky, as it would integrate all actions, flows and recording of the parameters
    Thomas A Caswell
    @tacaswell
    If you can structure it as "set up parameters; set up pipe for big fast event stream to <some place else>; collect the results of that batch seconds-to-minutes later; clean up;" then I think bluesky can help you
    Janine Wezelenburg
    @janinew_be_gitlab
    It would probably look like a faster loop inside the slower loop you suggest. the slow loop to someplace else, the faster loop (10-100ms) for adaptation, a bit like the tutorial with slope adaptation of the sampling to achieve denser sampling around a ROI
    the fast loop with a callback to some local hosted compute
    Thomas A Caswell
    @tacaswell
    we call that sort of thing a "flyer"
    Janine Wezelenburg
    @janinew_be_gitlab
    My impression of the flyer was that it was open loop, or given your remark that bluesky !- DAQ that the fast inner loop would be below bluesky, i.e. part of the device
    At that point I have to confess being lazy ;-) integrating functions at different levels takes more effort, you may agee to have both loops described as bluesky threads/coroutines would make life simple
    I have to go for dinner now, I thank you kindly for the discussion and feedback, really appreciated
    Thomas A Caswell
    @tacaswell
    from what you have said, I think it can be made to work, but I am still not 100% that it is the best fit (but that may be lack of understanding of your problem on my part and a preference to under-promise)
    have a nice dinner!
    Janine Wezelenburg
    @janinew_be_gitlab
    :-) thanks ! Until we meet again, can you give me a pointer how to get started with the queue server, I have cloned the bluesky repos, but experiment with the standard install through anaconda
    Thomas A Caswell
    @tacaswell
    The instructions in the queueserver readme are the best place to start, if they are not undrestandable / complete please report a bug!
    that said, that repo is undergoing rapid development, as in I think the half-life of a line of code is ~5 days
    so if you were going to build a version of it anyway, you might come out ahead, if you want a stable turn-key solution wait a few months
    Janine Wezelenburg
    @janinew_be_gitlab
    Well, perhaps I can contribute, given my intended use case, to make the halflife even shorter ;-)
    Now I am -really- off to dinner. ttyl