Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 18:53
    w-gao opened #3596
  • 18:49

    w-gao on 2006-fix-aws-docs-ordering

    Fix aws docs numbering broken b… (compare)

  • 16:44
    DailyDreaming synchronize #3572
  • 16:44

    DailyDreaming on 3537-fix-cwl-test-20

    Fix type hints (#3590) Add type hints to misc.py (#359… Update cactus test to fix broke… and 1 more (compare)

  • 16:44
    DailyDreaming synchronize #3595
  • 16:44

    DailyDreaming on 3461-stream-from-cache

    Update cactus test to fix broke… Merge branch 'master' into issu… (compare)

  • 01:18

    DailyDreaming on 3593-update-cactus

    (compare)

  • 01:18

    DailyDreaming on master

    Update cactus test to fix broke… (compare)

  • 01:18
    DailyDreaming closed #3594
  • May 06 18:53

    DailyDreaming on 3593-update-cactus

    Cruft. (compare)

  • May 06 18:53
    DailyDreaming synchronize #3594
  • May 06 17:16
    DailyDreaming synchronize #3594
  • May 06 17:16

    DailyDreaming on 3593-update-cactus

    Add missing req. (compare)

  • May 06 04:57
    DailyDreaming closed #3519
  • May 05 21:02
    adamnovak review_requested #3595
  • May 05 21:02
    adamnovak opened #3595
  • May 05 20:59

    adamnovak on 3461-stream-from-cache

    Handle streaming reads from the… (compare)

  • May 05 20:11
    jonathanxu18 commented #3592
  • May 05 18:42
    adamnovak review_requested #3572
  • May 05 18:41
    adamnovak synchronize #3572
serut
@serut

@ArtRand I want to be able to schedule a task that can be executed only if the mesos resource fits the required attribute. I can't just use the docker image to provide the correct environment to the task success, as the server (mesos agent) is not located on the same "datacenter". We do not want to register Toil with a specific role, as we want to discuss with only 1 Toil that can distribute tasks on the right mesos resource depending of its requirements. So yes, the binary must be present and can will be if Toil uses the right mesos attribute.

I'm sorry to bump that thread again , I really need advice on this one ! :innocent:

Michael R. Crusoe
@mr-c
@serut that ability doesn't exist to day, but it could be written. Do you have engineering resources available?
Arthur Rand
@ArtRand
I may have some time to POC a feature like that, but with the new K8s cluster functionality - adding a new resource requirement might require architectural discussions. I can take a look.
serut
@serut
@mr-c Yes we have engineering resources that we can dedicate to this specific enhancement to make the enhancement for everyone. We need the clear conception from your point of view and the estimated effort, but I think we can add this feature. On another hand, if @ArtRand has some time to POC it to help our engineer to get a good base it would be very nice. We do not have an active Toil contributor so it won't be easy for us to contribute and test it but I don't think this enhacement is too complicated. I think an issue on your tracker would be nice to collaborate on this issue.
Nikhil Kumar
@nikhil
Does anyone know if toil automatically kills its child jobs when the leader is terminated? I see batchsystems have the kill functionality, but I don't see it in use anywhere.
Michael R. Crusoe
@mr-c
@serut That's great to hear! I only contribute to Toil on the CWL side, so I can't comment on the changes needed elsewhere. Opening an issue is a great idea. We can also make it a topic of a future edition of our weekly CWL videos chats. @DailyDreaming is a regular attendee of those.
Peter Amstutz
@tetron
Lon Blauvelt
@DailyDreaming
Ugh, gitter is hooked into our slack service, which is where I usually get notifications. Our bot hasn't been syncing, so Adam and I haven't been getting notifications from here since January. >.<
Michael R. Crusoe
@mr-c
@nikhil I don't know, offhand. Perhaps @DailyDreaming does?
Lon Blauvelt
@DailyDreaming
@nikhil It should. We recommend using either the "toil kill" or "toil destroy-cluster" commands. If you hit a situation where this doesn't happen, it's definitely a bug and should be submitted as an issue.
Lon Blauvelt
@DailyDreaming
[Lon Blauvelt, UCSC GI] New slack integration test.
New slack integration reverse test.
Adam Novak
@adamnovak
"integration"
pvanheus
@pvanheus
@douglowe I found with the 5.2.0 conda package I had to install enlighten manually. did you see anything like this?
1 reply
pvanheus
@pvanheus
heya Toil folks - I've written this workflow, the first step of which is just about processing an input directory full of files to generate a list of lists (of pairs of files): https://gist.github.com/pvanheus/cd4c730ec429741d0e5567b33fb38b85
I run it like toil-cwl-runner --singularity --stats --clusterStats --retryCount=0 --batchSystem slurm --disableCaching --tmpdir-prefix $(pwd)/tmp --tmp-outdir-prefix $(pwd)/tmp --workDir $(pwd)/work --logFile crypt-tb-profiler-toil.log --jobStore $(pwd)/crypticJobStore process_all_reads_tb_profiler.cwl cryptic-input.yml
this is toil 5.2.0 running from conda
and the first step results in all the files in the input directory being copied into ./work/node-913aeead[....] - is there a way to avoid this behaviour? the ExpressionTool I am using is just aiming to transform a Directory to a File[File[]] essentially
pvanheus
@pvanheus
(the directory in question has 12730 files (6365 samples) totally 1.9 TB so copying it is a bit heavyweight :) )
pvanheus
@pvanheus

and the main part of the workflow is now running but there are a lot of

sacct: error: slurm_persist_conn_open: Something happened with the receiving/processing of the persistent connection init message to localhost:6819: Unable to connect to database
sacct: error: slurmdbd: Sending PersistInit msg: No error

errors

Lon Blauvelt
@DailyDreaming
[Adam Novak, UCSC GI] Maybe throwing --linkImports at it would help? That asks the importer to use symlinks when it can.
pvanheus
@pvanheus
ok I'll try that
Lon Blauvelt
@DailyDreaming
@pvanheus I would also try to install Toil from source. We'll do a release soon, but the latest code tries to symlink for CWL where possible: DataBiosphere/toil#3445
pvanheus
@pvanheus
Thanks for the tips... I've added the --linkImports and installed from source (so my toil-cwl-runner is now version 5.3.0a1) and it is still copying everything into the work directory...
pvanheus
@pvanheus
and then after doing so it does some kind of reading of each file (perhaps populating "contents"?). I'm running it on a subset of 200 files to examine behaviour more closely
pvanheus
@pvanheus
oh... it copies each file into the jobStore dir. sigh unfortunately still a lot of copying
after all this is completed though, this version is much better at keeping my cluster busy :)
Lon Blauvelt
@DailyDreaming
@pvanheus That's odd, and I wouldn't expect it to still be copying.
I could try to run it from my end if you have a reproducible workflow that you wanted to make an issue for: https://github.com/DataBiosphere/toil/issues
Vijay Lakhujani
@vlakhujani
the clusterStats option does not produce a json output, am I missing something ?
Lon Blauvelt
@DailyDreaming
@vlakhujani That option could be worded better, as it only works with mesos (and therefore AWS). If using a different cluster, try the --stats option. If using mesos/aws, then let me know because then that's a bug we need to fix.
1 reply
I'll go ahead and change the wording on the option to explain that it only works on mesos/aws.
Lon Blauvelt
@DailyDreaming
[Adam Novak, UCSC GI] Should we throw if we're trying to use it not on Mesos/AWS?
Lon Blauvelt
@DailyDreaming
That's probably a good idea Adam. I'll add that in too.
Douglas Lowe
@douglowe
should the cwltool:overrides: notation work when using toil-cwl-runner? I have an input file containing this, which does what I expect when I try running it with cwltool, but does not when I try using toil-cwl-runner
Peter Amstutz
@tetron
I don't think so, I think that's still a cwltool specific feature. it might show up in a future CWL spec revision
Douglas Lowe
@douglowe
ahh, okay
I'll stop trying to fix my syntax then :-/
Lon Blauvelt
@DailyDreaming
[Lon Blauvelt, UCSC GI] @douglowe We have an issue for this, so it's on the roadmap, but not currently being worked on.
karma29
@karma29:matrix.org
[m]
hi! i'm new to CWL & Toil. can i discuss this issue DataBiosphere/toil#3469 in this channel?
Peter Amstutz
@tetron
@karma29:matrix.org yes that would be appropriate. also https://gitter.im/common-workflow-language/common-workflow-language
Lon Blauvelt
@DailyDreaming
[Adam Novak, UCSC GI] Uh-oh, does the Matrix bridge not talk to the Slack bridge?
Lon Blauvelt
@DailyDreaming
@karma29:matrix.org What's the issue? @adamnovak D:
Lon Blauvelt
@DailyDreaming
[Adam Novak, UCSC GI] Oh, looks like it works.
karma29
@karma29:matrix.org
[m]

hello! just curious about the code here : https://github.com/DataBiosphere/toil/blob/master/src/toil/cwl/cwltoil.py#L736

mutable is by default set to false, which means that there is no downloaded copy of the file (and a link to it is created instead). in the function call examples, there wasn't any explicit declaration of mutable to true, so do they exhibit "streamable" properties? what more changes should we make here?

karma29
@karma29:matrix.org
[m]
actually i'm a gsoc'21 applicant so i wasn't sure if it would be more suitable to reply to the comment threads on github or join the irc. what's recommended though? i'm fine with either! 😃
Lon Blauvelt
@DailyDreaming
[Adam Novak, UCSC GI] We watch Github issues, and the chat here, but not really Github code comments. Either there or here is fine; here is maybe better fro questions that are not themselves bugs.
[Adam Novak, UCSC GI] I think the idea behind CWL streamable is that you will get a pipe (a FIFO) presented to the tool instead of a normal file? I'm not really sure.
Peter Amstutz
@tetron
yes
Lon Blauvelt
@DailyDreaming
[Adam Novak, UCSC GI] Anyway, readGlobalFile produces an ordinary file, with a filename and all the data on disk. It might be given via a symlink, and the lack of mutable means the user code isn't allowed to modify it, as other jobs may be using the same copy.