Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Jeff Gentry
    @geoffjentry
    @drdozer Great! If you get to a point where you want to share workflows with others, I’d recommend looking at one of the two resources I referenced above for hosting
    Sergey Svitkov
    @likeanowl

    Hi everyone! :)
    I am trying to contribute to Cromwell (https://github.com/broadinstitute/cromwell) sometimes. Recently, while browsing JIRA for a new issue to work on, I found this one: https://broadworkbench.atlassian.net/browse/BA-5929
    As a fix for this issue I decided to add a BigDecimal type support to Cromwell's WDL parser and came out with the following solution: broadinstitute/cromwell#5135

    I did not knew that any changes related to WDL should be discussed with openWDL maintainers and also did not knew that a change, partially fixing this issue, was already proposed to openWDL: openwdl/wdl#250. However, I think that my changes are still could be useful, because 'BigInt' and 'Long' types do not support operations with floating point numbers "out of the box", but the 'BigDecimal' does.

    Therefore, I want to discuss my proposal of addition a 'BigDecimal' type to openWDL format. I hope that I choosed a correct gitter chat for this :)
    Thanks, have a nice day!

    Jeff Gentry
    @geoffjentry
    Hi @likeanowl - FWIW whether or not it’s a BigDecimal is an implementation detail. For instance, what about MiniWDL, which is written in Python?
    Patrick Magee
    @patmagee
    @likeanowl tha ks for the interest in making some contributions to wdl! As a few people have already mentioned, wdl is no longer under the control of the cromwell team but managed by the openwdl organization. We welcome changes to the specification through our RfC process https://github.com/openwdl/wdl/blob/master/RFC.md. this is the only way that changes may be introduced into the wdl specifications. If you are interested in making changes please check it out!
    SergeiIonin
    @SergeiIonin

    Hi, I'm trying to solve the issue 'Assigning inputs to other inputs in WDL / Cromwell' (https://broadworkbench.atlassian.net/browse/BA-5811, which is currently in backlog). I need some help to figure out how the processing of the single task differs from the processing of the workflow. A lot of efforts were made so far and the issue seems to be tricky. I made some progress in solution of this task and I will appreciate if anybody can give me a quick introduction on how .wdl is parsing in Cromwell (graph construction etc) in case of workflow and task inputs.

    I understand that this question is related to the Cromwell's wdl implementation, but I haven't found the corresponding gitter chat

    Thanks in advance!

    Patrick Magee
    @patmagee
    Hello @SergeiIonin thanks for the question. As this is a question regarding implementation details of an engine and not actually WDL I would suggest that we try to stay away from having any conversation around this here. You can ask the broad team over at https://gatkforums.broadinstitute.org/wdl/categories/ask-the-wdl-team. Or also make a comment on the actual issue in jira
    Denis Yuen
    @denis-yuen
    This section of the forum is no longer actively monitored. We are working on a support migration plan that we will share here shortly. Apologies for this inconvenience. What is the successor, would it be https://github.com/broadinstitute/cromwell/issues or something inside https://support.terra.bio/hc/en-us/community/topics ?
    Jeff Gentry
    @geoffjentry
    For now, for these kinds of questions, the best route is to go via the github issues path. The Terra Support is really for Terra questions. They’re building out a Broad-managed WDL support path but that doesn’t yet exist.
    There does exist a “cromwell slack” but it is completely unofficial and few, if any, Broad folks monitor it. With that glowing endorsement out of the way, if anyone would like an invite let me know
    Denis Yuen
    @denis-yuen
    :+1:
    SergeiIonin
    @SergeiIonin
    @patmagee thanks! I will proceed there
    Jeff Gentry
    @geoffjentry
    @SergeiIonin your best bet is to comment on the JIRA ticket. There is no official chat/message board staffed by the Cromwell developers, so going anywhere else is going to wind up at best leading to someone pointing you to another place
    SergeiIonin
    @SergeiIonin
    @geoffjentry ok, totally agree with you
    Matthew Pocock
    @drdozer
    hey - I've written a scatter that calls the ncbi eutils to fetch a bunch of genbank files
    because it executes in parallel, I got booted for initiating more than 3 requests in a second
    is there some way to tell a scatter to minimise the number of concurrent executions?
    I essentially need to throttle the execution rate to e.g. no more than 2 per second
    Matthew Pocock
    @drdozer
    Ideally I need a command either in the task or the call that steers the parallelism e.g.
    call my_task {
      execution: {
        maximumRate: {
          rate: 3
          per: "second"
        }
      }
      ...
    }
    Matthew Pocock
    @drdozer
    Matthew Pocock
    @drdozer
    Michael Franklin
    @illusional

    Currently the runtime block is probably what you want, technically it's a k-v store so you can put any fields in there you like. However, this is changing very soon: openwdl/wdl#315

    Either way, this is more an engine thing, which Cromwell does support job-rate limiting, but I'll comment on your open/wdl card with more stuff soon.

    Patrick Magee
    @patmagee
    I would agree that's this is more an engine configuration and not a specific language issue. The pr you linked also adds hints which is a way to specify metadata like this to engines which understand it. Imo I do not think the spec should have first class support for this but we shall see how the discussion goes!
    Matthew Pocock
    @drdozer
    Thanks
    Eric T. Dawson
    @edawson
    @drdozer If you still want examples, I have some in this repo and a few more (though some out of date) in this one
    Matthew Pocock
    @drdozer
    mm, getting an exception
    trying to call toUpper on an input variable, getting NoSuchMethodException for WdlStadnardLibraryFunction$$annon$1.toUpper(scala.collection.Seq)
    Matthew Pocock
    @drdozer
    weird - I can't actually see any definiton for toLower or toUpper in the cromwell source
    are these in the spec but not in cromwell?
    Patrick Magee
    @patmagee
    @drdozer they do not exist in the spec eitehr
    Jeff Gentry
    @geoffjentry
    That have to have been hand typed. WdlStadnard…, annon ….
    Matthew Pocock
    @drdozer
    @patmagee Ouch - I'd got the wdl docs muddled up with the ms workflow description language docs -- how embarasing
    Jeff Gentry
    @geoffjentry
    toUpper looks to be a scala string operator
    Chris
    @0x5f3759df_gitlab
    in the scatter gather example, you show how to iterate over an array of files... how do you iterate over a Map[String, File]?
    in a workflow block
    and get both elements
    Michael Franklin
    @illusional
    Chris
    @0x5f3759df_gitlab
    cool thanks.
    Matthew Pocock
    @drdozer
    can I ask cromwell to run a wdl file in an incremental update mode?
    Jeff Gentry
    @geoffjentry
    What do you mean?
    Matthew Pocock
    @drdozer
    @geoffjentry So during development I have a wdl workflow - I run it, then I add another task, then run it again. Ideally it would do the minimal extra work to re-run the workflow by stealing results from the previous run.
    Jeff Gentry
    @geoffjentry
    Gotcha. If you have a persistent database configured or if you’re using Cromwell in server mode w/ an in memory DB and haven’t turned it off in between, it’ll Just Magically Work like that via call caching
    Matthew Pocock
    @drdozer
    thanks - I'll look at that
    Evan Clark
    @djevo1_gitlab
    Anyone here use cromwell and cwl, having this issue with packed cwl Cannot find a tool or workflow with ID 'None'
    Patrick Magee
    @patmagee
    @djevo1_gitlab I woud suggest going over to the cwl channel or cromwell documentation
    Evan Clark
    @djevo1_gitlab
    @patmagee Thanks, is there by chance a cromwell channel?
    Jeff Gentry
    @geoffjentry
    No. There’s a slack but it’s unofficial and there’s no guarantee someone from the team will answer. If that didn’t dissuade you send me an email for you and I can send you an invite. Otherwise your best bet is to open a ticket on github
    Anton Kulaga
    @antonkulaga
    is it active? It is stil better than nothing
    It looks like that cromwell team is mostly focused on cloud API while needs of users with localbackend are often neglected
    Jeff Gentry
    @geoffjentry
    it == the slack channel? Not very, but there are some folks who post there
    The Cromwell team needs to focus on their own priorities first, that’s what pays the bills.