Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 17 15:33
    ionox0 opened #1234
  • Jan 17 10:18

    mr-c on 2.0.20200117101530

    (compare)

  • Jan 17 10:15
    mr-c commented #102
  • Jan 17 10:15

    mr-c on yellow-for-optionals

    (compare)

  • Jan 17 10:15

    mr-c on master

    Generate yellow badges for the … Merge pull request #102 from co… (compare)

  • Jan 17 10:15
    mr-c closed #102
  • Jan 17 09:27
    codecov[bot] commented #102
  • Jan 17 09:26
    codecov[bot] commented #102
  • Jan 17 09:25
    codecov[bot] commented #102
  • Jan 17 09:25
    codecov[bot] commented #102
  • Jan 17 09:06
    tom-tan review_requested #102
  • Jan 17 09:02
    tom-tan opened #102
  • Jan 17 08:29

    tom-tan on yellow-for-optionals

    Generate yellow badges for the … (compare)

  • Jan 17 03:33
    illusional edited #1232
  • Jan 17 03:32
    illusional edited #1233
  • Jan 17 03:32
    cwl-bot commented #1233
  • Jan 17 03:31
    illusional opened #1233
  • Jan 17 03:29
    illusional opened #1232
  • Jan 15 18:38

    tetron on 1.0.20190906212748

    (compare)

  • Jan 14 18:20
    bogdang989 opened #893
bogdang989
@bogdang989
Am I missing something or is the CWL v1.1 spec page missing an option to have strings in secondaryFiles? I am looking at https://www.commonwl.org/v1.1/CommandLineTool.html#CommandInputParameter which says it can be SecondaryFileSchema or a list of SecondaryFileSchema but not that elements of the list can also be string/expression. I would expect that there was something in https://www.commonwl.org/v1.1/CommandLineTool.html#SecondaryFileSchema that would mention that use case, but there isn't as well.
I know that the resolving of cwlv1.1 apps works fine for strings in secondaryFiles, but someone looking for the first time would not expect strings to be allowed there
Michael R. Crusoe
@mr-c
@bogdang989 did you check the v1.1 schema salad docs?
I know that isn't the best place for this particular piece of information...
bogdang989
@bogdang989
Thanks a lot @mr-c , this (https://www.commonwl.org/v1.1/SchemaSalad.html#Domain_Specific_Language_for_secondary_files) is exactly the info I was hoping to see in the spec. Would it make sense to somehow add this to https://www.commonwl.org/v1.1/CommandLineTool.html#SecondaryFileSchema (at least a link to the SchemaSalad spec)
Michael R. Crusoe
@mr-c
@bogdang989 you're welcome. I agree, can you open an issue or send a PR?
bogdang989
@bogdang989
:thumbsup:
Peter Amstutz
@tetron
happy new year everyone!
Denis Yuen
@denis-yuen
:+1:
Michael R. Crusoe
@mr-c
w00t!
Ian
@ionox0
:fireworks:
Matthew Garvin
@mattgarvin1

@tetron , @mr-c I really wish we could go back in time and change all the places in the spec where we allow a map and list to just map. It makes writing code so much more cumbersome ...

I see that this issue has been mentioned - I was wondering if there was any plan or initiative to go one way or another with the schema, as far as uniformly requiring the list data structure or uniformly requiring the map structure?

additionally, for all the places in the schema where the option is given for the user to provide either type <T> or an array of type <T> - this seems like another instance where, why not just define the schema such that the type is always an array, and if the user has only one value to put there, then it's a list with one item in it, and that would be fine
Peter Amstutz
@tetron
@mattgarvin1 so technically, the underlying data model only allows the list data structure, the complexity comes because there's a pre-processing layer on top that accepts maps and turns them into lists
@mattgarvin1 and so if you're dealing with CWL as its actually written and not the normalized (i.e. after processing) version then it is more complex
Matthew Garvin
@mattgarvin1
@tetron I guess my question is then, why is it "advertised" to users of CWL that they can choose either data structure? why not just say "this is a list - be sure to use a list."
Peter Amstutz
@tetron
the map version is easier to read
Matthew Garvin
@mattgarvin1
alright, then why not enforce the map data structure for writing CWL, and then have whatever processing tools expect the map and just turn it into a list for further processing
Peter Amstutz
@tetron
well, except then the normalized version would no longer be valid and it becomes an incompatible dialect, there's tradeoffs each way
the next question should be, "so why is the normalized version using lists?" and the answer is "so a normalized CWL document would be valid JSON-LD and convertable to a knowledge graph" which is a cool feature that most users don't care about
Matthew Garvin
@mattgarvin1
I maintain that, regardless of the details, picking one method of representation and sticking to it for writing and processing CWL in this case might be a valuable action to take
re: the maps vs. list issue, as well as the <T> vs. array of <T> issue
allowing for those different representations makes processing CWL preventably clumsier and more painful than it would otherwise be
as other folks have also already pointed out
Kaushik Ghose
@kaushik-work
@tetron I don't know why the map form can't be used for a knowledge graph, but the map form is convertible to a list (and poses no possibility for error, unlike the list, which maybe missing an id field), so I don't understand why this is a blocker.
Also, are we meeting today? Or is it too early in the year?
Michael R. Crusoe
@mr-c
CWL video chat starting soon
Michael R. Crusoe
@mr-c
@mattgarvin1 we're discussing map vs. list form right now at https://meet.jit.si/cwl
Michael R. Crusoe
@mr-c
Evan Clark
@djevo1_gitlab
I have created a poll to see what dates work best for an official NYC CWL Meetup. Please fill in which two dates (Thursday & Friday) work best for you. https://doodle.com/poll/e4dw4ccetg9x536t
Matthew Garvin
@mattgarvin1
sorry, just checking this now, is the video chat still going on?
@mr-c thanks for the notes and sharing the proposal - looks great!
Peter Amstutz
@tetron
@mattgarvin1 the meeting just ended, but I think we addressed your concerns, it won't be a short-term change but could happen in CWL 2.0
Matthew Garvin
@mattgarvin1
@tetron @mr-c sounds good - thanks so much for taking action so promptly and discussing the issue
Andrey Kartashov
@portah
Looks cool especially designed for ML
And Lyft still using Apache Airflow a lot. So, looks like for ML was not enough
Peter Amstutz
@tetron
@mr-c I have a conflicting meeting, I might be late to the CWL video chat today
Michael R. Crusoe
@mr-c
@tetron okay
Might just be me today...
Tom Morris
@tfmorris
Quotes from Gingko Bioworks on CWL, WDL, & worfklow engines (to consider when deciding how to promote/counter) https://youtu.be/arDI64ja6KA?t=2121
Michael R. Crusoe
@mr-c
I think only one of those statements is about CWL directly
(perceived) weaknesses in the ecosystem do challenge the perception of the whole
Peter Amstutz
@tetron
the CWL chat yesterday we talked about communications/marketing and dividing the web site into different sections for different audiences
Michael R. Crusoe
@mr-c
Microsoft is looking for a Senior Scientist (Bioinformatics) with CWL experience https://www.linkedin.com/jobs/view/1683549207/