Also, that version of cwltool is 9 months out of date..
and when it works the output is much nicer:
$ cwltool --make-template https://dockstore.org:443/api/api/ga4gh/v2/tools/quay.io%Fwtsicgp%Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor/%FDockstore.cwl
/home/mcrusoe/cwltool/env3/bin/cwltool 1.0.20181217162649
sample: a_string # type "string"
reference: # type "File"
class: File
path: a/file/path
mmqcfrac: 0.05 # default value of type "float". (optional)
mmqc: false # type "boolean"
groupinfo: # type "File" (optional)
class: File
path: a/file/path
bwa_idx: # type "File"
class: File
path: a/file/path
bwa: ' -Y -K 100000000' # default value of type "string". (optional)
bams_in: # array of type "File" (optional)
- class: File
path: a/file/path
An interesting issue here:
I have a workflow registerd on Dockstore, it runs fine with this command:
dockstore workflow launch --entry github.com/cancerit/workflow-seq-import/chksum_seqval_wf_interleaved_fq:0.4.0 --json chksum_seqval_wf_interleaved_fq_with_headers.jsonit'll fail if I run it like below:
cwl-runner https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/%2Fcwls%2Fchksum_seqval_wf_interleaved_fq.cwl chksum_seqval_wf_interleaved_fq_with_headers.json
Tool definition failed initialization:
Tool definition https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl failed validation:
https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl:27:1: checking field `steps`
https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl:28:3: checking object `https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl#if_file_name_is_bz2_then_return_null_else_return_in_json_to_output`
https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl:35:5: Field `run` contains undefined reference to `https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_file_name_is_bz2_then_return_null_else_return_in_json_to_output.cwl`
too old?
Not Found : version found, but file not found (bad filename, invalid file, etc.)
Looks like an issue with dockstore.org itself
One other thought curl -X GET "https://dockstore.org/api/api/ga4gh/v2/tools/%2523workflow%252Fgithub.com%252Fcancerit%252Fworkflow-seq-import%252Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/CWL/files" -H "accept: application/json" gives a list of files that we're aware of
https://dockstore.org/api/api/ga4gh/v2/tools/%2523workflow%252Fgithub.com%252Fcancerit%252Fworkflow-seq-import%252Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/PLAIN_CWL/descriptor/if_file_name_is_bz2_then_return_null_else_return_in_json_to_output.cwl
is listed but the CWL (output) seems to refer to a toolkit directory
is listed but the CWL (output) seems to refer to a toolkit directory
Skipping dockstore.org, everything seems to validate okay: https://view.commonwl.org/workflows/github.com/cancerit/workflow-seq-import/blob/0.4.0/cwls/chksum_seqval_wf_interleaved_fq.cwl
l$ cwltool --validate https://github.com/cancerit/workflow-seq-import/raw/0.4.0/cwls/chksum_seqval_wf_interleaved_fq.cwl
/home/michael/cwltool/env3/bin/cwltool 1.0.20181217162649
URI prefix 'foaf' of 'foaf:name' not recognized, are you missing a $namespaces section?
URI prefix 'foaf' of 'foaf:mbox' not recognized, are you missing a $namespaces section?
URI prefix 'foaf' of 'foaf:name' not recognized, are you missing a $namespaces section?
URI prefix 'foaf' of 'foaf:mbox' not recognized, are you missing a $namespaces section?
URI prefix 'foaf' of 'foaf:name' not recognized, are you missing a $namespaces section?
URI prefix 'foaf' of 'foaf:mbox' not recognized, are you missing a $namespaces section?
https://github.com/cancerit/workflow-seq-import/raw/0.4.0/cwls/chksum_seqval_wf_interleaved_fq.cwl is valid CWL.
Yeah, I think it looks like an indexing error. For some reason, it looks like toolkit/if_input_is_bz2_convert_to_gz_else_just_rename.cwl and toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl are being indexed in the toolkit directory. if_file_name_is_bz2_then_return_null_else_return_in_json_to_output.cwl is being indexed in the pwd, possibly because its an import in an import. What I can't explain is why it launches properly using the very first command but I think its still a bug one way or another. I'll create an issue for it
When it's fixed, will this registry be fixed automatically?
Created ga4gh/dockstore#2154 , so chances are if its an indexing error. We'll need to fix the refresh behaviour and then you'll need to hit the refresh button when we do a release
I'm having difficulty explaining why the first launch command works since it uses cwltool under the hood, so there may be something more complicated I'm missing. But that's my first explaination
We are working towards to have all the endpoints
We have done some changes to the API and probably a PR will come next week
we have extend the classification to container image to provide more details about the tipe of container
like conda package, or singularity or Docker
this will allow us to connect to workflows that use conda for example
make sense?
lot of work going on now in our UI
@denis-yuen can I attend to the Monday meeting, is that remotly I guess
?
https://github.com/ga4gh/wiki/wiki#weekly-work-stream-meeting has the schedule
so 7am pacific is 3pm gmt?