Options for recording the Docker container identifier into a file:
--record-container-id
If enabled, store the Docker container ID into a file.
See --cidfile-dir to specify the directory.
--cidfile-dir CIDFILE_DIR
Directory for storing the Docker container ID file.
The default is the current directory
--cidfile-prefix CIDFILE_PREFIX
Specify a prefix to the container ID filename. Final
file name will be followed by a timestamp. The default
is no prefix.
dockstore tool launch --local-entry Dockstore.cwl --json test_run_L000013.json
and as far as I can tell the container itself is happy. The pipeline within the container is throwing errors which I think are being saved to logs on the container-local filesystem. Can I view these logs? Does the container instance die when the pipeline terminates?
cwltool.errors.WorkflowException: Did not find output file with glob pattern: '['WGS_*_vs_*.result.tar.gz']'
bring to mind any general ideas what could be happening?
battenberg.pl -o /var/spool/cwl/WGS_2016-2066_vs_2016-2067/battenberg -u /var/spool/cwl/reference_files/battenberg/1000genomesloci -e /var/spool/cwl/reference_files/battenbe
rg/impute/impute_info.txt -c /var/spool/cwl/reference_files/battenberg/probloci.txt -r /var/spool/cwl/reference_files/genome.fa.fai -ig /var/spool/cwl/reference_files/battenberg/ignore_contigs.txt
-ge XX -tb /var/spool/cwl/tmp/2016-2066.bam -nb /var/spool/cwl/tmp/2016-2067.bam -p allelecount -nl 50 -t 40 >& /var/spool/cwl/timings/WGS_2016-2066_vs_2016-2067.time.alleleCount
where I don't see anything like the glob above
cwl-runner
and cwltool but I find the cwltool
versions fail to resolve linked files:$ cwltool --make-template quay.io/wtsicgp/dockstore-cgpmap:3.0.0
/usr/bin/cwltool 1.0.20180302231433
Resolved 'quay.io/wtsicgp/dockstore-cgpmap:3.0.0' to 'https://dockstore.org:8443/api/ga4gh/v1/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor'
No handlers could be found for logger "rdflib.term"
Tool definition failed initialization:
https://dockstore.org:8443/api/ga4gh/v1/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor:21:5: ('https://dockstore.org:8443/api/ga4gh/v1/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml', HTTPError(u'404 Client Error: Not Found for url: https://dockstore.org:8443/api/ga4gh/v1/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml',))
$ echo $?
1
$ cwl-runner --make-template https://dockstore.org:443/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor/%2FDockstore.cwl
/usr/bin/cwl-runner 1.0.20180302231433
No handlers could be found for logger "rdflib.term"
bams_in: null
bwa: null
bwa_idx:
class: File
path: default/file/path
groupinfo: null
mmqc: false
mmqcfrac: null
reference:
class: File
path: default/file/path
sample: default_string
$ cwltool --make-template https://dockstore.org:443/api/api/ga4gh/v2/tools/quay.io%Fwtsicgp%Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor/%FDockstore.cwl
/home/mcrusoe/cwltool/env3/bin/cwltool 1.0.20181217162649
sample: a_string # type "string"
reference: # type "File"
class: File
path: a/file/path
mmqcfrac: 0.05 # default value of type "float". (optional)
mmqc: false # type "boolean"
groupinfo: # type "File" (optional)
class: File
path: a/file/path
bwa_idx: # type "File"
class: File
path: a/file/path
bwa: ' -Y -K 100000000' # default value of type "string". (optional)
bams_in: # array of type "File" (optional)
- class: File
path: a/file/path
An interesting issue here:
I have a workflow registerd on Dockstore, it runs fine with this command:
dockstore workflow launch --entry github.com/cancerit/workflow-seq-import/chksum_seqval_wf_interleaved_fq:0.4.0 --json chksum_seqval_wf_interleaved_fq_with_headers.json
it'll fail if I run it like below:
cwl-runner https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/%2Fcwls%2Fchksum_seqval_wf_interleaved_fq.cwl chksum_seqval_wf_interleaved_fq_with_headers.json
Tool definition failed initialization:
Tool definition https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl failed validation:
https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl:27:1: checking field `steps`
https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl:28:3: checking object `https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl#if_file_name_is_bz2_then_return_null_else_return_in_json_to_output`
https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_input_is_bz2_generate_md5sum_else_return_input_chksum_json.cwl:35:5: Field `run` contains undefined reference to `https://dockstore.org:443/api/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fcancerit%2Fworkflow-seq-import%2Fchksum_seqval_wf_interleaved_fq/versions/0.4.0/plain-CWL/descriptor/toolkit/if_file_name_is_bz2_then_return_null_else_return_in_json_to_output.cwl`
Not Found : version found, but file not found (bad filename, invalid file, etc.)
l$ cwltool --validate https://github.com/cancerit/workflow-seq-import/raw/0.4.0/cwls/chksum_seqval_wf_interleaved_fq.cwl
/home/michael/cwltool/env3/bin/cwltool 1.0.20181217162649
URI prefix 'foaf' of 'foaf:name' not recognized, are you missing a $namespaces section?
URI prefix 'foaf' of 'foaf:mbox' not recognized, are you missing a $namespaces section?
URI prefix 'foaf' of 'foaf:name' not recognized, are you missing a $namespaces section?
URI prefix 'foaf' of 'foaf:mbox' not recognized, are you missing a $namespaces section?
URI prefix 'foaf' of 'foaf:name' not recognized, are you missing a $namespaces section?
URI prefix 'foaf' of 'foaf:mbox' not recognized, are you missing a $namespaces section?
https://github.com/cancerit/workflow-seq-import/raw/0.4.0/cwls/chksum_seqval_wf_interleaved_fq.cwl is valid CWL.