@BenjaminHCCarr it's sane to submit all of them to the queue - but I think it does do git checkout on submission (to check the file path) so it might still be slow. Here's how I do restore from backup:
https://github.com/common-workflow-language/cwlviewer/blob/1.3.0-RC1/load.py#L76
as you see I let it have 6 at a time, sleeping 0.5s to let queue shrink a bit
that one works fine for re-loading all ~4190 workflows
I have submitted the "CWL community" as a service provider to the EOSC Portal so that we may advertise the CWL Viewer itself: https://eosc-portal.eu/for-providers
Interesting, the CWL Viewer has over 100 000 downloads of the docker container, more than any other CWL organization container on the Docker Hub: https://hub.docker.com/u/commonworkflowlanguage
Sounds like there could be a project to make some part of cwlviewer into a standalone tool
I think that came up at the Meetup a couple weeks ago
Hi, not sure if this is the right place to bring this up, but i hope so!? I'm interested in embedding the interactive workflow image(s) within (publishable) dataset webpages, for us, the simpliest way to do that would be to add them as a widget view. Does the CWL-viewer tool already have this, or would it be a feature you could add? By way of background, I'm looking to include author generated workflows within GigaDB datasets, we would like to do this in a similar way to how we include SketchFab interactive 3D models, and protocols.io. i.e. we just add the widget code to a dataset and the it displays the relevant item in a window within the dataset page. Here is a link to a SketchFab example; open the DOI http://dx.doi.org/10.5524/100539 navigate to the tab labelled "3D viewer" . We would like to add a tab in the framework for "CWL viewer" to allow those datasets that have it to display the interactive workflow there. Any comments? Thanks.
Are these workflows stored on GitHub, GitLab, or some other publicly accessible Git repository? If so then you can use the API: https://view.commonwl.org/apidocs
The example of the sketchfab widget does not have a workflow, it was just to demonstrate how I would like to be able to integrate external sources such as CWL-viewer. The most recent dataset we have published with a workflow (in GitHub) is one that has already been put through cwl-viewer, the only way I can link to it at the moment from GigaDB is just as a URL for users to click, see here http://dx.doi.org/10.5524/100553
@only1chunts Thanks for linking to the CWL Viewer! The
.cwl is missing from the link, here's the correct one: https://view.commonwl.org/workflows/github.com/vetscience/Assemblosis/blob/66660a64824e4c1019680c6c7ea489ea359cd5f6/Run/assembly.cwl
@only1chunts You could put that URL in an iframe? Oh, and you should use the permalink version instead of the view.commonwl.org version: https://w3id.org/cwl/view/git/66660a64824e4c1019680c6c7ea489ea359cd5f6/Run/assembly.cwl
but if the workflow is not in a git repository it's a bit more tricky because of versioning and relative links. In theory it will work straight on the web (we used to do that) but now we always checkout from git in a way to force people to do the "right thing"
@only1chunts - are you perhaps considering storing the CWL definitions also within GigaDB? We would need them to be directly on the web (no landing pages) and either all-in-one-file (which is not common in CWL community as people like to split out their tools) or with neighbouring CWL files available from the same base URI. If GitHub/GitLab is tricky for you to require, perhaps some in-between solution would be a downloadable .git zip archive which would be easier for us to integrate? Harder for users to upload to you though.
@mr-c thanks for the pointer about using the permalink, i thought i had! but obvious just cut and paste the wrong link. I'll change that now, it will also reduce the length which is why the .cwl was missing, (our database table truncated it). The iFrame is exactly what we want to do, but would like it to point to a place that has been specifically designed to view in a smaller frame (i.e. cut down to remove surround stuff like the the full list of tables below the interactive viewer is not required for a simple iframe view, If I can workout how to append an image I'll show you the view that we would like to see from within a GigaDB dataset , but if you dont feel the need for it then we could iframe the entire page within GigaDB.
I cant see how to add an image so I'll email it to you @mr-c
Hi @stain , sorry I perhaps have not explained my use case very well. We are not (at this point) concerned with how an author /user gets their workflow in CWL viewer, just how we can present it IF they have done so. A static SVG image is an option, but its not great to look at when its a complex workflow where the boxes are tiny, the zoom functions in the viewer are very useful then. Plus, I'm assuming at some point in the future the viewer will have more interactivity added for links to tools and intersecting workflows etc, which would then be lost in the SVG version. Thanks.
I think that could be doable by a small change to the templates
https://github.com/common-workflow-language/cwlviewer/blob/master/src/main/resources/templates/workflow.html is the normal workflow page template, note that it has the SVG viewer twice as there is also the modal pop-up. Perhaps adding
?embed=trueto the permalinks would be an option? It already works with ?format=html although that gives the big page now https://w3id.org/cwl/view/git/f55f0011de7db66f873ffb0f700f3fe26b33911e/genomel/cwl/workflows/harmonization/harmonization_bwa_mem.cwl?format=html
Should the steps, inputs & output be included? Or should they be replaced with a link to the "full" view?
Thanks for thinking I might be a developer, but sadly you've just gone way past my knowledge on html (or programming of any sort). If developing a widget viewer is not something anyone else needs/wants and its not on your todo list then thats fine, we'll just have to stick with the URL link for now.
also be cool on the CWL Viewer?
@stain time to reboot?
Error: cwltool version failed to run on this workflow:
! Cwltool ran successfully, but an unexpected error occurred in CWLViewer!
Help us by reporting it on Gitter or a Github issue
hmm.. I get the same error as with cwltool --validate really.
Viewer:
Tool definition failed validation:
../data/git/aa420273539ad7bcf384c37614f6f636810042f2/Workflows/bams2gvcf.woBQSR_male_chrXY_wXTR.cwl:170:16: Workflow step output 'vcf_tbi' does not correspond to
../data/git/aa420273539ad7bcf384c37614f6f636810042f2/Tools/gatk3-HaplotypeCaller-XCORE1-ploidy1.cwl:51:1: tool output (expected 'vcf', 'log')Command line:
(base) stain@biggie:~$ cwltool --validate https://raw.githubusercontent.com/ddbj/human-reseq/master/Workflows/bams2gvcf.woBQSR_male_chrXY_wXTR.cwl
/home/stain/miniconda3/bin/cwltool 1.0.20190228200117
Tool definition failed validation:
https://raw.githubusercontent.com/ddbj/human-reseq/master/Workflows/bams2gvcf.woBQSR_male_chrXY_wXTR.cwl:170:16: Workflow step output 'vcf_tbi' does not correspond to
https://raw.githubusercontent.com/ddbj/human-reseq/master/Tools/gatk3-HaplotypeCaller-XCORE1-ploidy1.cwl:51:1: tool output (expected 'vcf', 'log')
btw, that badge is just
https://img.shields.io/badge/cwl-valid-B5314C.svg?logo=common-workflow-language but the viewer would need to have an in-between resource that checks it live.
(shields is thought to have timeout issues; badgen is said to be faster in rendering)
@stain I think this was the commit making the problem I reported earlier? But now it works! https://github.com/ddbj/human-reseq/blob/9064c301d858a796f8340d1e762e5916fd199da0/Workflows/bams2gvcf.woBQSR_male_chrXY_wXTR.cwl
also the old
sbg: 'https://sevenbridges.com'
raised as DataBiosphere/topmed-workflows#97 although there is some other bug as https://view.commonwl.org/workflows/github.com/stain/topmed-workflows/blob/namespaces/aligner/sbg-alignment-cwl/topmed-alignment.cwl# turns out empty - seems to be because id confuses the CWL Viewer :(
https://view.commonwl.org/workflows/github.com/stain/topmed-workflows/blob/no-id/aligner/sbg-alignment-cwl/topmed-alignment.cwl works when I deleted id as well