scipipe/scipipe

on writing workflows with scipipe

Samuel Lampa

@samuell

I think there is already a hard requirement for Go 1.8+ or something, but I think it is great to keep up with the changes coming in Go now.

Drayton Munster

@dwmunster

Cool, I put 1.13 in there but we can always bump it up if we decide we want a newer feature later

Samuel Lampa

@samuell

:thumbsup:

Drayton Munster

@dwmunster

What would you think about disabling the windows tests for now? The configuration means that the PR can still run Windows tests, but for now they just are noise on the PR

Samuel Lampa

@samuell

Indeed, yes, better to inactivate for now

Drayton Munster

@dwmunster

As far as I can tell, I think the best way to do that is on your side, you can disable the webhook and we can re-enable it later

Samuel Lampa

@samuell

Aight

Drayton Munster

@dwmunster

https://github.com/scipipe/scipipe/settings/hooks

It's been a while since I've used appveyor, at least I think it's done with webhooks

Samuel Lampa

@samuell

I don't find it there, but I'll dig it up somehow ...

It was under "Installed GitHub Apps" ... Removed now

Drayton Munster

@dwmunster

:+1:

Drayton Munster

@dwmunster

What was the motivation for Check and CheckWithMsg to die (via os.Exit(1)) instead of bubbling the error back to the user?

from a practical point of view, it makes it hard to test something that I know should error. In particular, making sure the segfault from #94 doesn't occur and that it errors instead

Samuel Lampa

@samuell

from a practical point of view, it makes it hard to test something that I know should error. In particular, making sure the segfault from #94 doesn't occur and that it errors instead

I see.

The reason has been primarily to simplify the API, with the thinking that the errors that are sent to Check/WithMsg() would be of the kind that would require a restart of a workflow in any case.

I guess one reason for simplifying as much as possible is that SciPipe was from the start aimed a lot towards bioinformatics, which has had a lot of non-programming-savvy people (wet lab folks) dipping their toes into it, and as such you would have to simplify as much as you can.

Though not sure that will/should be the primary user group as we go forward.

Drayton Munster

@dwmunster

An option would be to use errors internally and have something at the top that catches them and dies

But that is a valid point

Samuel Lampa

@samuell

An option would be to use errors internally and have something at the top that catches them and dies

Sounds reasonable

Jonathan Alvarsson

@jonalv

In the video at: https://youtu.be/hi0Uqwddrtg you seem to have a scipipe program on your path. I have installed scipipe using go get github.com/scipipe/scipipe/... but I still don't have that command on my path. How is that setup?

Samuel Lampa

@samuell

@jonalv Hmm, what do you get if you execute:

echo $GOPATH/bin

... and:

ls $GOPATH/bin

Jonathan Alvarsson

@jonalv

% echo $GOPATH/bin
/bin

bah why won't it honour my newline :(

Samuel Lampa

@samuell

Hmm ...

ls $GOPATH/bin then?

Jonathan Alvarsson

@jonalv

% ls $GOPATH/bin
[        dd        launchctl    pwd        test
bash        df        link        rm        unlink
cat        echo        ln        rmdir        wait4path
chmod        ed        ls        sh        zsh
cp        expr        mkdir        sleep
csh        hostname    mv        stt
dash        kill        pax        sync
date        ksh        ps        tcsh

Samuel Lampa

@samuell

Ok, it seems $GOPATH is not set, so it just lists everything under your system-wide /bin/ instead

what about $GOROOT?

Jonathan Alvarsson

@jonalv

empty

Samuel Lampa

@samuell

I'm not sure how Go installs itself on Macs these days, but I use to set these things up in my .bashrc to be sure:

export GOROOT=$HOME/go
export GOPATH=$HOME/proj/go

export PATH=$HOME/go/bin:$PATH
export PATH=$HOME/proj/go/bin:$PATH

... given that I install Go in $HOME, and use $HOME/proj/go as my code folder for Go stuff

Jonathan Alvarsson

@jonalv

go is in /usr/local/bin/go

Samuel Lampa

@samuell

So, as you might guess from this, $GOROOT is for the Go installation, and $GOPATH is for your own Go code, and stuff that you install with go get.

Hmm

samuell @samuell brb, grabbing tea

Jonathan Alvarsson

@jonalv

but I don't know where my /proj/go would be.

I mean I did go get on scipipe but where did it end up? It didn't say a word that command... :/

Samuel Lampa

@samuell

That's a good question indeed

Jonathan Alvarsson

@jonalv

I am gonna call this a day. My workflow works fine so far but I had to write it manually cause I couldn't access that fancy new workflow thing because of this path stuff... :/

Samuel Lampa

@samuell

I c... well, I'll see if I can find some pointers about the env variable stuff on Mac ...

The Go folks have tried to get away from using a lot of env variables ... but I'm not sure how it is supposed to work instead :P

@jonalv Just for ref (for monday :) ), I think you would be able to solve this with creating a folder proj/go (or something else), and adding these to your bash profile file (however that is named on Mac):

export GOPATH=$HOME/proj/go
export PATH=$HOME/proj/go/bin:$PATH

The next question is whether this is the "canonical" way to solve it these days, but it should work I think.

Samuel Lampa

@samuell

This message was deleted

Samuel Lampa

@samuell

On twitter I got confirmed that the above is how to do it (set $GOPATH, and then include $GOPATH/bin in $PATH). Someone also pointed out that you might be using zsh as default shell now, as of Catalina?

Samuel Lampa

@samuell

Hooray, new release: https://github.com/scipipe/scipipe/releases/tag/v0.9.7

Samuel Lampa

@samuell

And, new release: https://github.com/scipipe/scipipe/releases/tag/v0.9.8

Samuel Lampa

@samuell

New release again: https://github.com/scipipe/scipipe/releases/tag/v0.9.10
Thanks to @szliao for the contribution!

Samuel Lampa

@samuell

New:
https://github.com/scipipe/scipipe/releases/tag/v0.9.12
https://github.com/scipipe/scipipe/releases/tag/v0.9.13
Some 15 issues fixed or otherwise closed off. Phew.