These are chat archives for nextflow-io/nextflow

26th
Aug 2016
Johan Viklund
@viklund
Aug 26 2016 07:58
regarding running directly from github. In the documentation it says that nextflow will use the "Git repository default branch (default: master)" https://www.nextflow.io/docs/latest/config.html#config-manifest
Paolo Di Tommaso
@pditommaso
Aug 26 2016 07:59
yes
Johan Viklund
@viklund
Aug 26 2016 07:59
but when I have switched default branch in github it just complains
Project `NBISweden/test-nextflow` currently is sticked on revision: develop -- you need to specify explicitly a revision with the option `-r` to use it
I thought it would use either of the branches
I coudn't figure out which one from the documentation though, since "git default branch" is not necesary master
anyhow, I didn't expect it to complain
Paolo Di Tommaso
@pditommaso
Aug 26 2016 08:01
do you mean that the default branch of your GH repo is not master ?
Johan Viklund
@viklund
Aug 26 2016 08:01
exactly, I've changed it to develop
Paolo Di Tommaso
@pditommaso
Aug 26 2016 08:01
let me check a bit
Johan Viklund
@viklund
Aug 26 2016 08:02
I'm testing these kind of things with this repo https://github.com/NBISweden/test-nextflow (if you want to have a look, I just added a commit though)
Paolo Di Tommaso
@pditommaso
Aug 26 2016 08:03
I think you need to add in the nextflow.config file the following section
Johan Viklund
@viklund
Aug 26 2016 08:03
I'm doing that now
:D
Paolo Di Tommaso
@pditommaso
Aug 26 2016 08:03
manifest {
  defaultBranch = 'develop'
}
I don't see it in the repo
Johan Viklund
@viklund
Aug 26 2016 08:03
check again :)
Paolo Di Tommaso
@pditommaso
Aug 26 2016 08:04
nope
Johan Viklund
@viklund
Aug 26 2016 08:04
ahh, I only pushed to the master branch
Paolo Di Tommaso
@pditommaso
Aug 26 2016 08:04
oops ;)
Johan Viklund
@viklund
Aug 26 2016 08:04
now it's in both branches and it still complains
I have defaultBranch set to master in nextflow
(I am removing the assets directory between invocation to make sure my cache is emptied)
Paolo Di Tommaso
@pditommaso
Aug 26 2016 08:05
it's a bit messed up
in the master it says the default it's master ..
Johan Viklund
@viklund
Aug 26 2016 08:06
I want the default on GITHUB to be develop but the default for NEXTFLOW to be master.
think of master as release
but it doesn't seem to matter what I put in nextflow.config
it always complain
I have to go now though, I'll be back later
Paolo Di Tommaso
@pditommaso
Aug 26 2016 08:08
ok
Johan Viklund
@viklund
Aug 26 2016 08:08
(if you want I can submit an issue)
Paolo Di Tommaso
@pditommaso
Aug 26 2016 08:08
let me check it better
Paolo Di Tommaso
@pditommaso
Aug 26 2016 08:59
@viklund It works correctly, you can check with this command
nextflow run pditommaso/testrepo
I think the problem is that you are using a wrong syntax in the config file
Johan Viklund
@viklund
Aug 26 2016 09:00
pebkac, as usual ;)
Paolo Di Tommaso
@pditommaso
Aug 26 2016 09:00
It should be defaultBranch = 'master'
with = not :
Johan Viklund
@viklund
Aug 26 2016 09:00
...
:D thanks
Paolo Di Tommaso
@pditommaso
Aug 26 2016 09:01
welcome!
Johan Viklund
@viklund
Aug 26 2016 09:04
but if you were to specify master in the config file it stops working again, at least for me
Paolo Di Tommaso
@pditommaso
Aug 26 2016 09:06
(tel)
Johan Viklund
@viklund
Aug 26 2016 09:06
take your time, I just add info here :)
I cloned your repo and switched branch, got the "sticked on revision" error again https://github.com/viklund/testrepo.git
Paolo Di Tommaso
@pditommaso
Aug 26 2016 09:46
@viklund can you show the exact sequence of commands ?
Johan Viklund
@viklund
Aug 26 2016 10:45
@pditommaso:
$ rm -rf ~/.nextflow/assets/NBISweden/test-nextflow/
$ nextflow run viklund/testrepo
N E X T F L O W ~ version 0.21.2
Project viklund/testrepo currently is sticked on revision: dev -- you need to specify explicitly a revision with the option -r to use it```
Paolo Di Tommaso
@pditommaso
Aug 26 2016 11:28
@viklund em, but why the config in your close says the default branch is master ?
Johan Viklund
@viklund
Aug 26 2016 11:28
exactly
Paolo Di Tommaso
@pditommaso
Aug 26 2016 11:29
?
Johan Viklund
@viklund
Aug 26 2016 11:29
I want nextflow to run from master even though I have specified on github that the default branch is dev
Paolo Di Tommaso
@pditommaso
Aug 26 2016 11:30
I will start to observe that your config is wrong, that setting must match the GH default branch
Johan Viklund
@viklund
Aug 26 2016 11:30
what?
it's not to choose branch to run?
Paolo Di Tommaso
@pditommaso
Aug 26 2016 11:30
nope
Johan Viklund
@viklund
Aug 26 2016 11:30
I thought it was instead of the -r switch
or rather, that -r overrode defaultBranch
What's the point of it then? If it has to be the same as the default branch, why have the setting at all?
(the default configured for the git repo that is)
Paolo Di Tommaso
@pditommaso
Aug 26 2016 11:37
because it is needed to know from which branch you need initially clone your repo
Johan Viklund
@viklund
Aug 26 2016 11:37
isn't that implicit when doing a normal git clone on the repo?
(I realize that it might be something with a java library somewhere or something)
Paolo Di Tommaso
@pditommaso
Aug 26 2016 11:38
I don't remember exactly all the steps behind, but I think to do what u want to do, you can
nextflow pull <repo>
nextflow run <repo> -r <branch>
Johan Viklund
@viklund
Aug 26 2016 11:39
(this is a minor issue, really, I was just surprised that the defaultBranch setting wasn't for specifying a custom default branch)
Paolo Di Tommaso
@pditommaso
Aug 26 2016 11:40
yes, it is but must much with the once you set in the GH repo
otherwise it will run in an inconsistent state
Johan Viklund
@viklund
Aug 26 2016 11:42
I imagined it to follow the following procedure. 1. git pull, 2. check configfile, 3. git checkout, 4. start workflow
I don't understand how it can be inconsistent, I'm probably missing something, but I don't want to waste any more of your time on this
Paolo Di Tommaso
@pditommaso
Aug 26 2016 11:45
if you want, you can waste your time :)
Johan Viklund
@viklund
Aug 26 2016 11:45
:D
Paolo Di Tommaso
@pditommaso
Aug 26 2016 11:53
BTW I will try to upload a patch release for your issue with blanks in the path later today
Johan Viklund
@viklund
Aug 26 2016 11:53
ah, nice
Paolo Di Tommaso
@pditommaso
Aug 26 2016 11:54
until u won't have a more consistent file system that ban blanks
:):):)
Johan Viklund
@viklund
Aug 26 2016 12:52
can I create a channel from a list of files?
it should emit one file at a time, preferably
hmm, maybe with flatten...
It seems I can't have an array as argument for Channel.create()
no, my bad
I must of course use Channel.from()
it seems to work like I wanted
Paolo Di Tommaso
@pditommaso
Aug 26 2016 12:57
I think you need Channel.fromPath
Johan Viklund
@viklund
Aug 26 2016 12:58
yes, but it didn't work with this argument ['file1', 'file2']
this particular case could have been written as 'file*' but this is just a small test
Paolo Di Tommaso
@pditommaso
Aug 26 2016 12:59
ah no, it works with glob pattern
so also {file1,file2}
Johan Viklund
@viklund
Aug 26 2016 12:59
I did it with Channel.from(array).flatten()
can I have {file1, file2} in the config file then?
Paolo Di Tommaso
@pditommaso
Aug 26 2016 13:00
yes
as a string
Johan Viklund
@viklund
Aug 26 2016 13:00
yea, right, I'm globbing
Paolo Di Tommaso
@pditommaso
Aug 26 2016 13:00
params.something = '/some/path/{file1,file2}'
Johan Viklund
@viklund
Aug 26 2016 13:01
I do that all the time on the command line, but was not thinking in that mode here
thanks
Paolo Di Tommaso
@pditommaso
Aug 26 2016 13:02
welcome
Johan Viklund
@viklund
Aug 26 2016 13:48
It looks like I can't have a glob pattern with the files like this:
{/tmp/file1,/tmp/file2}
Paolo Di Tommaso
@pditommaso
Aug 26 2016 13:51
/tmp/{file1,file2}
is not an option ?
Johan Viklund
@viklund
Aug 26 2016 13:51
well, could be, now I'm getting some permission issues though
Paolo Di Tommaso
@pditommaso
Aug 26 2016 13:52
it must share a base path
Johan Viklund
@viklund
Aug 26 2016 13:52
they must?
oh, well, I think I'll just use my array with flatten() then
Paolo Di Tommaso
@pditommaso
Aug 26 2016 13:52
:+1:
make sure to have an array of file(..) not just path strings
Johan Viklund
@viklund
Aug 26 2016 13:53
yes, it will be files.collect { file(it) } somewhere
Paolo Di Tommaso
@pditommaso
Aug 26 2016 13:53
perfect
than just do .channel()
ie. files.collect { file(it) } .channel()
Johan Viklund
@viklund
Aug 26 2016 13:54
oh, you have done that, nice! :sparkles:
didn't know you had added that method to the ArrayList :D
Paolo Di Tommaso
@pditommaso
Aug 26 2016 13:57
a lot of tricks that I forget them as well :)
Paolo Di Tommaso
@pditommaso
Aug 26 2016 15:02
@viklund just upload 0.21.3-SNAPSHOT fixing the issue with blanks in the path
you may want to give a try:
NXF_VER=0.21.3-SNAPSHOT nextflow run ..
Paolo Di Tommaso
@pditommaso
Aug 26 2016 19:26
BTW the way the snapshot was not good, instead the version just release fix the problem
you can get it in the usual way
nextflow -self-update