nextflow-io/nextflow

Nextflow community chat moved to Slack! https://www.nextflow.io/blog/2022/nextflow-is-moving-to-slack.html

Shellfishgene

@Shellfishgene

It seems it is, if I add a bunch of newlines before and after $it the message stays. Should this be considered a bug? Maybe those messages should get their own line, like the warnings.

Paolo Di Tommaso

@pditommaso

@evanbiederstedt what's it somatic-variant-comparison-roslin-v4 (1).html ?

@Shellfishgene use .println

evanbiederstedt

@evanbiederstedt

@pditommaso Apologies, please delete. I tried to put an html in the browser quickly, and I think Gitter was open

Sorry all

Paolo Di Tommaso

@pditommaso

I see, no pb

evanbiederstedt

@evanbiederstedt

:embarrassment emoji:
grazie

Stijn van Dongen

@micans

@Shellfishgene I've improved the solution a bit; I wasn't happy about sorting based on filename; the solution below actually uses the sample names and sorts the lists based on that.

R1 = Channel.create()
R2 = Channel.create()

Channel.from(['Sample_a', ['a.R1.fastq', 'a.R2.fastq']]
           , ['Sample_b', ['b.R1.fastq', 'b.R2.fastq']]
           , ['Sample_c', ['c.R1.fastq', 'c.R2.fastq']])

  .separate ( R1, R2 ) { it -> (id, reads) = it; [ [id, reads[0]], [id, reads[1]] ] }

def sortfqlist(thelist) {
   thelist
   .sort { a,b -> a[0] <=> b[0] }
   .collect { it[1] }
}

R1.toList()
  .subscribe onNext: { list = sortfqlist(it); println "list1: " + list }

R2.toList()
  .subscribe onNext: { list = sortfqlist(it); println "list2: " + list }

Paolo Di Tommaso

@pditommaso

well, it looks NF is powering cool stuff

Stijn van Dongen

@micans

well you know I love NF. it's a lot of fun programming in it!

Paolo Di Tommaso

@pditommaso

I was referring @evanbiederstedt link :)

Stijn van Dongen

@micans

hahaha

that's so funny :-)

It's mine, I'm claiming it

Paolo Di Tommaso

@pditommaso

your love is unquestioned, tho :)

Stijn van Dongen

@micans

it's powerful love

Paolo Di Tommaso

@pditommaso

the best one

Stijn van Dongen

@micans

I know, reminds me of that year there were three songs all called the 'the power of love'

Huwey Lewis and the News, Jennifer Rush, and Frankie goes to Hollywood IIRC

Paolo Di Tommaso

@pditommaso

lol

Seth

@sstadick

Hi! Is there any documentation surrounding the -params-file option? I see it referenced in old issues, and kicking around in the source code, but it doesn't look like it's used anymore?

Tobias "Tobi" Schraink

@tobsecret

@sstadick I typically write bash scripts that run nextflow with params specified

Paolo Di Tommaso

@pditommaso

Just specify a yaml or json file

Seth

@sstadick

:thumbsup: I didn't realize it was going to be that simple.

Paolo Di Tommaso

@pditommaso

:stuck_out_tongue_closed_eyes:

Shellfishgene

@Shellfishgene

@micans Thanks for the update!

Chadi Saad

@chadisaad

Good morning ! How can I get easly for a specific nextflow job, the list of all child process (executed via slurm) ?

Pierre Lindenbaum

@lindenb

following this conversation on twitter with @pditommaso ( https://twitter.com/yokofakun/status/1159072968158457856 ) I'm trying to implement some new operators. It's not the final design of the source, (I don't think using the notation 'splitter' is a good choice because we don't really read any line). However , it doesn't work.
I implemented a splitter: https://github.com/lindenb/nextflow/blob/pl_htsjdk_op/modules/nextflow/src/main/groovy/nextflow/extension/htsjdk/SamplesSplitter.groovy
I use this splitter in OperatorExt : https://github.com/lindenb/nextflow/blob/pl_htsjdk_op/modules/nextflow/src/main/groovy/nextflow/extension/OperatorEx.groovy#L1623
I compile './gradlew build -x test ` and when I want to run, I get the following error:

 cat test.nf 
Channel.fromPath("/home/lindenb/src/jvarkit-git/src/test/resources/toy.bam").
    splitSamples().
    dump()
$ ./nextflow run test.nf
N E X T F L O W  ~  version 19.08.0-SNAPSHOT
Launching `test.nf` [goofy_goldstine] - revision: f56eb762e4
Unknown method `splitSamples` on channel type

NF cannot find my operator. what am i missing in the code ?

Vũ Mạnh Tú

@glmanhtu

Hello guys. I'm trying to run nextflow with Kubernetes and have encountered some sort of problem with non-root user. Nextflow was trying to initialise a Pod but got an exception of "container init caused \"mkdir /mnt/tvu: permission denied". Below is the Kubernetes manifest generated by nextflow:

apiVersion: v1
kind: Pod
metadata:
  name: nf-68b883e7affa9dc5d6d5e721c75b21c4
  namespace: default
  labels: {app: nextflow, runName: sleepy-goodall, taskName: indexPeptides, processName: indexPeptides,
    sessionId: uuid-0e5b6c81-4527-4213-a9ba-16e40e424220}
spec:
  restartPolicy: Never
  containers:
  - name: nf-68b883e7affa9dc5d6d5e721c75b21c4
    image: omicsdi/crux:latest
    command: [/bin/bash, -ue, .command.run]
    workingDir: /mnt/tvu/work/68/b883e7affa9dc5d6d5e721c75b21c4
    volumeMounts:
    - {name: vol-1, mountPath: /mnt}
  securityContext: {runAsUser: 2801}
  volumes:
  - name: vol-1
    persistentVolumeClaim: {claimName: pride-pv-claim}

I have assigned rw permission to the persistent volume. When I removed the workingDir, the pod was started successfully and I was be able to cd into the /mnt/tvu/work/68/b883e7affa9dc5d6d5e721c75b21c4 folder. So, I suspect that the runAsUser and workingDir couldn't get along.
Is there anyone have experienced this issue?

Paolo Di Tommaso

@pditommaso

@lindenb try using ./launch.sh instead of nextflow script

then much simpler to test it using a test class

have a look at this example

https://github.com/nextflow-io/nextflow/blob/b49e6d74dae782def6be2b087d06fdd8fa6b9291/modules/nextflow/src/test/groovy/nextflow/splitter/TextSplitterTest.groovy#L31-L37

and the invocation as operator

https://github.com/nextflow-io/nextflow/blob/a8563a92af1b0e80c052951f2a5d563a512f5c0b/modules/nextflow/src/test/groovy/nextflow/extension/SplitTextOpTest.groovy#L13-L28

I would suggest also to open a draft pull request and keep a discuss thread there

Pierre Lindenbaum

@lindenb

@pditommaso :thumbsup:

Philip Jonsson

@kpjonsson

Question: In Nextflow version 19.07.0.5106, I get WARN: The channelcreatemethod is deprecated -- it will be removed in a future release when I use the Channel.create() method, but I find nowhere any mention of what's replacing it. Is the warning wrong or is there some undocumented feature I'm missing?