Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 25 16:07
    pditommaso closed #3435
  • Nov 25 16:07

    pditommaso on master

    Add support for custom conda ch… (compare)

  • Nov 25 16:07

    pditommaso on conda-custom-channels

    (compare)

  • Nov 25 16:01
    pditommaso synchronize #3435
  • Nov 25 16:00

    pditommaso on conda-custom-channels

    Fix conda defualt channels orde… (compare)

  • Nov 25 14:49
    jordeu opened #3438
  • Nov 25 14:45

    jordeu on tower_archive_basedir_fix

    TowerArchiver resolve envar pat… (compare)

  • Nov 25 13:49
    keiranmraine opened #3437
  • Nov 25 12:49

    pditommaso on master

    Add time directive to AWS Batch… (compare)

  • Nov 25 12:49
    pditommaso closed #3436
  • Nov 25 12:47
    llewellyn-sl opened #3436
  • Nov 25 11:55
    amizeranschi closed #3434
  • Nov 25 11:55
    amizeranschi commented #3434
  • Nov 25 11:54
    amizeranschi closed #3433
  • Nov 25 11:54
    amizeranschi commented #3433
  • Nov 25 11:38
    pditommaso commented #3434
  • Nov 25 11:27
    pditommaso review_requested #3435
  • Nov 25 11:27
    pditommaso opened #3435
  • Nov 25 11:23

    pditommaso on conda-custom-channels

    Add support for custom conda ch… (compare)

  • Nov 25 10:49
    nvnieuwk commented #3370
Pierre Lindenbaum
@lindenb
following this conversation on twitter with @pditommaso ( https://twitter.com/yokofakun/status/1159072968158457856 ) I'm trying to implement some new operators. It's not the final design of the source, (I don't think using the notation 'splitter' is a good choice because we don't really read any line). However , it doesn't work.
I implemented a splitter: https://github.com/lindenb/nextflow/blob/pl_htsjdk_op/modules/nextflow/src/main/groovy/nextflow/extension/htsjdk/SamplesSplitter.groovy
I use this splitter in OperatorExt : https://github.com/lindenb/nextflow/blob/pl_htsjdk_op/modules/nextflow/src/main/groovy/nextflow/extension/OperatorEx.groovy#L1623
I compile './gradlew build -x test ` and when I want to run, I get the following error:
 cat test.nf 
Channel.fromPath("/home/lindenb/src/jvarkit-git/src/test/resources/toy.bam").
    splitSamples().
    dump()
$ ./nextflow run test.nf
N E X T F L O W  ~  version 19.08.0-SNAPSHOT
Launching `test.nf` [goofy_goldstine] - revision: f56eb762e4
Unknown method `splitSamples` on channel type
NF cannot find my operator. what am i missing in the code ?
Vũ Mạnh Tú
@glmanhtu

Hello guys. I'm trying to run nextflow with Kubernetes and have encountered some sort of problem with non-root user. Nextflow was trying to initialise a Pod but got an exception of "container init caused \"mkdir /mnt/tvu: permission denied". Below is the Kubernetes manifest generated by nextflow:

apiVersion: v1
kind: Pod
metadata:
  name: nf-68b883e7affa9dc5d6d5e721c75b21c4
  namespace: default
  labels: {app: nextflow, runName: sleepy-goodall, taskName: indexPeptides, processName: indexPeptides,
    sessionId: uuid-0e5b6c81-4527-4213-a9ba-16e40e424220}
spec:
  restartPolicy: Never
  containers:
  - name: nf-68b883e7affa9dc5d6d5e721c75b21c4
    image: omicsdi/crux:latest
    command: [/bin/bash, -ue, .command.run]
    workingDir: /mnt/tvu/work/68/b883e7affa9dc5d6d5e721c75b21c4
    volumeMounts:
    - {name: vol-1, mountPath: /mnt}
  securityContext: {runAsUser: 2801}
  volumes:
  - name: vol-1
    persistentVolumeClaim: {claimName: pride-pv-claim}

I have assigned rw permission to the persistent volume. When I removed the workingDir, the pod was started successfully and I was be able to cd into the /mnt/tvu/work/68/b883e7affa9dc5d6d5e721c75b21c4 folder. So, I suspect that the runAsUser and workingDir couldn't get along.
Is there anyone have experienced this issue?

Paolo Di Tommaso
@pditommaso
@lindenb try using ./launch.sh instead of nextflow script
then much simpler to test it using a test class
have a look at this example
and the invocation as operator
I would suggest also to open a draft pull request and keep a discuss thread there
Pierre Lindenbaum
@lindenb
@pditommaso :thumbsup:
Philip Jonsson
@kpjonsson
Question: In Nextflow version 19.07.0.5106, I get WARN: The channelcreatemethod is deprecated -- it will be removed in a future release when I use the Channel.create() method, but I find nowhere any mention of what's replacing it. Is the warning wrong or is there some undocumented feature I'm missing?
Ghost
@ghost~57581dbcc2f0db084a1ccd04
Don't worry, it won't happen overnight
AFAIK you can safely already remove the stuff like ch_input = Channel.create()
Philip Jonsson
@kpjonsson
@MaxUlysse Got it, thanks.
Ghost
@ghost~57581dbcc2f0db084a1ccd04

And I do believe that was the important part:

to me the only use for create that makes sense is when used with choice

Paolo Di Tommaso
@pditommaso
this is just great, isn't it ?
Hugues Fontenelle
@huguesfontenelle
hi!
How do I read the documentation for an older release? ie 19.04
well I guess I can simply read in github ..
Paolo Di Tommaso
@pditommaso
indeed
evanbiederstedt
@evanbiederstedt

@pditommaso
this is just great, isn't it ?
https://twitter.com/yokofakun/status/1159468857934929922

Confirmed, this is great

evanbiederstedt
@evanbiederstedt

https://gitter.im/nextflow-io/nextflow?at=5d4802a1475c0a0feb021c1b

Could you give us a few examples @pditommaso to illustrate this point? It still feels a bit abstract without a few concrete examples

Pierre Lindenbaum
@lindenb
@evanbiederstedt @pditommaso thanks ! I'm still not convinced it's the right way to implement the idea. 1) one can still use a bash script to extract the data 2) or it would be better using a 'static' method: something like:Channel.fromMap("my.bam").flatMap(Htsjdk::extractSamples)
Taylor Falk
@taylor.f_gitlab
Is there an easy way to pass two channel outputs and some associated strings to a process? I'm currently trying output = Channel.from(['filter1', channel1_out], ['filter2', channel2_out]) but this only returns the blank DataflowQueue(queue=[]) string inside the process.
Stijn van Dongen
@micans
@taylor.f_gitlab that would mean you'd have channels in a channel. What are you trying to achieve? What's in channel1_out and channel2_out?
Taylor Falk
@taylor.f_gitlab
@micans those channels contain singular files, so I am really just trying to run the next process on each output from channel1 and channel2, and passing along the correct filter string. How do I move those files into one channel, concat?
Stijn van Dongen
@micans

@taylor.f_gitlab do you have a source process? Normally you'd see something like

output: set val('filter'), file('*.txt') into channel1

then if there are multiple files in that output that you want to flatten you can use transpose().

Taylor Falk
@taylor.f_gitlab
Oh that's a good idea, let me try it that way.
Gabriel Abud
@G_Abud_twitter
I'm getting the error: fatal error: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied when trying to run a pipeline on AWS Batch. Pretty sure it's due to the s3 access of the workDir but I've double checked and I am the owner of that bucket. Any ideas?
Combiz
@combiz_k_twitter
Hi, this may be more of a singularity question than a nextflow one. I'm trying to run a test hello world nextflow script with -with-singularity image.sif on an HPC cluster. There seems to be an issue with mounting as the short script runs but produces an error `Command error: Fatal error: cannot create 'R_TempDir'. Any ideas? My guess is the singularity container can't write files to the HPC filesystem? Thanks for any pointers.
Combiz
@combiz_k_twitter
Ok so the script now runs with containerOptions '-B $PWD:/tmp' in the process
Johannes Alneberg
@alneberg

A question regarding input of memory specifications. Using the config

params {
  memParam = 7.GB
}

process {
  withName:sayHello {
    memory = {2 * params.memParam}
  }
}

works fine when memParam is not used, but when it is: nextflow run main.nf -c base.conf --memParam 8.GB, I get either

ERROR ~ Error executing process > 'sayHello (3)'

Caused by:
  No signature of method: java.lang.Integer.multiply() is applicable for argument types: (java.lang.String) values: [8.GB]
Possible solutions: multiply(java.lang.Character), multiply(java.lang.Number)

or

Error executing process > 'sayHello (1)'

Caused by:
  Not a valid 'memory' value in process definition: 8.GB8.GB

depending on the order of the multiplication. What am I missing? (I'm on Nextflow version 19.04.1 build 5072)

Paolo Di Tommaso
@pditommaso
umm, because when passing on the command line the string 8.GB does not get parse to Memory unit object
therefore for the interpreter it's a string
Johannes Alneberg
@alneberg
Is it possible to go around?
Paolo Di Tommaso
@pditommaso
you may try 2 * (params.memParam as nextflow.util.MemUnit)
quite ugly
Johannes Alneberg
@alneberg
Well, rather ugly than broken, I'll give it a try
it throws me unable to resolve class nextflow.util.MemUnit
Alaa Badredine
@AlaaBadredine_twitter
not Util ?
Johannes Alneberg
@alneberg
Same error using Util I'm afraid
Alaa Badredine
@AlaaBadredine_twitter
did you put the parenthesis ?
Johannes Alneberg
@alneberg
Yes:
process {
  withName:sayHello {
    memory = {2 * (params.memParam as nextflow.util.MemUnit)}
  }
}
Alaa Badredine
@AlaaBadredine_twitter
nextflow.util.MemUnit()
maybe he meant it like that ?
Johannes Alneberg
@alneberg
That gave me some syntax error: expecting ')', found '(' @ line 7, column 60.
I think I fixed it