Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 11:12
    abhi18av commented #3015
  • 11:08
    abhi18av synchronize #3015
  • 11:08

    abhi18av on pbspro-cluster-options

    Add Flux executor to nextflow (… Improve file porter logging Si… Bump nf-amazon@1.12.0 Signed-o… and 18 more (compare)

  • Nov 27 16:10
    mribeirodantas review_requested #3408
  • Nov 27 16:10
    mribeirodantas commented #3408
  • Nov 27 16:10
    mribeirodantas synchronize #3408
  • Nov 27 16:10
    mribeirodantas commented #3405
  • Nov 27 16:09
    mribeirodantas synchronize #3405
  • Nov 27 12:59

    pditommaso on master

    Add support for Java 19 Signed… (compare)

  • Nov 25 16:07
    pditommaso closed #3435
  • Nov 25 16:07

    pditommaso on master

    Add support for custom conda ch… (compare)

  • Nov 25 16:07

    pditommaso on conda-custom-channels

    (compare)

  • Nov 25 16:01
    pditommaso synchronize #3435
  • Nov 25 16:00

    pditommaso on conda-custom-channels

    Fix conda defualt channels orde… (compare)

  • Nov 25 14:49
    jordeu opened #3438
  • Nov 25 14:45

    jordeu on tower_archive_basedir_fix

    TowerArchiver resolve envar pat… (compare)

  • Nov 25 13:49
    keiranmraine opened #3437
  • Nov 25 12:49

    pditommaso on master

    Add time directive to AWS Batch… (compare)

  • Nov 25 12:49
    pditommaso closed #3436
  • Nov 25 12:47
    llewellyn-sl opened #3436
banjosnapper
@banjosnapper
Okay so I worked out the isse. It was because I did not give an absolute path name to the mmuChr.gff3 in the perl script. Many thank for your help @AlaaBadredine_twitter
Alaa Badredine
@AlaaBadredine_twitter
@banjosnapper no problem
mmatthews06
@mmatthews06
@pditommaso or anyone else, did anybody catch my question about attaching a debugger to Nextflow? Or running Nextflow in IntelliJ in debug mode, to set breakpoints, etc.? I'm starting back working on that, just thought I'd ask for any quick hints, since I assume somebody has already done it.
Paolo Di Tommaso
@pditommaso
it's possible for the nextflow runtime development, but I guess you want for nextflow scripts
mmatthews06
@mmatthews06
No, runtime development. I'm mucking around Nextflow internals for the time being.
Paolo Di Tommaso
@pditommaso
then it's straightforward, use ./launch.sh -remote-debug run .. etc
mmatthews06
@mmatthews06
Ah, alright, I'll try that. Thanks!
Paolo Di Tommaso
@pditommaso
Workflow components in the pipeline :sunglasses:
Stephen Kelly
@stevekm
@banjosnapper put your scripts in a directory called bin adjacent to the main nextflow script; example here: https://github.com/stevekm/nextflow-demos/tree/1238d0c444f388cb1ee79c351a57610e03e4bbb6/R-Python
as long as the scripts are executable then you can just invoke them directly from within your task
Guillaume Theaud
@GuillaumeTh
@pditommaso I saw that now in the .command.run the values for the environment variable are written like MY_VAR=\"0\" whereas in the 19.04 the variable was MY_VAR="0". This modification is normal ?
Stephen Kelly
@stevekm

@banjosnapper instead of calling your script with perl myscript.pl you should instead put a shebang at the very first line that invokes the perl interpreter, see this Python script that includes one: https://github.com/stevekm/nextflow-demos/blob/1238d0c444f388cb1ee79c351a57610e03e4bbb6/R-Python/bin/test.py

this allows you to invoke the script as simply myscript.pl

Paolo Di Tommaso
@pditommaso
@GuillaumeTh :point_right: nextflow-io/nextflow#1146
Guillaume Theaud
@GuillaumeTh
@pditommaso thanks
@pditommaso The problem that I have in my case is that in the config file I set env {NB_PROCESSES=1} But with the fix the value of NB_PROCESSES="1" but some scripts requiered a int not a string. Do you have a idea to set a int instead of a string in the nextflow.config ?
Paolo Di Tommaso
@pditommaso
Don't think env vars distinguish between num and string
Stephen Kelly
@stevekm

sometimes when I restart a Nexflow pipeline with -resume, I get errors inside my processes such as java.lang.NullPointerException: Cannot get property 'outputDir' on null object. These come from processes that look like this:

process vcf_to_tsv {
    publishDir "${params.outputDir}/VEP/vcf_tsv", mode: 'copy'
    // stuff
}

so it seems like params is not always getting initialized correctly when the pipeline resumes.

I have the same error for dict objects that I initialize from reading in a JSON. Sometimes, the pipeline errors out when I try to access the object's keys inside a process. Problem is, all of these errors are completely random and not reproducible. Eventually if I restart the pipeline enough times with -resume, they go away. Or not, and I just start the pipeline from scratch... is this a known bug? I think we are still using 19.01

here is one such stack trace:
Dec-18 09:01:45.818 [PathVisitor-1] ERROR nextflow.Channel - Cannot get property 'outputDir' on null object
java.lang.NullPointerException: Cannot get property 'outputDir' on null object
        at org.codehaus.groovy.runtime.NullObject.getProperty(NullObject.java:60)
        at org.codehaus.groovy.runtime.InvokerHelper.getProperty(InvokerHelper.java:190)
        at org.codehaus.groovy.runtime.callsite.NullCallSite.getProperty(NullCallSite.java:46)
        at org.codehaus.groovy.runtime.callsite.GetEffectivePogoPropertySite.callGetProperty(GetEffectivePogoPropertySite.java:45)
        at _nf_script_121eedc3$_run_closure117$_closure442.doCall(_nf_script_121eedc3:2192)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:104)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:326)
        at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1041)
        at groovy.lang.Closure.call(Closure.java:411)
        at groovy.lang.Closure.call(Closure.java:405)
        at groovy.lang.GString.writeTo(GString.java:189)
        at groovy.lang.GString.toString(GString.java:153)
        at org.codehaus.groovy.runtime.typehandling.ShortTypeHandling.castToString(ShortTypeHandling.java:55)
        at nextflow.extension.Bolts.asType(Bolts.groovy:474)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.groovy.runtime.metaclass.ReflectionMetaMethod.invoke(ReflectionMetaMethod.java:54)
        at org.codehaus.groovy.runtime.metaclass.NewInstanceMetaMethod.invoke(NewInstanceMetaMethod.java:56)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:326)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1235)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1041)
        ......
Paolo Di Tommaso
@pditommaso
it seems an app issue, outputDir is not a NF variable
Stephen Kelly
@stevekm
It is defined inside the main.nf pipeline;
params.outputDir = "output"

process vcf_to_tsv {
      publishDir "${params.outputDir}/VEP/vcf_tsv", mode: 'copy'
....
....
}
this way the user is able to override it from the CLI
nextflow run main.nf --outputDir foo/
I thought this was standard practice?
Paolo Di Tommaso
@pditommaso
weird, try the latest version, if the problem persits open an issue
Stephen Kelly
@stevekm
ok
what was the arg to disable the ANSI logger in the latest?
Paolo Di Tommaso
@pditommaso
-ansi-log false
Stephen Kelly
@stevekm
oh sorry I meant the environment variable...
Paolo Di Tommaso
@pditommaso
NXF_ANSI_LOG=false
Stephen Kelly
@stevekm
I get a lot of these... WARN: The channel 'create' method is deprecated -- it will be removed in a future release
:(
Ha Le
@haqle314
Yeah, disabling ANSI logger just means that nextflow will print out plain logs without color, '\r', etc. I think nextflow should print warning messages to stderr instead of stdout as it currently does. This way, people that dont want to see the warnings can do nextflow ... 2>/dev/null.
Stijn van Dongen
@micans
Is it possible not to use the time directive? In some cases I'd like to depend on whatever the (LSF) queue maximum is, perhaps in a process-specific manner. This would also avoid the problem that it is a (bsub) error to specify a time limit larger than the queue limit. And can this be done also using override, so e.g. default might be 12.h but for some process I want to turn it off.
Adrian Viehweger
@phiweger
using DSL2, how can I specify that a process uses 2 channels as input, when one channel only contains 1 item, ie channel A has "asm = [s1, s2]" and channel B has "db = [db]" and I want to perform some process with [s1, db] and [s2, db]. I tried foo(asm, db) but this starts only [s1, db] and ignores [s2, db].
Steve Frenk
@sfrenk
@phiweger Are you sure that s1 and s2 are being emitted separately rather than as a single list? eg. you may need to use flatten() or flatMap()
Has anyone had experience using Spock or JUnit (or other unit testing framework) with DSL2? I'm trying to figure out a way of getting the Specification class to have access to processes and channels.
Krittin Phornsiricharoenphant
@sinonkt
@pditommaso Just want you guys know that DSL2 is really awesome, That helps a lot Thank you so much!
@zhqu1148980644 Thank you! for pointing penv out, I never noticed this before! That’s improve readability a lot.
Krittin Phornsiricharoenphant
@sinonkt
I wonder, how can i import/include snippet of groovy code from url like github source for some of my helper function to be able to share on many project as a kind of lib.
I familiar that i found some gist that can do some kind of source from url in groovy code.
deo999
@deo999
hi
nextflow run <pipeline name> -with-report [file name]
i want to run this but have confusion what i should give in pipeline name and file name
Alaa Badredine
@AlaaBadredine_twitter
ermm
deo999
@deo999
file name will be .nf file then what will be the pipeline name
Alaa Badredine
@AlaaBadredine_twitter
the pipeline name stands for your script.nf and the -with-report is anything u want
deo999
@deo999
what about with docker
nextflow run main.nf -with-docker -with-report result
does this commnd ok with docker