Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 13 20:12
    haqle314 edited #1425
  • Dec 13 18:37
    kemin711 opened #1426
  • Dec 13 17:32
    bobamess commented #1414
  • Dec 13 17:23
    tverbeiren commented #1424
  • Dec 13 17:22
    tverbeiren commented #1424
  • Dec 13 17:21
    haqle314 opened #1425
  • Dec 13 15:44
    JonathanCSmith commented #1424
  • Dec 13 15:25
    tverbeiren commented #1424
  • Dec 13 14:39
    tverbeiren commented #1249
  • Dec 13 11:34
    bobamess commented #1414
  • Dec 13 11:00
    JonathanCSmith opened #1424
  • Dec 13 10:20
    pditommaso commented #1409
  • Dec 12 23:40
    haqle314 commented #1422
  • Dec 12 19:08
    emorice commented #1419
  • Dec 12 18:50
    brgew opened #1423
  • Dec 12 16:52
    micans opened #1422
  • Dec 12 15:25
    daudn edited #1421
  • Dec 12 15:24
    daudn edited #1421
  • Dec 12 15:21
    daudn opened #1421
  • Dec 12 15:15
    emorice synchronize #1419
HashRocketSyntax
@HashRocketSyntax
Or, when handling spot instance interrupt, should a resume be attempted on another server (is that possible?)
mmatthews06
@mmatthews06
On a completely different note, version 19.07.0 is giving me GitHub authorization issues that version 19.04.1 is not. Or, sometimes, this error:
WARN: Cannot read project manifest -- Cause: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
mmatthews06
@mmatthews06
mmatthews06
@mmatthews06
ardydavari
@ardydavari
I have a newbie question regarding passing the same variable multiple times to a command.
```
srr_ch = Channel.from( "SRR519926", "SRR1553607")

 process fastqdump {
   container 'quay.io/biocontainers/parallel-fastq-dump:0.6.5--py_0'

   input:
   each srr from srr_ch
   output:
   file("*.fastq") into fastq_ch

   """
   prefetch ${srr} && parallel-fastq-dump -t 8 -X 10000 --split-files -s ${srr}
   """
   }
Channel `srr_ch` has been used twice as an input by process `fastqdump` and another operator
ardydavari
@ardydavari
I believe i found the error:
srr_ch = Channel.from( "SRR519926", "SRR1553607")
srr_ch.println()


 process fastqdump {
   container 'quay.io/biocontainers/parallel-fastq-dump:0.6.5--py_0'

   input:
   each srr from srr_ch
   output:
   file("*.fastq") into fastq_ch

   """
   prefetch ${srr} && parallel-fastq-dump -t 8 -X 10000 --split-files -s ${srr}
   """
   }
println() was consuming the channel prior to routing to fastqdump
Paolo Di Tommaso
@pditommaso
indeed!
Ashley S Doane
@DoaneAS

Hi @pditommaso and all. I'm running a nextflow pipeline for realigning bam files. I have 4000+ samples, and ~2600 of these have completed. Running nextflow -resume main.nf crashed our interactive server, which has 192 CPUs and 10T RAM.

The admins told me that system-wide CPU use from nextflow processes was over %1000. I am using sge as executor and the crash seems to happen when nextflow is reading which samples are cached. Here's output when system became unstable:

executor >  sge (3)
[6e/65371e] process > speedyseq (EGAR00001156495_10698_5_7_cram) [100%] 2613 of 2616, cached: 2613
Rad Suchecki
@rsuchecki
Is that about excess cpu use on the head node? Can you submit your nextflow process as a job? @DoaneAS
Rad Suchecki
@rsuchecki
Paolo et al will correct me if I am wrong but I think one possibility could be that JVM heap space was running low, leading to excessive GC activity. Is pipeline code available to view? @DoaneAS
Daniel E Cook
@danielecook
Anyone know why Date.parse doesn't work?
Maxime Garcia
@MaxUlysse
Hi @pditommaso I have some follow up question on @drpatelh question here
I understand why we can't have traverse a directory over http, but is it possible to have something like https://raw.githubusercontent.com/maxulysse/test-datasets/sarek/file{1,2}.ext become two files: https://raw.githubusercontent.com/maxulysse/test-datasets/sarek/file1.ext https://raw.githubusercontent.com/maxulysse/test-datasets/sarek/file2.ext as it is with a regular path?
Paolo Di Tommaso
@pditommaso
yes, indeed
the glob is expanded to the actual files
Daniel E Cook
@danielecook
This works:
import java.text.SimpleDateFormat;
date_parse = new SimpleDateFormat("yyyy-MM-dd");
date_filter = date_parse.parse(params.date)
Paolo Di Tommaso
@pditommaso
well, datetime is a mess in any programing lang
Maxime Garcia
@MaxUlysse
It is?? I'll go check my tests then, I just tried and it wasn't working
Paolo Di Tommaso
@pditommaso
well, actually I'm not sure :joy:
Maxime Garcia
@MaxUlysse
It works
I must had another issue when I tried earlier
I'm guessing it'll work as well with s3://
Paolo Di Tommaso
@pditommaso
s3 can be traversed as a regular fs
Maxime Garcia
@MaxUlysse
Ok
thanks
Ashley S Doane
@DoaneAS
@rsuchecki thanks, yes pipeline is here: https://github.com/DoaneAS/realign.nf
Paolo Di Tommaso
@pditommaso
there could be something wrong in your script, NF is not supposed all that CPU
Daniel E Cook
@danielecook
@pditommaso agreed - but I'm curious why the groovy Date module doesn't work? Is it not imported?
Paolo Di Tommaso
@pditommaso
actually I'm not aware of it, therefore is not imported :smile:
is tehre a groovy-date module ?
Ashley S Doane
@DoaneAS
@rsuchecki pretty simple nextflow with only 1 process actually. Just takes a sampleindex.csv file with sample name, sample type (tumor or normal), and bam file path, and does a realignment with speedseq. The executor is sge, and the error seems to happen when nextflow is determining which results are cached.
Daniel E Cook
@danielecook
err maybe not - but the language spec suggests you should be able to do just Date.parse(format, input)
examples also present here: http://groovy-lang.org/single-page-documentation.html; Just curious maybe I'm missing something here
Paolo Di Tommaso
@pditommaso
can't check now, you may wont to report an issue
Daniel E Cook
@danielecook
I can do that
thanks
Paolo Di Tommaso
@pditommaso
welcome
Maxime Garcia
@MaxUlysse
OK, so I made a mistake earlier, it is not working in fact
I'll make a minimal example and an issue
Sri Harsha Meghadri
@harshameghadri
Hey folks, I am pretty new to using docker and singularity. I want to use the nf-core/rnaseq for my analysis. I consistently get errors while trying to execute this command singularity pull --name nf-core-rnaseq-1.3.img docker://nf-core/rnaseq:1.3
Unable to pull docker://nf-core/rnaseq:1.3: conveyor failed to get: Error reading manifest 1.3 in docker.io/nf-core/rnaseq: errors:
I am trying to pull to rackham. My analysis needs to be executed on bianca. Any tips are super appreciated.
Maxime Garcia
@MaxUlysse
try singularity pull --name nf-core-rnaseq-1.3.img docker://nfcore/rnaseq:1.3 instead, without the - in nf-core
I'm guessing if you have more question about nf-core pipelines, it should be better on our slack: https://nf-co.re/join
Sri Harsha Meghadri
@harshameghadri
I tried that as well getting the same error @MaxUlysse
Maxime Garcia
@MaxUlysse
You tried that on bianca?
Sri Harsha Meghadri
@harshameghadri
nope on rackham, I guess it needs internet