Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 14 2017 11:03
    bmpvieira unassigned #16
  • Jun 02 2017 13:52
    bmpvieira commented #16
  • Jun 02 2017 13:44
    thejmazz commented #16
  • Jun 02 2017 12:05
    bmpvieira assigned #16
  • Jun 02 2017 00:12
    bmpvieira labeled #16
  • Jun 02 2017 00:09
    bmpvieira labeled #16
  • Jun 01 2017 23:46
    bmpvieira labeled #15
  • Jun 01 2017 22:10
    bmpvieira labeled #15
  • Jun 01 2017 22:10

    bmpvieira on master

    Create CODE_OF_CONDUCT.md addr… Merge pull request #15 from aya… (compare)

  • Jun 01 2017 22:10
    bmpvieira closed #15
  • Jun 01 2017 22:09
    bmpvieira review_requested #15
  • Jun 01 2017 22:09
    bmpvieira assigned #15
  • Jun 01 2017 21:09
    bmpvieira opened #16
  • Jun 01 2017 16:55
    ayangromano opened #15
  • Apr 10 2017 14:43
    bmpvieira unlabeled #9
  • Apr 10 2017 13:05
    bmpvieira unlabeled #12
  • Apr 06 2017 08:13
    bmpvieira commented #14
  • Apr 06 2017 07:46
    pditommaso commented #14
  • Apr 05 2017 18:07
    bmpvieira review_requested #14
  • Apr 05 2017 18:07
    bmpvieira assigned #14
Julian Mazzitelli
@thejmazz
At some point I'll go through the old private chat and bring out important points/links that I think should be here. A public chat is better b/c we can now collaborate with others much simpler!
Julian Mazzitelli
@thejmazz
Whitepaper on Nextflow + Docker + Univa Grid Engine on cluster: http://www.nextflow.io/misc/Univa-Docker-Whitepaper_FINAL.pdf
Julian Mazzitelli
@thejmazz
Julian Mazzitelli
@thejmazz
@/all draft post on NGS workflows: bash, make, snakemake, nextflow, bionode proposal https://github.com/thejmazz/jmazz.me/blob/master/content/post/ngs-workflow.md
Max Ogden
@maxogden
@thejmazz can you link me to the hangout?
@thejmazz i have https://plus.google.com/events/clroo9s4r4tqvj49v1bco7aebd8 but it wotn let me join
Julian Mazzitelli
@thejmazz
sec, yeah I cant start it b/c bionode@bionode.io needs to have hangouts enabled from google apps admin
so Im making my own right now
or someone call volunteer to screen record lol
Max Ogden
@maxogden
ok we can use https://bit.ly/datchat if you want
oh you wanna record it, gotcha
ya just make your own then
Julian Mazzitelli
@thejmazz
yeah, we have meet.bionode.io already for misc non recorded chats lol
Max Ogden
@maxogden
can you linkme to the actual hangout
i cant join from that page
@thejmazz i think you have to open the hangout page and link the url from the popup window
Julian Mazzitelli
@thejmazz
true
I also invited to email
Julian Mazzitelli
@thejmazz
where do I go to see the video now loll..
Julian Mazzitelli
@thejmazz
Hey I got a first basic pipeline with somewhat of the proposed api working. It's blocking, so does sra download, then uses globby based on provided output pattern to call fastq-dump with proper param
Im going home now but gonna work more tonight
For handling steams for this cat sra | fastq-dump, I think there are 2 ways to go:
Have ncbi.downloadStream(db, accession) which wont give progress but the actual http stream (could also be a search and then fetch I just realized) OR
check fs for files matching the pattern either on a repeat timeout or on first .on('data') or .on('start'),and then make an fs.createReadStream to pipe that to stdin for fastq-dump
@maxogden @mafintosh
Julian Mazzitelli
@thejmazz
probably worthwhile looking at the commits which will better explain how I attempted approaching specific things
Julian Mazzitelli
@thejmazz
Im also thinking instead of curry functions, tasks could be even emitters that throw finished when they have data to pass into the next task in the "task stack"
I feel as well am I kinda stumbling into the way middleware call stack might work in express
*event not even. Though that might be obvious lol
Max Ogden
@maxogden
@bmpvieira which of us is doing the mid term evaluation btw?
Bruno Vieira
@bmpvieira
@maxogden @mafintosh @yannickwurm I'll share a gdoc and once we all agree I can submit it.
Julian Mazzitelli
@thejmazz
@mafintosh asked what this line does
Julian Mazzitelli
@thejmazz

it is using ES6 arrow functions to make writing a curry function easier, consider

const odds = [0, 2, 4].map(num => num+1)

which could also have been

const odds = [0, 2, 4].map(function(num) {
   return num+1
})

Same thing with functions:

const task = (props, cb) => (next) => () => {
   // stuff
}

which could also have been

function task(props, cb) {
  return function(next) {
    return function() {
        // stuff
    }
  }
}

its just thunked to delay execuation so that you need to do

pipeline().on('data', ...)

rather than

pipeline.on('data')

and making it a curry function instead of function(props, cb, next) (which it was in the previous commit), was so that join can make the glue next functions programmtically - each task should not have to declare whats next, but you should be able to then join them

and as well mafintosh also asked:
"why are tasks not duplex streams instead of taking input/output?"
to which I say, this fist example is blocking, assumes the task completes fully before moving onto the next, will be using duplex streams for tasks that can be streamed into/out of each other - but we also need to handle the case where it is blocking and needs to finish. so that could be a job for join or depend on how the task is written
Max Ogden
@maxogden
@bmpvieira hey how are we submitting the final evaluation? it says only 1 mentor can submit... should i do it?
Julian Mazzitelli
@thejmazz
XD
Max Ogden
@maxogden
@thejmazz :P
@bmpvieira oh wait nevermind, when i go into https://summerofcode.withgoogle.com/dashboard/ it says its completed now, ill just ignore the email