Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 12 22:56
    dependabot[bot] labeled #7
  • Aug 12 22:56
    dependabot[bot] labeled #7
  • Aug 12 22:56
    dependabot[bot] opened #7
  • Aug 12 22:56
    dependabot[bot] opened #7
  • Aug 12 22:56

    dependabot[bot] on npm_and_yarn

    Bump path-parse from 1.0.6 to 1… (compare)

  • Aug 12 22:56

    dependabot[bot] on npm_and_yarn

    Bump path-parse from 1.0.6 to 1… (compare)

  • May 11 23:24
    dependabot[bot] labeled #6
  • May 11 23:24
    dependabot[bot] labeled #6
  • May 11 23:24
    dependabot[bot] opened #6
  • May 11 23:24
    dependabot[bot] opened #6
  • May 11 23:24

    dependabot[bot] on npm_and_yarn

    Bump lodash from 4.17.20 to 4.1… (compare)

  • May 11 23:24

    dependabot[bot] on npm_and_yarn

    Bump lodash from 4.17.20 to 4.1… (compare)

  • Nov 20 2020 18:04

    bmpvieira on master

    Refactor to add CLI 3.0.0 (compare)

  • Nov 20 2020 18:04

    bmpvieira on master

    Refactor to add CLI 3.0.0 (compare)

  • Sep 02 2020 15:27
    dependabot[bot] labeled #6
  • Sep 02 2020 15:27
    dependabot[bot] labeled #6
  • Sep 02 2020 15:27
    dependabot[bot] opened #6
  • Sep 02 2020 15:27

    dependabot[bot] on npm_and_yarn

    Bump bl from 1.2.2 to 4.0.3 Bu… (compare)

  • Sep 02 2020 15:27
    dependabot[bot] opened #6
  • Sep 02 2020 15:27

    dependabot[bot] on npm_and_yarn

    Bump bl from 1.2.2 to 4.0.3 Bu… (compare)

Anders Pitman
@anderspitman
Just trying to minimize dependencies. The repo we're adding the web app is already like 60% R, 20% Perl, and 2 or 3 other things mixed in. I'd prefer to avoid adding node
The project is also a soft intro to JS for a few of the people in our lab
Julian Mazzitelli
@thejmazz

anyways, if you have any ideas what would be a good bionode-fasta demo (for a clientside react/angular/vue app), let me know
hmmm. well. are you running seqentiall steps for a pipeline with scripts in those languages?

I don't mean to come off aggressive about it (sorry if so), just curious and trying to gain perspective

Anders Pitman
@anderspitman
you're fine
yeah it's a pipeline
do you really need a demo for fasta?
seems pretty ubiquitous
Julian Mazzitelli
@thejmazz

our goal with watermill is to have a node pipeline runner manager and oversee the pipeline process

there are some neat things to show off using https ndjson streams clientside + webgl visualizations

but yah fasta on its own somewhat difficult to demo lol
we would like to add play-with-docker images though
where you could be in a container with that specific cli already installed
Tiago Jesus
@tiagofilipe12
fasta-parseris a dependency of bionode-fasta, so I think the one to use is bionode-fasta. The other one is just a lib, but maybe it can interest someone anyway. I have had this discussion with @bmpvieira before. And we should make this more clear in the Readme of both repos.
Bruno Vieira
@bmpvieira
fasta-parser is a "Buffer Stream parser from FASTA to JSON.", which means it's just a Node.JS Stream that takes an input Streamed Buffer containing data in fasta format and Streams out Buffer containing JSON. There should be no major speed issue since it doesn't not serialise anything to object. You can think of fasta-parser as just the parsing algorithm (and maybe we can even improve it for speed).
bionode-fasta is a bionode module that uses fasta-parser as a dependency and does a lot more, to follow the bionode standards and be more user friendly (i.e. read fasta files, provide a command line interface, etc)
bionode-fasta only serialises the data to JSON if you use it in Object mode (i.e., fasta.obj(), please see https://github.com/bionode/bionode-fasta#usage)
Bruno Vieira
@bmpvieira
if you do npm run build-browser inside bionode-fasta git repository, it should generate a bionode-fasta.min.js that you can use browser side
Bruno Vieira
@bmpvieira
this is achieved with browserify and the file is a bit big (408k at the moment) because it kind of includes Node's API. If you end up using other Node modules, you can browserify them all together so that you don't duplicate Node's code for each one in the minified files
If you find any issue or have more questions, please let us know here or on GitHub :)
Anders Pitman
@anderspitman
:thumbsup:
Do you happen to know about how big fasta-parser is off the top of your head?
Hitesh Joshi
@evoxtorm
Hey I have knowledge of ES5 and ES6 and started working on nodejs , so now which project is good for working as a beginner in nodejs
Tiago Jesus
@tiagofilipe12
Nowadays our main focus of development is focused in bionode-watermill (https://github.com/bionode/bionode-watermill). We are starting to have more documentation but since it is something that me and @thejmazz are working on a daily basis some things might get difficult to understand. We are currently working on the somewhat stable dev branch. However, bionode-watermill might be hard for beginners, but if you want to give it a try... we are here to help.
Tiago Jesus
@tiagofilipe12
Then, there are a lot of other modules still in draft or under-documented (https://github.com/bionode) that are simple and you can also have a look at. For instance, bionode-fastq (https://github.com/bionode/bionode-fastq), I have made a branch for it but I would like that someone else gives it a look and it still needs a CLI (https://github.com/bionode/bionode-fastq/tree/Refactoring).
You can have a look at the issues of each bionode module and see if you can start with some of those issues. Otherwise, you may test some of the documented modules and start playing around to see if something can be improved.
Hitesh Joshi
@evoxtorm
Thanks for the guidance @tiagofilipe12
Julian Mazzitelli
@thejmazz
Hitesh Joshi
@evoxtorm
hey I have question??? The bionode-watermill is purely made on nodejs and html or any other language is used in it!!!!
Julian Mazzitelli
@thejmazz
Yes its only javascript. But it is used to call CLI programs which can be in any language
Tiago Jesus
@tiagofilipe12
And html is just a quick visualization tool for pipeline workflow graph
Hitesh Joshi
@evoxtorm
thanks
cli- command line interface or anything else
Julian Mazzitelli
@thejmazz
Look at nodejs child_process, if you return a string from an operationCreator for a task in watermill it will use child_process.spawn with { shell: true } (so you can use unix pipes)
Otherwise it can return a function with callback, promise, or node stream (though I think the code that checks for this is WIP - and probably we should add an option to task for user to declare what their operationCreator returns)
So basically anything you can run from terminal
Julian Mazzitelli
@thejmazz
And the stdout, stdin, stderr of that process are made available as node streams by the child_process module
Julian Mazzitelli
@thejmazz

One cool thing to do would be to use ES6 template literals to do something like

const { task, python } = require('bionode-watermill')

const mytask = task({
  input: '*.txt',
  output: '*.foo',
  name: 'My task'
}, ({ input }) => python`
  with open('output.foo') as output:
      with open('${input}') as input:
          for line in input:
            output.write(line.upper())
`)

// where `python` returns something like
// { language: 'python', executer: '/usr/bin/env python3', code }
// watermill writes the script, marks it executable, stores it in task dir

then it would be frictionless to experiment with quick scripts in a bunch of languages, write small simple scripts for visualization with R, etc. using ${variable in source code for a script is kinda sketchy but maybe OK for quick experimentaion. Nextflow has something similar for scripts in any language (you just put #!/usr/bin/env python3 at top)

OR maybe that program should go under a prop of the task, so it is called as a CLI like usual

(excuse my python im not sure if its correct)
the operationCreator is the second paramater passed to task, the first is the object props
minoltafan
@minoltafan
hello, world =)
Hitesh Joshi
@evoxtorm
hey bionode-ncbi is a beautiful library and I used this library in my project
and I'm thinking I will give a small presentation on bionode-ncbi telling other guys how to use it
My professors are loving this library
Bruno Vieira
@bmpvieira
Hey @evoxtorm ! Super happy to read that! :smile:
Thanks for using it! Would love to see those slides and know more about how you are using bionode-ncbi in your work
Hitesh Joshi
@evoxtorm
ok sure sure I'll share with you
Hitesh Joshi
@evoxtorm
Hey guyz,
Does anyone have data flow diagram of bionode-ncbi because I have to show how this library actually works
Julian Mazzitelli
@thejmazz
its mostly filtering/wrapping ncbi API calls, making a stream out of paginated response s, etc, there is no diagram I am aware of. is this use case being able to explain underlying technicalities so that others feel more comfortable with the software?