Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 10 19:58
    BoPeng opened #320
  • May 28 15:18

    BoPeng on test

    Skip a test (compare)

  • May 28 14:36
    BoPeng commented #319
  • May 28 14:36
    BoPeng closed #319
  • May 28 14:35

    BoPeng on master

    Release sos notebook 0.22.5 (compare)

  • May 28 14:26

    BoPeng on master

    Add a dependency Add toggle class with_console_p… Add toggle class active to pane… and 7 more (compare)

  • May 28 14:25

    BoPeng on console_css

    Remove with_console_panel, whic… (compare)

  • May 28 10:41
    csemken commented #319
  • May 28 10:40
    csemken commented #319
  • May 27 22:17

    BoPeng on stash

    Stash and restore output files (compare)

  • May 27 18:31
    BoPeng commented #319
  • May 27 18:30

    BoPeng on console_css

    Keep displayed settings in note… (compare)

  • May 27 14:22
    BoPeng commented #319
  • May 27 14:21
    BoPeng commented #319
  • May 27 08:46
    csemken commented #319
  • May 27 08:36
    csemken commented #319
  • May 26 23:17
    BoPeng commented #319
  • May 26 23:16

    BoPeng on console_css

    Add a dependency Add toggle class with_console_p… Add toggle class active to pane… and 5 more (compare)

  • May 26 14:39
    BoPeng commented #319
  • May 26 12:54
    csemken commented #319
Christoph Semken
@csemken
Dear @BoPeng, thank you for providing us with SoS, it is an impressive and wonderful project. I am using it in a workshop on programming paradigms and tools for scientific programming. Specifically, I am using sos-notebook and RISE slideshow to run code during the presentations. It works great. The only problem I am having is that the Console opens automatically, interfering with the slides wherever I cannot close it manually first. This happens with the RISE presenter view (see screenshot above) and decktape pdf export. Is there a way to disable the console view by default?
Bo
@BoPeng
RISE slideshow looks cool and I have never used it. The feature is not available now but it is certainly valid, and relatively easy to add. I will check standard practice for configurations for third-party plugins but generally speaking it can be added to jupyter configuration file, or for sos notebook, ~/.sos/config.yml. The name of the confgiraution could be 'notebook_console_panel=on/off`, but in any case could be turned on or off from the toolbar.
It is now vatlab/sos-notebook#319 . @csemken BTW, can you use RISE presenter with Jupyterlab, potentially with jupyterlab-sos?
Christoph Semken
@csemken

RISE slideshow looks cool and I have never used it. The feature is not available now but it is certainly valid, and relatively easy to add. I will check standard practice for configurations for third-party plugins but generally speaking it can be added to jupyter configuration file, or for sos notebook, ~/.sos/config.yml. The name of the confgiraution could be 'notebook_console_panel=on/off`, but in any case could be turned on or off from the toolbar.

Thanks for the quick response! This should do the trick.

It is now vatlab/sos-notebook#319 . @csemken BTW, can you use RISE presenter with Jupyterlab, potentially with jupyterlab-sos?

Unfortunately not. RISE is not yet compatible with Jupyterlab, see damianavila/RISE#270.

Please let me know if you need any other info. I am happy to add it here or in vatlab/sos-notebook#319
Bo
@BoPeng
@csemken Push a quick fix, please comment on the ticket to see if it works. I do not really like the look of the notebook without console panel. but let us first see if the patch works for you.
Patrick Cudahy
@pgcudahy
Hello, I have what I think is a simple question, but can't quite find out how to do it. Some of my workflow steps take in dozens of files from the prior step but also need a single file from a totally different step. For example I need map hundreds of fastq files, but only to a single reference that is downloaded in a separate step. This doesn't work:
[index-reference]
output: f'/home/pgc29/scratch60/helen_mixed_infection/dataraw/NC_000962/NC_000962.3.fasta',
f'/home/pgc29/scratch60/helen_mixed_infection/dataraw/NC_000962/NC_000962.3.fasta.bwt'
task: walltime='00:05:00', mem='1G', trunk_size=1, trunk_workers=1, 
    workdir='/home/pgc29/scratch60/helen_mixed_infection/dataraw/NC_000962'

run: expand=True
    module load miniconda
    conda activate entrez-direct
    esearch -db nuccore -query "NC_000962.3" | efetch -format fasta > \
        /home/pgc29/scratch60/helen_mixed_infection/dataraw/NC_000962/NC_000962.3.fasta
    module load BWA
    bwa index /home/pgc29/scratch60/helen_mixed_infection/dataraw/NC_000962/NC_000962.3.fasta

[map]
input: output_from("trimmomatic"), group_by=1, f'/home/pgc29/scratch60/helen_mixed_infection/dataraw/NC_000962/NC_000962.3.fasta'
output: f'/home/pgc29/scratch60/helen_mixed_infection/data/bam/\
{os.path.split(_input[0].resolve())[1].split("_reads", 1)[0]}_"GCF_000195955.2"_filtered_sorted.bam'
task: walltime='00:10:00', mem='10G', trunk_size=38, trunk_workers=10, 
    workdir='/home/pgc29/scratch60/helen_mixed_infection/data/bam'

id=os.path.split(_input[0].resolve())[1].split("_mtb_", 1)[0]
rg=f'\"@RG\\tID:{id}\\tPL:Illumina\\tSM:{id}\"'

run: expand=True
    module load BWA SAMtools
    bwa mem -v 3 -Y -R {rg} \
        {_input[0]} \ 
{_input[1]} | \
    samtools view -bu - | \
    samtools sort -T /home/pgc29/scratch60/helen_mixed_infection/data/bam/tmp1.{id} -o {_output}
Bo
@BoPeng
Did you check https://vatlab.github.io/sos-docs/doc/user_guide/group_by.html ? it looks like group_by='pairlabel'
Bo
@BoPeng
Or you could put group_by inside output_from to have some thing like output_form('trimmomatic', group-by=1), output_from('index-reference'). Use print(_input) to test your step before running anything.
Patrick Cudahy
@pgcudahy
Ah, with pairlabel, would it group the fastq files by 1?
When I've tried using multiple output_from inputs it usually complains if they're of unequal lengths
Bo
@BoPeng
Yes, and there are also pairlabel2, pairlabel3 etc. There is a summary table over there.
When I've tried using multiple output_from inputs it usually complains if they're of unequal lengths
Patrick Cudahy
@pgcudahy
Got it! Thanks very much for your patient help
Bo
@BoPeng
When one of the output_from has a single group, it will propagate to match the number of groups of other output_from.
I think moving group_by inside output_from is the most natural method.
Milo Shields
@miloshields:matrix.org
[m]
Hi! I'm trying to link SoS with this in-browser javascript kernel: https://github.com/stdlib-js/jupyter-stdlib-browser-kernel, and I was wondering if it would be as simple as adding the name of the kernel to the sos-javascript module, as it's JavaScript, although I'm not sure it's technically supported. Any pointers would be greatly appreciated!
Bo
@BoPeng
You can try it with
%use JavaScript --kernel kernel_name
here kernel_name should be the one listed with jupyter kernelspec list
If it works, I will be happy to add it to the supported kernel list for JavaScript.
@miloshields:matrix.org
Milo Shields
@miloshields:matrix.org
[m]
Thanks! I'll try that out right now.
Bo
@BoPeng
I guess it did not work?
Milo Shields
@miloshields:matrix.org
[m]
Yep... Getting this error:
Exception in event handler for select.Cell TypeError: Cannot read property 'send' of null
    at Kernel._send (main.min.js?v=931f67…555340d3f25c5:38180)
    at Kernel.send_shell_message (main.min.js?v=931f67…555340d3f25c5:38197)
    at Comm.send (main.min.js?v=931f67…555340d3f25c5:37361)
    at send_kernel_msg (kernel.js?v=20210616115701:1169)
    at window._Events.notify_cell_kernel (kernel.js?v=20210616115701:1264)
    at window._Events.dispatch (main.min.js?v=931f67…eb96555340d3f25c5:2)
    at window._Events.v.handle (main.min.js?v=931f67…eb96555340d3f25c5:2)
    at Object.trigger (main.min.js?v=931f67…eb96555340d3f25c5:2)
    at window._Events.<anonymous> (main.min.js?v=931f67…eb96555340d3f25c5:2)
    at Function.each (main.min.js?v=931f67…eb96555340d3f25c5:2) 
Arguments(2) ["select.Cell", {…}, callee: (...), Symbol(Symbol.iterator): ƒ]
I'll do some digging and see if I can find anything useful.
Bo
@BoPeng
ok, let me have a look.
The installation process of this kernel looks very dangerous (going to jupyter directory and git clone)...
Bo
@BoPeng
Saw the error. The kernel was trying to execute something at https://github.com/stdlib-js/jupyter-stdlib-browser-kernel/blob/5844a62eca5ca4e0ffa992c5f307149b57ba3895/kernel.js#L174 but I am not sure what it is trying to do.
So, what exactly does "in-browser" means? Does it use the browser's JS engine to compuete stuff, even controls the Jupyter cell interface directly?
Milo Shields
@miloshields:matrix.org
[m]
The evil() function is an alias for eval(), so it's just running the code in the cell using that function and then printing the result.
In my version, I'm trying to replace that with a link to another javascript engine, but I don't think that's what the error is coming from.
In terms of communication, I'm pretty sure it follows the same specs that all kernels use, with the 5 (i think) sockets performing different functions
But yeah, the evaluation is in the browser itself
Bo
@BoPeng
I think the problem is that this kernel have direct reference to the JS kernel object that calls its execute function.
However, SoS wraps each kernel in order to pass additional parameters to the SoS kernel before sending stuff to the underlying kernel.
Milo Shields
@miloshields:matrix.org
[m]
interesting
Bo
@BoPeng
  1. take whatever user input
  1. add sos options such as what is the kernel of the current cell
  1. pass to sos kernel for execution
  1. sos then processes the information and call the underlying kernel
Milo Shields
@miloshields:matrix.org
[m]
Right. This is great, thanks!
Bo
@BoPeng
I am not sure at which step this kernel conflicts with SoS but this line of code https://github.com/stdlib-js/jupyter-stdlib-browser-kernel/blob/5844a62eca5ca4e0ffa992c5f307149b57ba3895/kernel.js#L95 , namely kernel.execute = execute basically replaces the SoS my_execute with its own version, so SoS will not be able to function properly.
Bo
@BoPeng
Note that even if SoS does not wrap execute, this kernel tries to execute the cell directly (instead of waiting for SoS to feed the processed code to it) so it will not be able to handle the SoS magics such as %get and %put.
Milo Shields
@miloshields:matrix.org
[m]
Got it. Thanks for your help! I may try modifying the iJavascript kernel instead... I basically just need to replace the actual execute step of any javascript kernel with my own method.
Bo
@BoPeng
Not sure what you are trying to achieve but good luck.
Milo Shields
@miloshields:matrix.org
[m]
Thank you, and thanks for the help!