Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 31 2019 17:58
    jorgemachucav starred galaxyproject/tools-iuc
  • Jan 31 2019 17:45
    bebatut opened #2270
  • Jan 31 2019 16:18
    cpreviti synchronize #2267
  • Jan 31 2019 14:15
    cpreviti synchronize #2267
  • Jan 31 2019 12:42
    bernt-matthias review_requested #2269
  • Jan 31 2019 12:42
    bernt-matthias edited #2269
  • Jan 31 2019 12:41
    bernt-matthias edited #2269
  • Jan 31 2019 12:40
    bernt-matthias synchronize #2269
  • Jan 31 2019 12:13
    cpreviti commented #2267
  • Jan 31 2019 12:07
    nsoranzo commented #2267
  • Jan 31 2019 12:01
    cpreviti synchronize #2267
  • Jan 31 2019 11:21
    cpreviti synchronize #2267
  • Jan 31 2019 09:47
    cpreviti synchronize #2267
  • Jan 31 2019 09:27
    cpreviti synchronize #2267
  • Jan 30 2019 20:38
    bernt-matthias commented #2131
  • Jan 30 2019 20:19
    hepcat72 commented #2239
  • Jan 30 2019 19:50
    lparsons commented #2239
  • Jan 30 2019 18:36
    bgruening commented #2268
  • Jan 30 2019 15:23
    nsoranzo commented #2268
  • Jan 30 2019 15:23
    nsoranzo commented #2267
bgruening
@bgruening:matrix.org
[m]
ok!
M Bernt
@bernt-matthias:matrix.org
[m]
Lets see if this helps. I kind of guess that the pathview bug that was just fixed is the culprit (the wrong version token led to building a mulled container which needed forever and probably also wasted lots of space).
Michal Klinka
@kulivox:matrix.org
[m]
Hi, as a part of my bachelor's thesis I'm creating a wrapper for a cerain tool bundle. One section of the thesis will describe the process of running a tool on Galaxy, but I wasn't able to find any tutorial page or ducumentation explaining what exactly happens when the tool is executed. Is there a documentation describing the process? If so, could someone please point me in the right direction? Thank you
bernt-matthias
@bernt-matthias:matrix.org
[m]

Hi Michal Klinka I'm afraid that there is no comprehensive doc on this problem .. but it would be cool to have :)

The topic is quite complex and depends in parts also on the Galaxy config, e.g. how requirements are resolved (conda/containers) and the the tool, e.g. if there are dynamically discovered datasets or the tool's profile version.

I think the most important part is the generation of the job's working dir. In particular the tool_script.sh (which is derived from interpreting the cheetah code in the tool's command block) and galaxy_JOBID.sh which sets up the environment. The later is the shell script that is executed (how is again depending on Galaxy's job configuration).

Maybe you can start by taking a look at these scripts for an example job? I guess we can help in case of questions.

Then after the job ran results are copied to the final destination and the DB needs some updates .. but I can't tell you much details about this part.

Also the tutorial Creating Galaxy tools from Conda Through Deployment over here https://training.galaxyproject.org/training-material/topics/dev/ may shed some light on some of the parts.
wm75 (Wolfgang Maier)
@wm75:matrix.org
[m]
Question about interaction between <version_command> and <command>:
I have a tool (pangolin), which might update its environment as part of its <command> section. The tool also comes with a helpful --all-versions option that lists its own version as well as those of core dependencies.
If I use that option in <version_command>, however, I'd be getting the versions that were in effect before the updates happening in the <command> block? Anything that could be done here?
Nicola Soranzo
@nsoranzo:matrix.org
[m]
Are the updates idempotent?
You could use a macro to run them in the version command as well
Or (my preferred option), keep version_command simple and print --all-versions at the start of command
wm75 (Wolfgang Maier)
@wm75:matrix.org
[m]
Thanks! I prefer the second option, too, but just for my understanding: are the two blocks using the same job environment?
Nicola Soranzo
@nsoranzo:matrix.org
[m]
version_command is run first, not sure if every runner use the same job env
wm75 (Wolfgang Maier)
@wm75:matrix.org
[m]
Would feel rather hackish to rely on that in any case.
Option 1, running the updates twice should work in principle, but would
come with extra overhead. Plus, I just found out that you can't use tool paramters in the <version_command> block so there's no way to tell whether updating should be done or not (depends on user settings).
Nicola Soranzo
@nsoranzo:matrix.org
[m]
Yup
M Bernt
@bernt-matthias:matrix.org
[m]

which might update its environment as part of its <command> section

The conda env? That sounds wrong to me.

Lucille Delisle
@lldelisle
If someone has time to review galaxyproject/tools-iuc#4497 this would help to reduce the number of steps in the Reference-based RNAseq GTN to make a heatmap with z-score.
cat-bro
@cat-bro
Can I use bwa_mem indexes for bwa_mem2? We've installed in on Galaxy Australia and only just realised it sources its indexes from a different table: bwa_mem2_indexes. There is no data manager for this.
1 reply
wm75 (Wolfgang Maier)
@wm75:matrix.org
[m]
Not an expert on this, but I don't think the index structure is compatible.
The bwa-mem2 index structure has also changed at least once in a backwards-incompatible way (in Oct 2020) so a data table should probably have a version column.
M Bernt
@bernt-matthias
@cat-bro @wm75:matrix.org for reference galaxyproject/tools-iuc#4319
cat-bro
@cat-bro
Thanks @wm75:matrix.org and @bernt-matthias
Michelle Savage
@hujambo-dunia:matrix.org
[m]
Hello 👋 Does anyone know why we include the utils\localization.js file in 100+ of our VUE & JS Galaxy files? Is this necessary since there are only 3 files in the codebase where language can be changed by the user? (e.g. Home Page, Login Page, and Preferences Page)
3 replies
bgruening
@bgruening:matrix.org
[m]
Another new contributor that struggles with the test size limitation: galaxyproject/tools-iuc#4516
Any idea wat to do here?
M Bernt
@bernt-matthias

@bgruening:matrix.org for the particular usecase I was wondering if the ncbi taxonomy data table could be reused.

manual merge and deploy? if this happens more often it would be cool to have a way to disable size checks for certain paths .. maybe an additional file like .tt_skip

on the long run progress with galaxyproject/galaxy#13495 :)

was just wondering if we could use the data on the CVMFS for this case (use cached in the tool's conditional). But its kind of circular, since we would need the data manager or at least the datatable + data on CVMFS before accepting the tool. Also CI would need to be adapted to set the --tool_data_table parameter of planemo test

wm75 (Wolfgang Maier)
@wm75:matrix.org
[m]
Would anyone have any insight into https://github.com/galaxyproject/tools-iuc/pull/4533#issuecomment-1114675492?
In particular, is there supposed to be any inherent difference in dependency management between a regular and a data manager tool?
Marius
@mvdbeek:matrix.org
[m]
As usual, a recent profile version would fix this
I'm writing a comment
wm75 (Wolfgang Maier)
@wm75:matrix.org
[m]
Thanks!
Marius
@mvdbeek:matrix.org
[m]
I'd totally appreciate a review/merge on galaxyproject/tools-iuc#4502
bgruening
@bgruening:matrix.org
[m]
on it
M Bernt
@bernt-matthias:matrix.org
[m]

Just checked recent weekly CI errors. Seems that samtools sort output may depend on the number of used CPUs (sorting is correct, just the order of alignments with the same mapping position may differ). Wondering how to deal with this ..

Just check for the number of lines and maybe some regex (does this work for bam?)?

M Bernt
@bernt-matthias
Currently looking at bowtie2 ... Maybe we just implement https://github.com/samtools/samtools/issues/891#issuecomment-404229654 .. mapper | samtools sort -@2 ... | samtools -@\$GALAXY_SLOTS .. which also solves the problem the parallel sorting and mapping may use to many CPUs
M Bernt
@bernt-matthias
i.e. use a fixed number of cores (1) for sorting and all cores for compressing to bam
Jory Schossau
@JorySchossau
Hi, I'm curious if anyone here could comment on if making a Galaxy Tool would be appropriate for our project? I'm assessing the best way to disseminate our multi-year project as a tool. The tool involves a small (<100) set of input sequences, then uses a combination of our method and a variety of published/vetted methods to produce unified predictions. If we optimize some code, we could get the runtimes down to perhaps less than a day. We can already make an offline tool, so our goal is about making use of better computer resources and a more open science workflow than just a user's laptop. Memory requirements are less than 32GB.
bgruening
@bgruening:matrix.org
[m]
Upps. Answered in the other chat.
But this one is the correct one.
Jory Schossau
@JorySchossau
Would you recommend I try to attend the upcoming Training Day or maybe CoFest if I'd like to learn/ramp up on Galaxy tool development?
bgruening
@bgruening:matrix.org
[m]
Always :)
But to be honest this training here is super cool and you will learn everything that is needed.
It will take you 6h and you know even the details.
We can help you then reviewing your tool before it gets uploaded to the ToolShed ...
Jory Schossau
@JorySchossau
@bgruening:matrix.org Thanks!
pvanheus
@pvanheus
hey there... I'm working on writing a data manager to manage pangolin's data assets. there are 3 of these - pangolin-data, constellations and pangolin-assignment - and they get updated on different schedules (although pangolin-data and pangolin-assignments try and stach in sync). so there are going to be 3 data tables. this is all for a single purpose though, so I'm wondering: make 3 DMs? or make 1 DM with options to update 3 tables? (i.e. either 1, 2 or all 3 tables)
pvanheus
@pvanheus
another DM question - how can the existing contents of the data table be queried? its not in the data table that gets passed to the DM. this was less of a concern for me before, but now I'm working on a DM that potentially could download 100s of MBs of data, and avoiding re-downloading what is already there would be useful.
pvanheus
@pvanheus
one can look inside $__app__ at tool_data_tables but this seems rather nasty....
pvanheus
@pvanheus
I'll note that I used $__app__.tool_data_tables in data_manager_primer_scheme_bedfiles, so I guess that's the way to go
M Bernt
@bernt-matthias
@pvanheus .. the best would be galaxyproject/galaxy#12941 .. unfortunately I did not find time to fix the tests .. and it will only be available from the nextr release
pvanheus
@pvanheus
thanks @bernt-matthias but that wouldn't fix this exact problem - in this case there's a list of releases (on Github) and then a user specifies a range (of dates or versions) that they want to capture... the DM then downloads the ones that are available but missing locally.
M Bernt
@bernt-matthias:matrix.org
[m]
Ah, OK. Fancy.. Then I think $__app__.tool_data_tables sounds reasonable