Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jun 24 20:27
    ksierks synchronize #1321
  • Jun 24 19:04
    ksierks synchronize #1321
  • Jun 24 18:53
    JeffreyThiessen synchronize #1258
  • Jun 24 18:51
    JeffreyThiessen synchronize #1258
  • Jun 24 18:37
    JeffreyThiessen edited #1258
  • Jun 24 18:37
    JeffreyThiessen edited #1258
  • Jun 24 18:35
    JeffreyThiessen synchronize #1258
  • Jun 24 17:01
    joshsadam synchronize #1324
  • Jun 24 16:24
    ksierks synchronize #1321
  • Jun 24 16:19
    ksierks synchronize #1321
  • Jun 24 16:15
    JeffreyThiessen synchronize #1258
  • Jun 23 20:04
    JeffreyThiessen ready_for_review #1258
  • Jun 23 20:04
    JeffreyThiessen edited #1258
  • Jun 23 19:57
    JeffreyThiessen synchronize #1258
  • Jun 23 19:45
    JeffreyThiessen synchronize #1258
  • Jun 23 19:10
    joshsadam closed #1261
  • Jun 23 18:37
    JeffreyThiessen synchronize #1258
  • Jun 23 18:04
    deepsidhu85 opened #1327
  • Jun 23 16:46
    deepsidhu85 synchronize #1261
  • Jun 23 15:04
    joshsadam synchronize #1324
But there's likely more than that
Spring is a big one though
Karin Lagesen
@karinlag
thanks @apetkau
that gives me something to go on :)
Jeevan Karloss
@ajkarloss

Good morning.

We are installing the tools for thee default (Assembly, Annotation, Mentalist, SISTR....) pipelines using shed-tools command mentioned in the documentation page (https://phac-nml.github.io/irida-documentation/administrator/galaxy/existing-galaxy/)

It got timed out while installing and the tools are not installed properly.

Is there a way to delete them and install them from start? When we execute the shed-tools command again, it says the tools are installed already

But they are not
@pvanheus any tips here?
pvanheus
@pvanheus
@ajkarloss this is one of the reasons why I switched to using Singularity for tools. I had to go through multiple steps of manual uninstalling and reinstalling to get everything working.
one workaround though to make conda less painful is to install mamba in the Galaxy conda environment, and set conda_exec in the Galaxy config to point to the mamba executable (i.e. conda_exec: /tool_deps/_conda/bin/mamba in our config)
Jeevan Karloss
@ajkarloss
Ok. I am deleting them from the browser and reinstalling them now
@pvanheus Thanks
Aaron Petkau
@apetkau
Thanks for the advice @pvanheus
If a tool fails to install I tend to delete and re-install from the Galaxy UI: https://galaxyproject.org/admin/tools/add-tool-from-toolshed-tutorial/. But that's maybe not ideal if there are many tools with issues.
pvanheus
@pvanheus
@apetkau its really a Galaxy fault if things don't install right, but unfortunately I see it happening too often - especially with complex tools with lots of (conda) dependencies.
Karin Lagesen
@karinlag
are there ways to do this from the commandline?
(says the person who is more than slightly naive on this front)
pvanheus
@pvanheus
to delete and re-install? not that I have found...
Karin Lagesen
@karinlag
:(
pvanheus
@pvanheus
btw I don't know if I see this problem more often because I have a particularly slow Galaxy server (I think it might be slow because the filesystem isn't great at handling metadata operations and conda does a lot of those)
Karin Lagesen
@karinlag
hmmm.... @ajkarloss didn't our other companion say something about our filesystem being slow?
could this be a potential source of this annoyance?
pvanheus
@pvanheus
this is something that perhaps should be raised in the Galaxy admins world / on their Gitter channel. of course, quantifying the problem first would be helpeful :)
Karin Lagesen
@karinlag
(thumbsup)
pvanheus
@pvanheus
hello IRIDA folks .... can this PR please be included in IRIDA 22.01? phac-nml/irida#1140
9 replies
Jeffrey Thiessen
@JeffreyThiessen
Irida Uploader version 0.7.1 is released. It contains a bugfix for the intermittent 10054 connection errors that some network-controllers/firewalls give when uploading data. Hopefully this problem is fixed for good now. https://github.com/phac-nml/irida-uploader/releases/tag/0.7.1 https://anaconda.org/bioconda/irida-uploader
Jeevan Karloss
@ajkarloss

Getting this error while submitting a pipeline. Where the problem could be? Is it in the communication between IRIDA and Galaxy or permission issues?

11 Jan 2022 09:26:30,737 DEBUG ca.corefacility.bioinformatics.irida.service.impl.AnalysisExecutionScheduledTaskImpl:117 - Preparing AnalysisSubmission [id=11, name=SISTR1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=NEW, analysisCleanedState=NOT_CLEANED]
11 Jan 2022 09:26:30,853 DEBUG ca.corefacility.bioinformatics.irida.service.analysis.execution.galaxy.AnalysisExecutionServiceGalaxyAsync:105 - Preparing submission for AnalysisSubmission [id=11, name=SISTR
1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=PREPARING, analysisCleanedState=NOT_CLEANED]
11 Jan 2022 09:26:30,868 ERROR ca.corefacility.bioinformatics.irida.service.analysis.execution.AnalysisExecutionServiceAspect:65 - Error occured for submission: AnalysisSubmission [id=11, name=SISTR__1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=PREPARING, analysisCleanedState=NOT_CLEANED] changing to state ERROR
ca.corefacility.bioinformatics.irida.exceptions.galaxy.WorkflowUploadException: Could not upload workflow from /workflows/1.1.1/irida_workflow_structure.ga
at ca.corefacility.bioinformatics.irida.pipeline.upload.galaxy.GalaxyWorkflowService.uploadGalaxyWorkflow(GalaxyWorkflowService.java:64) ~[classes/:?]

Jeevan Karloss
@ajkarloss
@everyone, Do anyone have IRIDA plugin .jar file for latest (0.2.4 or 0.2.5) Mentalist?
The .jar file available for is 0.1.9 from 2018
Jeevan Karloss
@ajkarloss
One more stuff: We have deleted some fastq files from our IRIDA installation. But the project and sample names are still there in IRIDA weeb. Since the data is not deleted physically from disk (by IRIDA), is there an easy way to re-associate the data to the project and sample instead of re-uploading?
Aaron Petkau
@apetkau
For the error with SISTR, can you try directly uploading the workflow into Galaxy (https://github.com/phac-nml/irida-plugin-sistr/blob/master/src/main/resources/workflows/1.1.1/irida_workflow_structure.ga )? I assume that Galaxy is still missing some tools.
There is no separate IRIDA plugin for MentaLiST, the built-in version is the latest
You may also encounter issues getting MentaLiST working in conda. If this is the case you could try installing the tools from this environment file: https://github.com/phac-nml/irida/blob/development/docker/virtual-machine/data/mentalist.env
For deleting fastq files, I'm not quite sure what you mean. You can remove samples from a project. But you are right it won't delete the fastq files. Do you mean how can you undelete already deleted fastq files?
Karin Lagesen
@karinlag
@apetkau yes
if we have "deleted" some files in IRIDA, can we somehow reconnect those files from the system inside so that they show up in the system again
Aaron Petkau
@apetkau
@karinlag ah, okay. There's no built-in button for undeleting a file. You can do it with custom SQL queries though (and we have done it before on our IRIDA instances). I can try to look through my notes to see what we did.
Karin Lagesen
@karinlag
that would be awesome!
thanks!
we should have that in our cheat sheet anyhow
users, amirite :D
(also known as I am the user in question and I'd like a reasonably painless way of undoing it)
Aaron Petkau
@apetkau
We're all users of something :laughing:
Karin Lagesen
@karinlag
yeah
Aaron Petkau
@apetkau
image.png

I'll write up some more detailed instructions someplace else (github maybe, or our docs) but in general restoring sequencing files deleted from a sample involves the table sample_sequencingobject which stores a single row which which sample is associated with which sequencing object:

select * from sample_sequencingobject;
+----+---------------------+-----------+---------------------+
| id | created_date        | sample_id | sequencingobject_id |
+----+---------------------+-----------+---------------------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |
+----+---------------------+-----------+---------------------+

The table sample_sequencingobject_AUD stores all modifications made to the sample_sequencingobject table:

select * from sample_sequencingobject_AUD;
+----+---------------------+-----------+---------------------+-----+---------+
| id | created_date        | sample_id | sequencingobject_id | REV | REVTYPE |
+----+---------------------+-----------+---------------------+-----+---------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |   6 |       0 |
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |  47 |       2 |
+----+---------------------+-----------+---------------------+-----+---------+

Here, REV is a unique identifier for every operation performed (e.g., create, update, delete). REVTYPE defines the specific operation performed (a REVTYPE of 2 means a deletion).

So if you look at:

select * from sample_sequencingobject_AUD where REVTYPE = 2;
+----+---------------------+-----------+---------------------+-----+---------+
| id | created_date        | sample_id | sequencingobject_id | REV | REVTYPE |
+----+---------------------+-----------+---------------------+-----+---------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |  47 |       2 |
+----+---------------------+-----------+---------------------+-----+---------+

You can see all delete operations on the sample_sequencingobject table. You can also see that the deleted information is still saved in this table (e.g., the specific sample_id=1 and sequencingobject_id=1 defining which sequencing object used to be linked with which sample).

To restore the link between the sample_id and sequencingobject_id you can re-insert this data into the sample_sequencingobject table like so:

start transaction;
insert into sample_sequencingobject (id,created_date,sample_id,sequencingobject_id) select id,created_date,sample_id,sequencingobject_id from sample_sequencingobject_AUD where REV in (47) and REVTYPE = 2;
commit;

This will re-insert the entry into the sample_sequencingobject table linking the sample and sequence data back up in IRIDA:

select * from sample_sequencingobject;
+----+---------------------+-----------+---------------------+
| id | created_date        | sample_id | sequencingobject_id |
+----+---------------------+-----------+---------------------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |
+----+---------------------+-----------+---------------------+

(see above screenshot)

However, realistically, depending on how much time you want to spend checking and validating your SQL (and learning the IRIDA database structure) it might just be easier to re-upload the sequence data anyways
Karin Lagesen
@karinlag
I was wondering about that
because I did have the impressin that reuploading it would just mean that we would have 2 copies of the data in the backend
Aaron Petkau
@apetkau
Yes, you would have 2 copies. But it's also less work than messing around with the internal tables/SQL in the database. Tradeoffs with each approach (we've re-uploaded a lot of data in our database too and have duplicate sequence files).