Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 04:04
    deepsidhu85 synchronize #1279
  • 03:55
    deepsidhu85 labeled #1279
  • 03:55
    deepsidhu85 opened #1279
  • 01:27
    joshsadam synchronize #1277
  • May 18 21:30
    JeffreyThiessen labeled #1278
  • May 18 21:30
    JeffreyThiessen opened #1278
  • May 18 18:34
    joshsadam opened #1277
  • May 18 14:53
    ericenns closed #1275
  • May 18 14:17
    joshsadam synchronize #1275
  • May 17 21:28
    joshsadam unassigned #1275
  • May 17 21:28
    joshsadam assigned #1275
  • May 17 20:03
    apetkau labeled #1276
  • May 17 20:03
    apetkau labeled #1276
  • May 17 20:03
    apetkau labeled #1276
  • May 17 20:03
    apetkau opened #1276
  • May 17 20:02
    apetkau closed #1274
  • May 17 20:01
    joshsadam synchronize #1275
  • May 17 19:37
    joshsadam synchronize #1275
  • May 17 19:34
    ericenns edited #1275
  • May 17 19:34
    ericenns closed #1260
Jeevan Karloss
@ajkarloss
But they are not
@pvanheus any tips here?
pvanheus
@pvanheus
@ajkarloss this is one of the reasons why I switched to using Singularity for tools. I had to go through multiple steps of manual uninstalling and reinstalling to get everything working.
one workaround though to make conda less painful is to install mamba in the Galaxy conda environment, and set conda_exec in the Galaxy config to point to the mamba executable (i.e. conda_exec: /tool_deps/_conda/bin/mamba in our config)
Jeevan Karloss
@ajkarloss
Ok. I am deleting them from the browser and reinstalling them now
@pvanheus Thanks
Aaron Petkau
@apetkau
Thanks for the advice @pvanheus
If a tool fails to install I tend to delete and re-install from the Galaxy UI: https://galaxyproject.org/admin/tools/add-tool-from-toolshed-tutorial/. But that's maybe not ideal if there are many tools with issues.
pvanheus
@pvanheus
@apetkau its really a Galaxy fault if things don't install right, but unfortunately I see it happening too often - especially with complex tools with lots of (conda) dependencies.
Karin Lagesen
@karinlag
are there ways to do this from the commandline?
(says the person who is more than slightly naive on this front)
pvanheus
@pvanheus
to delete and re-install? not that I have found...
Karin Lagesen
@karinlag
:(
pvanheus
@pvanheus
btw I don't know if I see this problem more often because I have a particularly slow Galaxy server (I think it might be slow because the filesystem isn't great at handling metadata operations and conda does a lot of those)
Karin Lagesen
@karinlag
hmmm.... @ajkarloss didn't our other companion say something about our filesystem being slow?
could this be a potential source of this annoyance?
pvanheus
@pvanheus
this is something that perhaps should be raised in the Galaxy admins world / on their Gitter channel. of course, quantifying the problem first would be helpeful :)
Karin Lagesen
@karinlag
(thumbsup)
pvanheus
@pvanheus
hello IRIDA folks .... can this PR please be included in IRIDA 22.01? phac-nml/irida#1140
9 replies
Jeffrey Thiessen
@JeffreyThiessen
Irida Uploader version 0.7.1 is released. It contains a bugfix for the intermittent 10054 connection errors that some network-controllers/firewalls give when uploading data. Hopefully this problem is fixed for good now. https://github.com/phac-nml/irida-uploader/releases/tag/0.7.1 https://anaconda.org/bioconda/irida-uploader
Jeevan Karloss
@ajkarloss

Getting this error while submitting a pipeline. Where the problem could be? Is it in the communication between IRIDA and Galaxy or permission issues?

11 Jan 2022 09:26:30,737 DEBUG ca.corefacility.bioinformatics.irida.service.impl.AnalysisExecutionScheduledTaskImpl:117 - Preparing AnalysisSubmission [id=11, name=SISTR1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=NEW, analysisCleanedState=NOT_CLEANED]
11 Jan 2022 09:26:30,853 DEBUG ca.corefacility.bioinformatics.irida.service.analysis.execution.galaxy.AnalysisExecutionServiceGalaxyAsync:105 - Preparing submission for AnalysisSubmission [id=11, name=SISTR
1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=PREPARING, analysisCleanedState=NOT_CLEANED]
11 Jan 2022 09:26:30,868 ERROR ca.corefacility.bioinformatics.irida.service.analysis.execution.AnalysisExecutionServiceAspect:65 - Error occured for submission: AnalysisSubmission [id=11, name=SISTR__1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=PREPARING, analysisCleanedState=NOT_CLEANED] changing to state ERROR
ca.corefacility.bioinformatics.irida.exceptions.galaxy.WorkflowUploadException: Could not upload workflow from /workflows/1.1.1/irida_workflow_structure.ga
at ca.corefacility.bioinformatics.irida.pipeline.upload.galaxy.GalaxyWorkflowService.uploadGalaxyWorkflow(GalaxyWorkflowService.java:64) ~[classes/:?]

Jeevan Karloss
@ajkarloss
@everyone, Do anyone have IRIDA plugin .jar file for latest (0.2.4 or 0.2.5) Mentalist?
The .jar file available for is 0.1.9 from 2018
Jeevan Karloss
@ajkarloss
One more stuff: We have deleted some fastq files from our IRIDA installation. But the project and sample names are still there in IRIDA weeb. Since the data is not deleted physically from disk (by IRIDA), is there an easy way to re-associate the data to the project and sample instead of re-uploading?
Aaron Petkau
@apetkau
For the error with SISTR, can you try directly uploading the workflow into Galaxy (https://github.com/phac-nml/irida-plugin-sistr/blob/master/src/main/resources/workflows/1.1.1/irida_workflow_structure.ga )? I assume that Galaxy is still missing some tools.
There is no separate IRIDA plugin for MentaLiST, the built-in version is the latest
You may also encounter issues getting MentaLiST working in conda. If this is the case you could try installing the tools from this environment file: https://github.com/phac-nml/irida/blob/development/docker/virtual-machine/data/mentalist.env
For deleting fastq files, I'm not quite sure what you mean. You can remove samples from a project. But you are right it won't delete the fastq files. Do you mean how can you undelete already deleted fastq files?
Karin Lagesen
@karinlag
@apetkau yes
if we have "deleted" some files in IRIDA, can we somehow reconnect those files from the system inside so that they show up in the system again
Aaron Petkau
@apetkau
@karinlag ah, okay. There's no built-in button for undeleting a file. You can do it with custom SQL queries though (and we have done it before on our IRIDA instances). I can try to look through my notes to see what we did.
Karin Lagesen
@karinlag
that would be awesome!
thanks!
we should have that in our cheat sheet anyhow
users, amirite :D
(also known as I am the user in question and I'd like a reasonably painless way of undoing it)
Aaron Petkau
@apetkau
We're all users of something :laughing:
Karin Lagesen
@karinlag
yeah
Aaron Petkau
@apetkau
image.png

I'll write up some more detailed instructions someplace else (github maybe, or our docs) but in general restoring sequencing files deleted from a sample involves the table sample_sequencingobject which stores a single row which which sample is associated with which sequencing object:

select * from sample_sequencingobject;
+----+---------------------+-----------+---------------------+
| id | created_date        | sample_id | sequencingobject_id |
+----+---------------------+-----------+---------------------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |
+----+---------------------+-----------+---------------------+

The table sample_sequencingobject_AUD stores all modifications made to the sample_sequencingobject table:

select * from sample_sequencingobject_AUD;
+----+---------------------+-----------+---------------------+-----+---------+
| id | created_date        | sample_id | sequencingobject_id | REV | REVTYPE |
+----+---------------------+-----------+---------------------+-----+---------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |   6 |       0 |
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |  47 |       2 |
+----+---------------------+-----------+---------------------+-----+---------+

Here, REV is a unique identifier for every operation performed (e.g., create, update, delete). REVTYPE defines the specific operation performed (a REVTYPE of 2 means a deletion).

So if you look at:

select * from sample_sequencingobject_AUD where REVTYPE = 2;
+----+---------------------+-----------+---------------------+-----+---------+
| id | created_date        | sample_id | sequencingobject_id | REV | REVTYPE |
+----+---------------------+-----------+---------------------+-----+---------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |  47 |       2 |
+----+---------------------+-----------+---------------------+-----+---------+

You can see all delete operations on the sample_sequencingobject table. You can also see that the deleted information is still saved in this table (e.g., the specific sample_id=1 and sequencingobject_id=1 defining which sequencing object used to be linked with which sample).

To restore the link between the sample_id and sequencingobject_id you can re-insert this data into the sample_sequencingobject table like so:

start transaction;
insert into sample_sequencingobject (id,created_date,sample_id,sequencingobject_id) select id,created_date,sample_id,sequencingobject_id from sample_sequencingobject_AUD where REV in (47) and REVTYPE = 2;
commit;

This will re-insert the entry into the sample_sequencingobject table linking the sample and sequence data back up in IRIDA:

select * from sample_sequencingobject;
+----+---------------------+-----------+---------------------+
| id | created_date        | sample_id | sequencingobject_id |
+----+---------------------+-----------+---------------------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |
+----+---------------------+-----------+---------------------+

(see above screenshot)

However, realistically, depending on how much time you want to spend checking and validating your SQL (and learning the IRIDA database structure) it might just be easier to re-upload the sequence data anyways
Karin Lagesen
@karinlag
I was wondering about that
because I did have the impressin that reuploading it would just mean that we would have 2 copies of the data in the backend
Aaron Petkau
@apetkau
Yes, you would have 2 copies. But it's also less work than messing around with the internal tables/SQL in the database. Tradeoffs with each approach (we've re-uploaded a lot of data in our database too and have duplicate sequence files).
Aaron Petkau
@apetkau

The actual code that gets called when you remove sequence data from a sample is:

https://github.com/phac-nml/irida/blob/development/src/main/java/ca/corefacility/bioinformatics/irida/ria/web/samples/SamplesController.java#L426-L445

Specifically, here you can see that what it does is removes the link between a sample and a sequencing object (that is a row from the sample_sequencingobject table). So restoring the entry in this table restores the sample/sequence data link.

Aaron Petkau
@apetkau
image.png
The different REVTYPE values in case you wanted to see what they all represent (from https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#envers)
Jeevan Karloss
@ajkarloss
@apetkau Thanks a lot.
Jeevan Karloss
@ajkarloss

@apetkau I am getting this Error

AnalysisSubmission [id=11, name=SISTR__1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=PREPARING, analysisCleanedState=NOT_CLEANED] changing to state ERROR
ca.corefacility.bioinformatics.irida.exceptions.galaxy.WorkflowUploadException: Could not upload workflow from /workflows/1.1.1/irida_workflow_structure.ga
at ca.corefacility.bioinformatics.irida.pipeline.upload.galaxy.GalaxyWorkflowService.uploadGalaxyWorkflow(GalaxyWorkflowService.java:64) ~[classes/:?]

No New history is being created

Is it something to do with file sharing or?

Aaron Petkau
@apetkau
Can you try uploading the SISTR workflow to Galaxy to see if all the tools are installed?