Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 21:28
    joshsadam unassigned #1275
  • 21:28
    joshsadam assigned #1275
  • 20:03
    apetkau labeled #1276
  • 20:03
    apetkau labeled #1276
  • 20:03
    apetkau labeled #1276
  • 20:03
    apetkau opened #1276
  • 20:02
    apetkau closed #1274
  • 20:01
    joshsadam synchronize #1275
  • 19:37
    joshsadam synchronize #1275
  • 19:34
    ericenns edited #1275
  • 19:34
    ericenns closed #1260
  • 16:45
    ksierks review_requested #1260
  • 16:45
    ksierks review_requested #1260
  • 16:44
    ksierks synchronize #1260
  • 16:11
    joshsadam assigned #1275
  • 16:11
    joshsadam review_requested #1275
  • 16:11
    joshsadam review_request_removed #1275
  • 16:11
    joshsadam review_requested #1275
  • 16:11
    joshsadam review_requested #1275
  • 16:11
    joshsadam opened #1275
pvanheus
@pvanheus
btw I don't know if I see this problem more often because I have a particularly slow Galaxy server (I think it might be slow because the filesystem isn't great at handling metadata operations and conda does a lot of those)
Karin Lagesen
@karinlag
hmmm.... @ajkarloss didn't our other companion say something about our filesystem being slow?
could this be a potential source of this annoyance?
pvanheus
@pvanheus
this is something that perhaps should be raised in the Galaxy admins world / on their Gitter channel. of course, quantifying the problem first would be helpeful :)
Karin Lagesen
@karinlag
(thumbsup)
pvanheus
@pvanheus
hello IRIDA folks .... can this PR please be included in IRIDA 22.01? phac-nml/irida#1140
9 replies
Jeffrey Thiessen
@JeffreyThiessen
Irida Uploader version 0.7.1 is released. It contains a bugfix for the intermittent 10054 connection errors that some network-controllers/firewalls give when uploading data. Hopefully this problem is fixed for good now. https://github.com/phac-nml/irida-uploader/releases/tag/0.7.1 https://anaconda.org/bioconda/irida-uploader
Jeevan Karloss
@ajkarloss

Getting this error while submitting a pipeline. Where the problem could be? Is it in the communication between IRIDA and Galaxy or permission issues?

11 Jan 2022 09:26:30,737 DEBUG ca.corefacility.bioinformatics.irida.service.impl.AnalysisExecutionScheduledTaskImpl:117 - Preparing AnalysisSubmission [id=11, name=SISTR1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=NEW, analysisCleanedState=NOT_CLEANED]
11 Jan 2022 09:26:30,853 DEBUG ca.corefacility.bioinformatics.irida.service.analysis.execution.galaxy.AnalysisExecutionServiceGalaxyAsync:105 - Preparing submission for AnalysisSubmission [id=11, name=SISTR
1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=PREPARING, analysisCleanedState=NOT_CLEANED]
11 Jan 2022 09:26:30,868 ERROR ca.corefacility.bioinformatics.irida.service.analysis.execution.AnalysisExecutionServiceAspect:65 - Error occured for submission: AnalysisSubmission [id=11, name=SISTR__1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=PREPARING, analysisCleanedState=NOT_CLEANED] changing to state ERROR
ca.corefacility.bioinformatics.irida.exceptions.galaxy.WorkflowUploadException: Could not upload workflow from /workflows/1.1.1/irida_workflow_structure.ga
at ca.corefacility.bioinformatics.irida.pipeline.upload.galaxy.GalaxyWorkflowService.uploadGalaxyWorkflow(GalaxyWorkflowService.java:64) ~[classes/:?]

Jeevan Karloss
@ajkarloss
@everyone, Do anyone have IRIDA plugin .jar file for latest (0.2.4 or 0.2.5) Mentalist?
The .jar file available for is 0.1.9 from 2018
Jeevan Karloss
@ajkarloss
One more stuff: We have deleted some fastq files from our IRIDA installation. But the project and sample names are still there in IRIDA weeb. Since the data is not deleted physically from disk (by IRIDA), is there an easy way to re-associate the data to the project and sample instead of re-uploading?
Aaron Petkau
@apetkau
For the error with SISTR, can you try directly uploading the workflow into Galaxy (https://github.com/phac-nml/irida-plugin-sistr/blob/master/src/main/resources/workflows/1.1.1/irida_workflow_structure.ga )? I assume that Galaxy is still missing some tools.
There is no separate IRIDA plugin for MentaLiST, the built-in version is the latest
You may also encounter issues getting MentaLiST working in conda. If this is the case you could try installing the tools from this environment file: https://github.com/phac-nml/irida/blob/development/docker/virtual-machine/data/mentalist.env
For deleting fastq files, I'm not quite sure what you mean. You can remove samples from a project. But you are right it won't delete the fastq files. Do you mean how can you undelete already deleted fastq files?
Karin Lagesen
@karinlag
@apetkau yes
if we have "deleted" some files in IRIDA, can we somehow reconnect those files from the system inside so that they show up in the system again
Aaron Petkau
@apetkau
@karinlag ah, okay. There's no built-in button for undeleting a file. You can do it with custom SQL queries though (and we have done it before on our IRIDA instances). I can try to look through my notes to see what we did.
Karin Lagesen
@karinlag
that would be awesome!
thanks!
we should have that in our cheat sheet anyhow
users, amirite :D
(also known as I am the user in question and I'd like a reasonably painless way of undoing it)
Aaron Petkau
@apetkau
We're all users of something :laughing:
Karin Lagesen
@karinlag
yeah
Aaron Petkau
@apetkau
image.png

I'll write up some more detailed instructions someplace else (github maybe, or our docs) but in general restoring sequencing files deleted from a sample involves the table sample_sequencingobject which stores a single row which which sample is associated with which sequencing object:

select * from sample_sequencingobject;
+----+---------------------+-----------+---------------------+
| id | created_date        | sample_id | sequencingobject_id |
+----+---------------------+-----------+---------------------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |
+----+---------------------+-----------+---------------------+

The table sample_sequencingobject_AUD stores all modifications made to the sample_sequencingobject table:

select * from sample_sequencingobject_AUD;
+----+---------------------+-----------+---------------------+-----+---------+
| id | created_date        | sample_id | sequencingobject_id | REV | REVTYPE |
+----+---------------------+-----------+---------------------+-----+---------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |   6 |       0 |
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |  47 |       2 |
+----+---------------------+-----------+---------------------+-----+---------+

Here, REV is a unique identifier for every operation performed (e.g., create, update, delete). REVTYPE defines the specific operation performed (a REVTYPE of 2 means a deletion).

So if you look at:

select * from sample_sequencingobject_AUD where REVTYPE = 2;
+----+---------------------+-----------+---------------------+-----+---------+
| id | created_date        | sample_id | sequencingobject_id | REV | REVTYPE |
+----+---------------------+-----------+---------------------+-----+---------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |  47 |       2 |
+----+---------------------+-----------+---------------------+-----+---------+

You can see all delete operations on the sample_sequencingobject table. You can also see that the deleted information is still saved in this table (e.g., the specific sample_id=1 and sequencingobject_id=1 defining which sequencing object used to be linked with which sample).

To restore the link between the sample_id and sequencingobject_id you can re-insert this data into the sample_sequencingobject table like so:

start transaction;
insert into sample_sequencingobject (id,created_date,sample_id,sequencingobject_id) select id,created_date,sample_id,sequencingobject_id from sample_sequencingobject_AUD where REV in (47) and REVTYPE = 2;
commit;

This will re-insert the entry into the sample_sequencingobject table linking the sample and sequence data back up in IRIDA:

select * from sample_sequencingobject;
+----+---------------------+-----------+---------------------+
| id | created_date        | sample_id | sequencingobject_id |
+----+---------------------+-----------+---------------------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |
+----+---------------------+-----------+---------------------+

(see above screenshot)

However, realistically, depending on how much time you want to spend checking and validating your SQL (and learning the IRIDA database structure) it might just be easier to re-upload the sequence data anyways
Karin Lagesen
@karinlag
I was wondering about that
because I did have the impressin that reuploading it would just mean that we would have 2 copies of the data in the backend
Aaron Petkau
@apetkau
Yes, you would have 2 copies. But it's also less work than messing around with the internal tables/SQL in the database. Tradeoffs with each approach (we've re-uploaded a lot of data in our database too and have duplicate sequence files).
Aaron Petkau
@apetkau

The actual code that gets called when you remove sequence data from a sample is:

https://github.com/phac-nml/irida/blob/development/src/main/java/ca/corefacility/bioinformatics/irida/ria/web/samples/SamplesController.java#L426-L445

Specifically, here you can see that what it does is removes the link between a sample and a sequencing object (that is a row from the sample_sequencingobject table). So restoring the entry in this table restores the sample/sequence data link.

Aaron Petkau
@apetkau
image.png
The different REVTYPE values in case you wanted to see what they all represent (from https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#envers)
Jeevan Karloss
@ajkarloss
@apetkau Thanks a lot.
Jeevan Karloss
@ajkarloss

@apetkau I am getting this Error

AnalysisSubmission [id=11, name=SISTR__1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=PREPARING, analysisCleanedState=NOT_CLEANED] changing to state ERROR
ca.corefacility.bioinformatics.irida.exceptions.galaxy.WorkflowUploadException: Could not upload workflow from /workflows/1.1.1/irida_workflow_structure.ga
at ca.corefacility.bioinformatics.irida.pipeline.upload.galaxy.GalaxyWorkflowService.uploadGalaxyWorkflow(GalaxyWorkflowService.java:64) ~[classes/:?]

No New history is being created

Is it something to do with file sharing or?

Aaron Petkau
@apetkau
Can you try uploading the SISTR workflow to Galaxy to see if all the tools are installed?
Jeevan Karloss
@ajkarloss
Yes, that works
Aaron Petkau
@apetkau
Are all other pipelines working, just not SISTR?
Is there anything in the Galaxy log files at around the same time as trying to run SISTR?
Eric Enns
@ericenns
@ajkarloss Is the IRIDA data filesystem accessible on the galaxy server? See point 2 here https://phac-nml.github.io/irida-documentation/administrator/galaxy/existing-galaxy/
Jeevan Karloss
@ajkarloss
@apetkau All the workflows are working fine in galaxy :)
@ericenns I am able to access the irida data from galaxy
Aaron Petkau
@apetkau
@ajkarloss all the workflows as in you are able to run SISTR now? Or is SISTR not working still?
Jeevan Karloss
@ajkarloss
SISTR: Is it possible to update the cgMLST330 to ~3000 loci?
Aaron Petkau
@apetkau
No, SISTR was designed with 330 loci.
You could look into other Salmonella MLST schemes though (e.g., from enterobase https://enterobase.warwick.ac.uk/)
Jeevan Karloss
@ajkarloss
@apetkau Thanks
Zipho Mashologu
@zipho

Hi, I am having this issue whenever trying to run an analysis

ca.corefacility.bioinformatics.irida.exceptions.galaxy.WorkflowUploadException: Could not upload workflow from /workflows/0.1.5/irida_workflow_structure.ga

Galaxy logs throw the uwsgi issue, could they be related.
[uwsgi-http key: localhost:9090 client_addr: 127.0.0.1 client_port: 50636] hr_instance_read(): Connection reset by peer [plugins/http/http.c line 647]