Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 04:04
    deepsidhu85 synchronize #1279
  • 03:55
    deepsidhu85 labeled #1279
  • 03:55
    deepsidhu85 opened #1279
  • 01:27
    joshsadam synchronize #1277
  • May 18 21:30
    JeffreyThiessen labeled #1278
  • May 18 21:30
    JeffreyThiessen opened #1278
  • May 18 18:34
    joshsadam opened #1277
  • May 18 14:53
    ericenns closed #1275
  • May 18 14:17
    joshsadam synchronize #1275
  • May 17 21:28
    joshsadam unassigned #1275
  • May 17 21:28
    joshsadam assigned #1275
  • May 17 20:03
    apetkau labeled #1276
  • May 17 20:03
    apetkau labeled #1276
  • May 17 20:03
    apetkau labeled #1276
  • May 17 20:03
    apetkau opened #1276
  • May 17 20:02
    apetkau closed #1274
  • May 17 20:01
    joshsadam synchronize #1275
  • May 17 19:37
    joshsadam synchronize #1275
  • May 17 19:34
    ericenns edited #1275
  • May 17 19:34
    ericenns closed #1260
Karin Lagesen
@karinlag
that would be awesome!
thanks!
we should have that in our cheat sheet anyhow
users, amirite :D
(also known as I am the user in question and I'd like a reasonably painless way of undoing it)
Aaron Petkau
@apetkau
We're all users of something :laughing:
Karin Lagesen
@karinlag
yeah
Aaron Petkau
@apetkau
image.png

I'll write up some more detailed instructions someplace else (github maybe, or our docs) but in general restoring sequencing files deleted from a sample involves the table sample_sequencingobject which stores a single row which which sample is associated with which sequencing object:

select * from sample_sequencingobject;
+----+---------------------+-----------+---------------------+
| id | created_date        | sample_id | sequencingobject_id |
+----+---------------------+-----------+---------------------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |
+----+---------------------+-----------+---------------------+

The table sample_sequencingobject_AUD stores all modifications made to the sample_sequencingobject table:

select * from sample_sequencingobject_AUD;
+----+---------------------+-----------+---------------------+-----+---------+
| id | created_date        | sample_id | sequencingobject_id | REV | REVTYPE |
+----+---------------------+-----------+---------------------+-----+---------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |   6 |       0 |
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |  47 |       2 |
+----+---------------------+-----------+---------------------+-----+---------+

Here, REV is a unique identifier for every operation performed (e.g., create, update, delete). REVTYPE defines the specific operation performed (a REVTYPE of 2 means a deletion).

So if you look at:

select * from sample_sequencingobject_AUD where REVTYPE = 2;
+----+---------------------+-----------+---------------------+-----+---------+
| id | created_date        | sample_id | sequencingobject_id | REV | REVTYPE |
+----+---------------------+-----------+---------------------+-----+---------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |  47 |       2 |
+----+---------------------+-----------+---------------------+-----+---------+

You can see all delete operations on the sample_sequencingobject table. You can also see that the deleted information is still saved in this table (e.g., the specific sample_id=1 and sequencingobject_id=1 defining which sequencing object used to be linked with which sample).

To restore the link between the sample_id and sequencingobject_id you can re-insert this data into the sample_sequencingobject table like so:

start transaction;
insert into sample_sequencingobject (id,created_date,sample_id,sequencingobject_id) select id,created_date,sample_id,sequencingobject_id from sample_sequencingobject_AUD where REV in (47) and REVTYPE = 2;
commit;

This will re-insert the entry into the sample_sequencingobject table linking the sample and sequence data back up in IRIDA:

select * from sample_sequencingobject;
+----+---------------------+-----------+---------------------+
| id | created_date        | sample_id | sequencingobject_id |
+----+---------------------+-----------+---------------------+
|  1 | 2022-01-11 12:38:16 |         1 |                   1 |
+----+---------------------+-----------+---------------------+

(see above screenshot)

However, realistically, depending on how much time you want to spend checking and validating your SQL (and learning the IRIDA database structure) it might just be easier to re-upload the sequence data anyways
Karin Lagesen
@karinlag
I was wondering about that
because I did have the impressin that reuploading it would just mean that we would have 2 copies of the data in the backend
Aaron Petkau
@apetkau
Yes, you would have 2 copies. But it's also less work than messing around with the internal tables/SQL in the database. Tradeoffs with each approach (we've re-uploaded a lot of data in our database too and have duplicate sequence files).
Aaron Petkau
@apetkau

The actual code that gets called when you remove sequence data from a sample is:

https://github.com/phac-nml/irida/blob/development/src/main/java/ca/corefacility/bioinformatics/irida/ria/web/samples/SamplesController.java#L426-L445

Specifically, here you can see that what it does is removes the link between a sample and a sequencing object (that is a row from the sample_sequencingobject table). So restoring the entry in this table restores the sample/sequence data link.

Aaron Petkau
@apetkau
image.png
The different REVTYPE values in case you wanted to see what they all represent (from https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#envers)
Jeevan Karloss
@ajkarloss
@apetkau Thanks a lot.
Jeevan Karloss
@ajkarloss

@apetkau I am getting this Error

AnalysisSubmission [id=11, name=SISTR__1-11-2022_Test_1, submitter=admin, workflowId=b21ea62c-7916-4ca6-96ba-90c20177b70f, analysisState=PREPARING, analysisCleanedState=NOT_CLEANED] changing to state ERROR
ca.corefacility.bioinformatics.irida.exceptions.galaxy.WorkflowUploadException: Could not upload workflow from /workflows/1.1.1/irida_workflow_structure.ga
at ca.corefacility.bioinformatics.irida.pipeline.upload.galaxy.GalaxyWorkflowService.uploadGalaxyWorkflow(GalaxyWorkflowService.java:64) ~[classes/:?]

No New history is being created

Is it something to do with file sharing or?

Aaron Petkau
@apetkau
Can you try uploading the SISTR workflow to Galaxy to see if all the tools are installed?
Jeevan Karloss
@ajkarloss
Yes, that works
Aaron Petkau
@apetkau
Are all other pipelines working, just not SISTR?
Is there anything in the Galaxy log files at around the same time as trying to run SISTR?
Eric Enns
@ericenns
@ajkarloss Is the IRIDA data filesystem accessible on the galaxy server? See point 2 here https://phac-nml.github.io/irida-documentation/administrator/galaxy/existing-galaxy/
Jeevan Karloss
@ajkarloss
@apetkau All the workflows are working fine in galaxy :)
@ericenns I am able to access the irida data from galaxy
Aaron Petkau
@apetkau
@ajkarloss all the workflows as in you are able to run SISTR now? Or is SISTR not working still?
Jeevan Karloss
@ajkarloss
SISTR: Is it possible to update the cgMLST330 to ~3000 loci?
Aaron Petkau
@apetkau
No, SISTR was designed with 330 loci.
You could look into other Salmonella MLST schemes though (e.g., from enterobase https://enterobase.warwick.ac.uk/)
Jeevan Karloss
@ajkarloss
@apetkau Thanks
Zipho Mashologu
@zipho

Hi, I am having this issue whenever trying to run an analysis

ca.corefacility.bioinformatics.irida.exceptions.galaxy.WorkflowUploadException: Could not upload workflow from /workflows/0.1.5/irida_workflow_structure.ga

Galaxy logs throw the uwsgi issue, could they be related.
[uwsgi-http key: localhost:9090 client_addr: 127.0.0.1 client_port: 50636] hr_instance_read(): Connection reset by peer [plugins/http/http.c line 647]

any pointers will be greatly appreciated.
Jeevan Karloss
@ajkarloss

@zipho, seems like irida/tomcat does not have permission to upload the file to galaxy.

Can galaxy user from galaxy VM able to login to irida VM and vice-versa?

Aaron Petkau
@apetkau
@zipho is this only for a particular analysis pipeline, or for all analysis pipelines?
I'm not sure if the uwsgi issue is related. I guess peer in this case could be IRIDA (if that message is from Galaxy) so could be produced during the error in uploading the workflow. But I'm not sure.
@ajkarloss do you mean does IRIDA and Galaxy share user accounts? No, they do not. A user account in Galaxy is different from a user account in IRIDA.
Or do you mean the user running the Galaxy and IRIDA software? This depends on how you have set things up, but in general no they are likely separate accounts.
Jeevan Karloss
@ajkarloss
@apetkau No, I dont mean " IRIDA and Galaxy share user accounts".
Jeevan Karloss
@ajkarloss

SNVPhy_Pipeline throws this exception when we use many samples.

Exception: Header does not correspond to PHYLIP header.

Any tips?

Aaron Petkau
@apetkau

It's likely that some genomes are either very low coverage or too different from the rest and all SNV/SNP data is removed from the phylip alignment file. You can check the mapping quality file (https://snvphyl.readthedocs.io/en/latest/user/output/#mapping-quality) for information on which genomes have low coverage. You can check the core positions file to find the percent of the reference genome considered for data analysis.

If you find any genomes with too low of coverage or too distantly related you can remove them from analysis and re-run SNVPhyl

You may also want to disable SNV density filtering too if it is enabled (https://phac-nml.github.io/irida-documentation/user/tutorials/snvphyl/#required-parameters-density-filtering). This can cause issues if some genomes are very distant from others (leading to many SNVs/SNPs between them)
Jeevan Karloss
@ajkarloss
@apetkau Thanks. I will test and try that :)
Eric Enns
@ericenns
January release of IRIDA is now available. https://github.com/phac-nml/irida/releases/tag/22.01
Jeevan Karloss
@ajkarloss
@ericenns Thanks s:)
Screenshot 2022-01-30 at 21.38.55.png
I am getting those error while loading the IRIDA plugins and I dont see the added JAR files in on the web. Any tips?
Aaron Petkau
@apetkau
@ajkarloss I can see multiple IRIDA war files listed in the log irida-21.09.2.war and irida.war. Is it possible you are deploying multiple instances of IRIDA and one of them is an old version?
You likely want to figure out which one you want to deploy and rename it to irida.war (so IRIDA is available at http://address/irida).
Jeevan Karloss
@ajkarloss
Thanks @apetkau