Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 07 2019 15:05
    colindaven starred ewels/MegaQC
  • Dec 10 2018 16:46
    vzg100 starred ewels/MegaQC
  • Dec 10 2018 16:39
    sridhar0605 starred ewels/MegaQC
  • Nov 21 2018 11:15
    ewels commented #39
  • Nov 20 2018 23:23
    lutsik commented #39
  • Nov 20 2018 23:13
    ewels commented #39
  • Nov 20 2018 23:10
    lutsik commented #39
  • Nov 20 2018 22:59
    ewels commented #39
  • Nov 20 2018 22:27
    lutsik commented #39
  • Nov 08 2018 13:08
    ebete starred ewels/MegaQC
  • Oct 17 2018 13:26
    tsnowlan commented #70
  • Oct 11 2018 14:09
    ewels commented #67
  • Oct 11 2018 12:06
    shafqatb commented #67
  • Oct 10 2018 15:14
    ewels labeled #70
  • Oct 10 2018 15:14
    ewels labeled #70
  • Oct 10 2018 15:14
    ewels opened #70
  • Oct 10 2018 11:23
    ewels commented #69
  • Oct 10 2018 11:09
    tsnowlan commented #69
  • Oct 10 2018 10:06
    ewels commented #68
  • Oct 10 2018 10:05

    ewels on FLASK_DEBUG

    (compare)

Phil Ewels
@ewels
I was hoping that I'd mapped that to something that could be filtered for in MegaQC
but it's entirely possible that I never did
Albert Vilella
@avilella
Gotcha
Another question: if I create a .csv or .tsv file with a bunch of metrics and name it with the suffix that MultiQC picks up, will they be in the .json and therefore in MegaQC?
I have done this in the past by adding _mqc.tsv to the file name
Phil Ewels
@ewels
yes, they should be
one feature that I would like to add and haven't is the option of having content in such files that is "silent" - that is, that never makes it in to MultiQC reports, but that would then be fed in to MegaQC.
Our use case for this would be to pass metadata from our LIMS through MultiQC and on to MegaQC, without ever printing it in the reports which are sent to users
In this way, MultiQC becomes a kind of normalisation filter for multiple infrastructure back-ends
(by accident)
Albert Vilella
@avilella
The custom _mqc.tsv seem to be loaded into MegaQC
It does give me though this message: Finished writing plot data (0 cfg, 164 data points) for report 22
which makes me think I need to add something to instruct how I want each field plotted. Any ideas how @ewels ?
Phil Ewels
@ewels
I'm not sure where that's being printed sorry
Albert Vilella
@avilella
Let's say I have a custom _mqc.tsv like the one below: foo name 1 sample1 2 sample2 3 sample3 4 sample4 5 sample5 6 sample6 7 sample7 8 sample8 9 sample9 10 sample10
foo     name
1       sample1
2       sample2
3       sample3
4       sample4
5       sample5
6       sample6
7       sample7
8       sample8
9       sample9
10      sample10
Do I need to add a config section somewhere so that foo becomes available to the list of metrics in MegaQC?
Ok, forgot what I said, it turns out the _mqc.tsv files are loaded into the .html report but are not found in the .json
So I will have to find a way to craft the .json file from the .tsv directly
Phil Ewels
@ewels
I'm a bit lost - the table renders in the MultiQC report correctly, but then isn't in the JSON?
I'm quite surprised if that's the case
I just doublechecked with some custom content files at my end and it worked ifne
{
    "report_general_stats_headers": [], 
    "config_version": "1.8.dev0 (ef579dc)", 
    "report_general_stats_data": [], 
    "config_creation_date": "2019-03-07, 17:42", 
    "config_title": null, 
    "config_report_comment": null, 
    "config_short_version": "1.8.dev0", 
    "config_intro_text": null, 
    "report_saved_raw_data": {
        "multiqc_my_section_with_a_table": {
            "samp_1": {
                "col1": 12.523, 
                "col2": 34.0
            }, 
            "samp_2": {
                "col1": 56.947, 
                "col2": 78.0
            }
        }
    }, 
    "report_data_sources": {}, 
    "report_multiqc_command": "/Users/ewels/miniconda2/envs/work/bin/multiqc -f table_headers_mqc.txt", 
    "config_analysis_dir": [
        "table_headers_mqc.txt"
    ], 
    "config_report_header_info": null, 
    "config_analysis_dir_abs": [
        "/Users/ewels/GitHub/MultiQC_TestData/data/custom_content/embedded_config/table_headers_mqc.txt"
    ], 
    "config_subtitle": null, 
    "report_plot_data": {}, 
    "config_script_path": "/Users/ewels/GitHub/MultiQC/multiqc/utils", 
    "config_git_hash": "ef579dcdcdb12a56ea516b2354472cb5f640353c"
}
Albert Vilella
@avilella
So in my case it produces a json with no data
avilella@avilellaM710t:~/multiqc/test$ find $PWD/ -name "*.json" | xargs cat
{
    "report_general_stats_headers": [], 
    "config_version": "1.5dev", 
    "report_general_stats_data": [], 
    "config_creation_date": "2019-03-07, 16:48", 
    "config_title": null, 
    "config_report_comment": null, 
    "config_short_version": "1.5dev", 
    "config_intro_text": null, 
    "report_saved_raw_data": {}, 
    "report_data_sources": {}, 
    "report_multiqc_command": "/usr/local/bin/multiqc --force --filename /home/avilella/multiqc/test/report__multiqc /home/avilella/multiqc/test", 
    "config_analysis_dir": [
        "/home/avilella/multiqc/test"
    ], 
    "config_report_header_info": null, 
    "config_analysis_dir_abs": [
        "/home/avilella/multiqc/test"
    ], 
    "config_subtitle": null, 
    "report_plot_data": {}, 
    "config_script_path": "/usr/local/lib/python2.7/dist-packages/multiqc/utils", 
    "config_git_hash": null
}
avilella@avilellaM710t:~/multiqc/test$ cat *.txt
# plot_type: 'table'
# section_name: 'My section with a table'
# description: 'a custom text introduction (a few sentences) for this section'
# pconfig:
#     namespace: 'Cust Data'
# headers:
#     col1:
#         title: 'My Column'
#         description: 'This is a longer hover text for my column'
#         format: '{:,.0f}'
#     col2:
#         title: 'Second Column'
#         description: 'Hover description text'
Sample            col1            col2
samp_1            12.523          34
samp_2            56.947          78
avilella@avilellaM710t:~/multiqc/test$ #/usr/local/bin/multiqc --force --filename /home/avilella/multiqc/test/report__multiqc /home/avilella/multiqc/test
Phil Ewels
@ewels
Hmm, MultiQC v1.5dev
that's pretty old
we're on 1.8dev now
My guess is that you need to update MultiQC for this to work with custom content then
Albert Vilella
@avilella
I updated to 1.7 and it worked, thanks.
Phil Ewels
@ewels
Ok great!
Sehrish Kanwal
@skanwal

Hi Phil,
I am testing the megaqc upload function (locally on my laptop) to get data into megaQC server.
I am getting Error - This function requires MultiQC to be installed..
Following this issue on github https://github.com/ewels/MegaQC/issues/66, I have multiqc installed on my laptop and I am running the upload command from the same environment where I used multiqc to generate the report/json file.
For context, I installed multiqc via conda. The multiqc conda environment is also active in my session. Trying from multiqc.utils import config as multiqc_config on python interpreter also works fine..

Any idea what could be the underlying issue?

Phil Ewels
@ewels
Hi Sehrish - that sounds very odd!
Are you super-sure that everything is definitely in the same environment? By checking which megaqc and which multiqc..?
Ah, and it’s definitely a recent version of MultiQC?
The code that triggers that message isn’t doing anything particularly special: https://github.com/ewels/MegaQC/blob/master/megaqc/commands.py#L173
Sehrish Kanwal
@skanwal

Ah, that’s going to be interesting.

which multiqc > /Users/kanwals/miniconda3/envs/multiqc/bin/multiqc
which megaqc > /Users/Kanwals/miniconda3/bin//megaqc

As you can tell, I installed both tools in separate conda environment. Aslo, used pip for megaqc installation.
Should I try installing megaqc in multiqc environment.?
Multiqc version is 1.7, so should definitely be not an issue.

Phil Ewels
@ewels
Yes they need to be installed in the same environment
Otherwise they are using different python installations, so they can’t see each other
Pip installs within a conda environment should be fine
which pip should show it in the conda env.
Sehrish Kanwal
@skanwal

Phil, tried reinstalling megaQC in the multiQC conda environment using pip. The installation went okay.
Both the tools are now installed in the same environment.
which megaqc > /Users/kanwals/miniconda3/envs/multiqc/bin/megaqc
which multiqc > /Users/kanwals/miniconda3/envs/multiqc/bin/multiqc

For testing, ran multiqc on a sample, and the report was successfully posted to megaqc server.
But upload is again throwing the same issue:

megaqc upload multiqc_data/multiqc_data.json
This is MegaQC v0.1.dev1

Env: test
 * Database type: sqlite
 * Database path: sqlite:////Users/kanwals/miniconda3/lib/python3.6/site-packages/test.db
Error - This function requires MultiQC to be installed
Phil Ewels
@ewels
Strange. Could you make an issue about this on the GitHub repo please?
I have no ideas as to how or why this could happen now though
It’s nothing special about MegaQC or MultiQC here - it’s just a simple python import.
Sehrish Kanwal
@skanwal
Thanks, Phil.
Sure, I'll open an issue on GitHub.
Michael Milton
@TMiguelT
I'd appreciate feedback from anyone on my Dash integration PR: ewels/MegaQC#81
Michael Milton
@TMiguelT
@ewels if you can get back to me about the dash PR and also the control chart thresholds, that would be great
I have an upcoming meeting at my work where I'll be talking about MegaQC, and if I can implement and then show off some of these features, there's a much better chance I'll be given time to work on MegaQC