Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Apr 09 00:16

    BenLangmead on master

    embed parsed_md (compare)

  • Apr 08 21:26

    BenLangmead on master

    make more portable (compare)

  • Apr 08 21:18

    BenLangmead on master

    some attempts to make this scri… (compare)

  • Dec 03 2018 15:25
    gianmaz edited #88
  • Dec 03 2018 14:25
    gianmaz opened #88
  • Mar 15 2018 20:05
    ChristopherWilks opened #87
  • Mar 15 2018 20:01

    ChristopherWilks on master

    switched to use sratoolkit 2.8.… (compare)

  • Mar 04 2018 22:50

    nellore on master

    patches bowtie2-build in travis… (compare)

  • Mar 04 2018 22:11

    nellore on master

    uses bowtie2 2.3.4.1 (compare)

  • Mar 04 2018 21:57

    nellore on master

    specifies samtools version to i… (compare)

  • Mar 04 2018 21:47

    nellore on master

    updates dependencies Merge branch 'master' of https:… (compare)

  • Mar 04 2018 21:39

    nellore on master

    quote rules Merge pull request #86 from Ben… (compare)

  • Mar 04 2018 21:39
    nellore closed #86
  • Feb 04 2018 01:11

    nellore on master

    enables --keep-alive for covera… (compare)

  • Jan 10 2018 13:09
    BenLangmead opened #86
  • Dec 29 2017 05:47

    nellore on master

    fixes unit test in bed_pre (compare)

  • Dec 29 2017 03:11

    nellore on master

    writes coverage tsv, optionally… (compare)

  • Dec 28 2017 20:59

    nellore on master

    fixes issue wipassing temp dir … Merge branch 'master' of https:… (compare)

  • Dec 28 2017 20:53

    nellore on master

    fixes issue wipassing temp dir … (compare)

  • Dec 23 2017 00:30

    nellore on master

    fixes misplaced joins in except… (compare)

abhinav
@nellore
but don't worry about the return now
keep your last change
the addition of the not
and just remove the --intermediate-lifetime -1 from your rail-rna command
i think the trouble is it's now sending an invalid number of days as expiration
Julia di Iulio
@juliadiiulio_twitter
wait it looks like without the --intermediate-lifetime -1 and keeping the "return" and the "not", it woks !!!! youhouhouhouh well I got another error, but at least it started :)
abhinav
@nellore
yay!
what's the other error?
Julia di Iulio
@juliadiiulio_twitter
NameError: global name 'Url' is not defined
but I might have messed the manifest file... let me check
abhinav
@nellore
wow
where are you getting that?
can you paste the full traceback?
Julia di Iulio
@juliadiiulio_twitter
Loading...
Checked all files listed in manifest file.
Copied Rail-RNA and bootstraps to S3.
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/raildotbio/rail-rna/main.py", line 975, in <module>
ec2_slave_security_group_id=args.ec2_slave_security_group_id
File "/usr/local/raildotbio/rail-rna/rna/driver/rna_config.py", line 6477, in init
profile=base.profile))
File "/usr/local/raildotbio/rail-rna/rna/driver/rna_config.py", line 3938, in init
if not Url(assembly).is_s3:
NameError: global name 'Url' is not defined
abhinav
@nellore
oh this is happening because you're not selecting one of hg19, hg38, mm9, mm10, dm3, dm6 as the assembly
can you paste me your rail-rna command
(it's also a bug in the code i'm just making a commit to fix now_
)
Julia di Iulio
@juliadiiulio_twitter
rail-rna go elastic -m s3://rail-rna-pdx/data/RNAseq/manifest_20170808_2samples_railrna.txt -a s3://rail-rna-pdx/Genome/Index/RailRNA/hg38_ERCC92.tgz -o s3://rail-rna-pdx/jdiiulio/storage/analysis/RNAseq/ElasticMode/20170816 --core-instance-type c3.2xlarge --master-instance-type c3.2xlarge -c 8 --region us-west-2 -d idx,tsv,bed,bam,bw,jx --verbose --action-on-failure "TERMINATE_CLUSTER" --master-instance-bid-price 0.15 --core-instance-bid-price 0.15
abhinav
@nellore
ahhh okay you want to use your own assembly
Julia di Iulio
@juliadiiulio_twitter
ya :)
abhinav
@nellore
ok /usr/local/raildotbio/rail-rna/rna/driver/rna_config.py, line 3939, change Url(assembly) to ab.Url(assembly)
sorry, there are so many combinations of parameters to test in this software
hard to be comprehensive
Julia di Iulio
@juliadiiulio_twitter
no worries at all
it's running for 27 seconds!! I'ts a record :smile:
abhinav
@nellore
yeah but it's gonna fail at align cuz of your local-mode error ;P
unless this is different data!
Julia di Iulio
@juliadiiulio_twitter
haha well it just failed :)
abhinav
@nellore
where?
Julia di Iulio
@juliadiiulio_twitter

~.oOo.>

00h:00m:00s || Read job flow from input JSON.
00h:00m:15s |
| Verified that output directories on S3 are writable.
00h:00m:35s |___| Set up output directories on S3.
*Errors encountered*
Job flow failed on Monday, Aug 21, 2017 at 10:17:01 PM UTC. Run time was 35.845 seconds.
Traceback (most recent call last):
File "app_main.py", line 75, in run_toplevel
File "/usr/local/raildotbio/rail-rna/dooplicity/emr_runner.py", line 285, in <module>
args.aws_exe, args.profile, args.region)
File "/usr/local/raildotbio/rail-rna/dooplicity/emr_runner.py", line 226, in run_job_flow
job_flow_response = aws_ansible.post_request(full_payload)
File "/usr/local/raildotbio/rail-rna/dooplicity/ansibles.py", line 719, in post_request
service=self.service
File "/usr/local/raildotbio/rail-rna/dooplicity/ansibles.py", line 449, in aws_signature
('AWS4' + secret_key).encode('utf-8'),
TypeError: unsupported operand type(s) for +: 'str' and 'NoneType'

abhinav
@nellore
whoa
ok
well
that's because
well, why is that
hm
can you less ~/.aws/credentials and tell me if your access key ID and secret access key are present there
don't paste them here!
Julia di Iulio
@juliadiiulio_twitter
they are commented out... it had something to do with the IAM role ... not sure why but devops, asked me to comment them, I 'll try to uncomment!
abhinav
@nellore
cool
Julia di Iulio
@juliadiiulio_twitter
Success!!! looks like it's working !!!
Thanks so much
abhinav
@nellore
sure, but we'll see :P
another thing
oh, nm
yeah, mention @nellore if something else comes up
Julia di Iulio
@juliadiiulio_twitter
:thumbsup:
Julia di Iulio
@juliadiiulio_twitter

Hey @nellore, turns out that if I uncomment my credentials, the IAM role that was attached to my instance by devops is no longer working... (I talked to them, and this is why they had previously asked me to comment out my credentials in ~/.aws/credentials) which yield to the following error

*Errors encountered*
Job flow failed on Monday, Aug 21, 2017 at 10:38:37 PM UTC. Run time was 142.681 seconds.
Traceback (most recent call last):
File "app_main.py", line 75, in run_toplevel
File "/usr/local/raildotbio/rail-rna/dooplicity/emr_runner.py", line 285, in <module>
args.aws_exe, args.profile, args.region)
File "/usr/local/raildotbio/rail-rna/dooplicity/emr_runner.py", line 230, in run_job_flow

+ (' ensure that IAM roles are '

RuntimeError: (HTTP Error 400: Bad Request); ensure that IAM roles are configured properly. This may require talking to your AWS account admin. ...

Do you think there is a way to hack rail to prevent the previous error (the one that I got when I didn't have my credentials in ~/.aws/credentials) ? or does that sound difficult ?

abhinav
@nellore
are you launching an EMR cluster from an EC2 instance or from your laptop?