These are chat archives for bio4j/bio4j

30th
Aug 2015
Eduardo Pareja Tobes
@eparejatobes
Aug 30 2015 08:06
Hey @nlakhani did you grant yourself access to the bio4j bucket? You need to
There's a sample policy in the docs IIRC
All this requester pays business can be a bit confusing a first
nlakhani
@nlakhani
Aug 30 2015 14:06
Yes I created a user in IAM and applied the policy from the docs
Eduardo Pareja Tobes
@eparejatobes
Aug 30 2015 15:29
OK got it @nlakhani
You need to put requester there
Not your keys or anything
--request-payer requester
Like that
Eduardo Pareja Tobes
@eparejatobes
Aug 30 2015 15:34
That should be clearer in the docs
This should work (modulo setting default region etc):
aws s3api get-object --request-payer requester --bucket eu-west-1.releases.bio4j.com/ --key 2014_12_03/bio4j_all_but_uniref_and_gi_index.tar bio4j.tar
I use this daily from other accounts, it is pretty well tested. But as I said, the docs should make clear that you actually need to pass 'requester'
nlakhani
@nlakhani
Aug 30 2015 15:41
i had that initially but was advised to change it when i posted here, the docs do say to use requester
let me try this which you noted just now
i get this error, which was the original one i came to this gitter for
A client error (NoSuchKey) occurred when calling the GetObject operation: The specified key does not exist.
Alexey Alekhin
@laughedelic
Aug 30 2015 15:43
I'm sorry for the confusion :sweat:
nlakhani
@nlakhani
Aug 30 2015 15:43
heh no problem, thanks very much for helping
im not very experienced with aws generally so it was an opportunity to learn as well
Eduardo Pareja Tobes
@eparejatobes
Aug 30 2015 15:45
@nlakhani I just tested this locally
nlakhani
@nlakhani
Aug 30 2015 15:45
so strange
aws s3api get-object --request-payer requester --bucket eu-west-1.releases.bio4j.com/ --key 2014_12_03/bio4j_all_but_uniref_and_gi_index.tar bio4j.tar
literally copied what you wrote
Eduardo Pareja Tobes
@eparejatobes
Aug 30 2015 15:47
aws s3api get-object --request-payer requester \
--bucket eu-west-1.releases.bio4j.com \
--key 2014_12_03/bio4j_all_but_uniref_and_gi_index.tar bio4j.tar
this works
just tested
nlakhani
@nlakhani
Aug 30 2015 15:48

aws s3api list-objects --bucket eu-west-1.releases.bio4j.com

A client error (AccessDenied) occurred when calling the ListObjects operation: Access Denied

not sure if that is supposed to work, i confirmed the policy is as in the docs
Eduardo Pareja Tobes
@eparejatobes
Aug 30 2015 15:49
I use Java/Scala for this so I'm not really familiar with the cmdline thing
but what I just wrote works from my local machine, with a user from a different account
nlakhani
@nlakhani
Aug 30 2015 15:49
i pasted that last one
and it appears to be working!?
lol i cannot for the life of me see the difference between that and your previous one
Eduardo Pareja Tobes
@eparejatobes
Aug 30 2015 15:50
haha but there is
look at the bucket param :)
nlakhani
@nlakhani
Aug 30 2015 15:51
wow the trailing slash
thanks very much :clap:
im excited to get working on this, appreciate all your guys' help!
Eduardo Pareja Tobes
@eparejatobes
Aug 30 2015 15:53
no problem, sorry that all this is so inconvenient through the command line
will improve the docs, glad to be of help!
nlakhani
@nlakhani
Aug 30 2015 15:54
no worries, I am happy to help update the docs
and will hang out here to help any other newbies
Alexey Alekhin
@laughedelic
Aug 30 2015 16:00
:+1: nice!
Eduardo Pareja Tobes
@eparejatobes
Aug 30 2015 16:07
@nlakhani I improved things a bit in bio4j/bio4j-titan#67
nlakhani
@nlakhani
Aug 30 2015 22:01
that is more helpful for sure
i think the other thing that may help (which i am wondering now as I extract it) is what its fully expanded size is
so that when making an ec2 instance and optimizing for cost effectiveness one can make the instance the smallest sufficient size