Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 03 2018 22:03
    jpeddicord unassigned #126
  • Dec 03 2018 22:03
    jpeddicord unassigned #47
  • Sep 06 2017 05:22
    rocifier commented #187
  • Jun 26 2017 20:04
    wy193777 commented #138
  • Mar 01 2017 22:38
    awood45 closed #225
  • Mar 01 2017 22:38
    awood45 commented #225
  • Mar 01 2017 22:36
    iamatypeofwalrus edited #225
  • Mar 01 2017 22:36
    iamatypeofwalrus edited #225
  • Mar 01 2017 22:36
    iamatypeofwalrus edited #225
  • Mar 01 2017 22:35
    iamatypeofwalrus edited #225
  • Mar 01 2017 22:34
    iamatypeofwalrus opened #225
  • Sep 01 2016 18:35
    bmedici commented #194
  • Aug 30 2016 12:35
    fcarrega commented #154
  • Jul 25 2016 16:11
    heaven commented #166
  • Jul 25 2016 15:57
    trevorrowe commented #166
  • Jul 25 2016 02:23
    heaven commented #166
  • May 15 2016 22:02
    StevenHarlow commented #166
  • May 15 2016 21:58
    StevenHarlow commented #166
  • May 15 2016 21:58
    StevenHarlow commented #166
  • May 15 2016 21:53
    StevenHarlow commented #166
Gerardo Santana
@santana

I can do that.

I would also love to see a single method call to iterate over each profile-region, as described above. Is there any interest in that too?

Trevor Rowe
@trevorrowe
Im interested if there is a mechanism to do this globally. As it, its a bit of a leaky abstraction that doesn’t work well for accounts not in the primary “public cloud”.
Gerardo Santana
@santana
ok
Christian Bay
@valleybay
I am trying to use this gem with Rails, but where do I actually configure the require?
do I just include it in config/initializers/aws.rb?
Gerardo Santana
@santana
is there a reason for allowing each Aws::SharedCredenials instance to have a different path to the credentials file?
if there’s none, we could parse the ini file once (instead of per instance) and provide a list of profiles from a class method, available to all instances
Trevor Rowe
@trevorrowe

@santana The purpose of supporting a path is to allow the instance profile credentials to be loaded in environments where a/the home directory is not available for the current user, but the file is available on disk.

Wouldn’t the following work?

Aws::InstanceProfileCredentials.new.each_profile do |profile_name, credentials|
  # yielded creds are Aws::Credentials
end
Ryan Romanchuk
@rromanchuk
Crypto should be the first thing to get ported
Trevor Rowe
@trevorrowe
@rromanchuk Are you referencing the #authentic? helper method on AWS::SNS::Message?
Ryan Romanchuk
@rromanchuk
@trevorrowe yeah sorry, extremely timesaving method when dealing with https SNS delivery
Trevor Rowe
@trevorrowe

@rromanchuk Would you like to take a look at a pull-request I made to add sns message authentication?

aws/aws-sdk-ruby#741

Ryan Romanchuk
@rromanchuk
@trevorrowe amazing!
<3
kevin-staiger
@kevin-staiger
I have noticed that sqs .poll and receive_message both automatically delete the message after receiving, is there any way to not delete it?
Erik Straub
@brkattk

Hey everyone. I’m attempting to get a stubbed Aws::AssumeRoleCredentials instance with the following

        aws_sts_client = Aws::STS::Client.new( credentials: credentials,
                                               region: DEFAULT_AWS_REGION)

        Aws::AssumeRoleCredentials.new( client: aws_sts_client,
                                        role_arn: profile[:role_arn],
                                        role_session_name: "#{profile[:name]}-session",
                                        duration_seconds: DEFAULT_AWS_TOKEN_DURATION)

credentials is either an instance of Aws::SharedCredentials or Aws::Credentials

My issue is I keep seeing this error:

NoMethodError:
       undefined method `access_key_id' for nil:NilClass
# /Users/brkattk/.rbenv/versions/2.2.2/gemsets/project-name/gems/aws-sdk-core-2.0.37/lib/aws-sdk-core/assume_role_credentials.rb:39:in `refresh'
# /Users/brkattk/.rbenv/versions/2.2.2/gemsets/project-name/gems/aws-sdk-core-2.0.37/lib/aws-sdk-core/refreshing_credentials.rb:20:in `initialize'
# /Users/brkattk/.rbenv/versions/2.2.2/gemsets/project-name/gems/aws-sdk-core-2.0.37/lib/aws-sdk-core/assume_role_credentials.rb:29:in `initialize'
# ./lib/project-name/configuration/aws_parser.rb:106:in `new'
# ./lib/project-name/configuration/aws_parser.rb:106:in `assume_role_credentials'
# ./lib/project-name/configuration/aws_parser.rb:83:in `build_credentials'
# ./lib/project-name/configuration/aws_parser.rb:63:in `block in validate_profiles'
# ./lib/project-name/configuration/aws_parser.rb:60:in `each'
# ./lib/project-name/configuration/aws_parser.rb:60:in `validate_profiles'
# ./lib/project-name/configuration/aws_parser.rb:49:in `validate!'
# ./lib/project-name/configuration/aws_parser.rb:29:in `parse!'
# ./spec/lib/project-name/configuration/aws_parser_spec.rb:186:in `block (9 levels) in <top (required)>'
# ./spec/lib/project-name/configuration/aws_parser_spec.rb:189:in `block (9 levels) in <top (required)>'
I’m assuming the general Aws.config[:stub_responses] = true does not construct the actual responses you need, just simply stops the HTTP call from happening, right?
Trevor Rowe
@trevorrowe
Thats correct. You can specify the data a stub should return, but by default it returns sample empty responses.
Erik Straub
@brkattk
ok. I guess I might as well use webmock and do it myself then. do you happen to know where I might find an example response? edit: http://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html
Trevor Rowe
@trevorrowe
If you can send an actual live request, then the SDK can provide you the http response. For example:
resp = aws_sts_client.assume_role(
  role_arn: profile[:role_arn],
  role_session_name: "#{profile[:name]}-session",
  duration_seconds: DEFAULT_AWS_TOKEN_DURATION)

resp.context.http_response.status_code
resp.context.http_response.headers
resp.context.http_response.body_contents
Erik Straub
@brkattk

I have an app that uses multiple different AWS credentials to do various things and eventually upload a file to S3. I’ve got an array of environments like so

[
  {
    :name=>"default”,
    :bucket_name=>”my.bucket”,
    :bucket_region=>”us-east-1”,
     :credentials=>#<Aws::SharedCredentials profile_name="default" path="/Users/brkattk/.aws/credentials”>
  },
  {
    :name=>”secondary",
    :bucket_name=>”my.bucket”,
    :bucket_region=>”us-east-1”,
    :credentials=>#<Aws::SharedCredentials profile_name=“secondary" path="/Users/brkattk/.aws/credentials”>
  }
]

I’ve got a Report class that will upload a CSV to an environment’s designated bucket:

class Report
  att_reader :csv, :environment
  def initialize(environment, csv)
    @environment = environment
    @csv = csv
  end

  def upload!
    s3_bucket.put_object(
          body: csv.to_csv,
          key: csv.to_filename,
          content_type: 'text/csv')
  end

  private

    def s3_resource
      ::Aws::S3::Resource.new(client: s3_client)
    end

    def s3_bucket
      s3_resource.bucket environment[:bucket_name]
    end

    def s3_client
      ::Aws::S3::Client.new(credentials: environment[:credentials], region: environment[:bucket_region])
    end
end

I’m looping each environment and producing a report, however, it is uploading both files to the last environment’s bucket.

Am I going insane?

Erik Straub
@brkattk
is Aws::S3::Client meant to be a singleton?
Trevor Rowe
@trevorrowe
No, you can construct as many instances of Aws::S3::Client as you require.
This should not affect your issue, but you can reduce some of that code by not constructing the Resource and bucket:
S3::Object.new(bucket_name, key).put(body:’…’, content_type: ‘…')
Also, your issue is that it is “uploading both files to the last environment’s bucket” — in the example you gave above, they list the same bucket name.
Erik Straub
@brkattk
Aws::SharedCredentials.new profile_name: ‘blah’
does not do what it advertises @trevorrowe
regardless of bucket name, it should tell which account by the credentials passed to Aws::S3::Client, right?
my issue is that the default doesn’t even have a bucket named my.bucket
I’ve got a test for this:

I have 2 profiles in my ~/.aws/credentials that are completely separate AWS accounts (one is my personal, one for my company)

I created an ran this script. I do not have a bucket named reports in my personal AWS account, but this puts the first object in my company account’s report bucket

require 'rubygems'
require 'aws-sdk'

creds1 = Aws::SharedCredentials.new(profile_name: 'erik')
s3_client1 = Aws::S3::Client.new(region: 'us-east-1', credentials: creds1)
s3_resource1 = Aws::S3::Resource.new(client: s3_client1)
s3_bucket1 = s3_resource1.bucket('reports')
object = s3_bucket1.objects.first
p object
Trevor Rowe
@trevorrowe
What is Aws::SharedCredentials.new doing if not what you expect? The ability to write to a bucket is not strictly limited to a single account. That is the default, only the bucket owner may write, but you can open the public for public writes, or even white listed users and accounts.
Erik Straub
@brkattk
ugh.. hang on this might be my own stupidity
Trevor Rowe
@trevorrowe
When you construct a shared credentials object, inspect the #access_key_id and #secret_access_key. Are they for the correct profile?
Erik Straub
@brkattk
:cry: :gun: I definitely had a policy on that bucket that was open to any authenticated user
now I’m getting the appropriate AccessDenied exception
Peter Mounce
@petemounce
Hi @trevorrowe - could you give me a steer on how to create a waiter for a windows ec2 password being updated since <some date>?
The reason being is that I have ec2 instances where ec2config is configured to set a new administrator password on each launch. A reboot counts (apparently), and so i'm in a position where i'm baking an AMI, and need to orchestrate reboots, and therefore need to wait for the the password to both be available and different.
The EC2:GetPasswordData operation helpfully has a timestamp - so I think i just need to know how to create the waiter which takes a date time as a parameter to compare against
Trevor Rowe
@trevorrowe
Currently the client waiter interface only accepts inputs that are valid request parameters. That said, there is a custom waiter interface on the resource classes where you can pass a block and wait on it to return a truthy value.
instance = Aws::EC2::Resource.new.instances.first
old_timestamp = instance.password_data.timestamp
instance.reboot
instance.wait_until { |i| i.password_data.timestamp != old_timestamp }
Peter Mounce
@petemounce
@trevorrowe thanks; perfect!
one thing i did run into was that response in the block was nil on at least the first attempt, which seemed odd
i wanted to do
... do |attempts, response|
  puts "#{attempts}; timestamp this time: #{response.password_data.timestamp}"
end
but response was nil
Trevor Rowe
@trevorrowe
@petemounce Do you mean for the before_wait callback?
Peter Mounce
@petemounce
@trevorrowe yes
      proc = Proc.new do |attempts, response|
        logger.debug "Waiting for password data timestamp to be newer", {pwd_timestamp: instance.password_data.timestamp, newer_than: since}
      end
      instance.wait_until(max_attempts: 30, delay: 10, before_wait: proc) { |i| i.password_data.timestamp != since }
i is an Instance resource
Peter Mounce
@petemounce
@trevorrowe is there a way to globally change the global waiter? I was using the standard settings, and I'm getting RequestLimitExceeded exceptions, presumably because of all the Describe* requests.
to combat that, i've changed my waiters to look like, for example,
w.max_attempts = 15
w.interval = 0
w.before_attempt do |a, resp|
  pause_exponentially n
  logger.debug "Waiting for password...", {instance_id: id, attempt_count: a}
end

def pause_exponentially(n, seed=2, exp=1.4)
  sleep(seed * (exp ** n))
end