aws/aws-sdk-core-ruby

Please join http://gitter.im/aws/aws-sdk-ruby instead.

Gerardo Santana

@santana

both, profiles: and regions: accept arrays, so you can iterate over the ones you want only; otherwise it iterates over all of them

Trevor Rowe

@trevorrowe

So, adding an enumerable interface to Aws::SharedCredentials would provide you the ability to enumerate profiles, but there is not currently anything analgous to AWS.regions in the v2 SDK.

The v1 SDK maintained a static json document of regions, but the v2 SDK does not ship with this, as it was a maintenance burden.

Gerardo Santana

@santana

can’t this help? http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeRegions.html

Trevor Rowe

@trevorrowe

This is an authenticated API call, that would require credentials, and only returns information about the public regions, e.g.

["eu-central-1", "sa-east-1", "ap-northeast-1", "eu-west-1", "us-east-1", "us-west-1", "us-west-2", "ap-southeast-2", "ap-southeast-1”]

- This currently omit gov cloud and the newer cn-north-1 region (as these require different accounts). In theory, calling this from a gov cloud endpoint would returns different results, as would calling it against a cn-north-1 endpoint would. I would need to know how your credentials are scoped to know what enpdoints they would be valid against. Without some initial config I can not bootstrap that call.

Gerardo Santana

@santana

Trevor Rowe

@trevorrowe

I’m open to suggestions and if there a clear solution I’d be willing to work towards it.

Gerardo Santana

@santana

I appreciate that, thanks. I was certainly relying on AWS.regions’ magic.

I’m not in the gov cloud

I wonder if there’s a way to make regions maintenance less painful

Gerardo Santana

@santana

I do have an idea

DescribeRegions is an authenticated call you say, and that’s fine

I’m reviewing my code, I make a call to AWS.config twice

I’m retrieving the regions after the first call to AWS.config

that is, 1) make a connection to a default region, 2) retrieve list of regions, 3) iterate and yield

(3) should actually be iterate over profile-region, call AWS.config, yield

Trevor Rowe

@trevorrowe

If you are not concerned about connecting to the gov cloud region(s), or the Bejing regions, then this would work. It would have the benefit that as new regions are rolled out you would start accessing them, without needing to update the SDK.

As far as I know, EC2 is deployed by default into every new region. If a new region ever cropped up and there was no EC2 there, this would not catch that. I don’t think that is something to worry about though.

Gerardo Santana

@santana

agreed

I just would like to get rid of maintaining my own gem. I’m happy to write a patch for the SDK v2 if necessary.

Trevor Rowe

@trevorrowe

I’d be open to accepting a pull-request that added a .each method to Aws::SharedCredentials. This would simply enumerate instances of Aws::SharedCredentials, once for each profile.

Gerardo Santana

@santana

I can do that.

I would also love to see a single method call to iterate over each profile-region, as described above. Is there any interest in that too?

Trevor Rowe

@trevorrowe

Im interested if there is a mechanism to do this globally. As it, its a bit of a leaky abstraction that doesn’t work well for accounts not in the primary “public cloud”.

Gerardo Santana

@santana

Christian Bay

@valleybay

I am trying to use this gem with Rails, but where do I actually configure the require?

do I just include it in config/initializers/aws.rb?

Gerardo Santana

@santana

is there a reason for allowing each Aws::SharedCredenials instance to have a different path to the credentials file?

if there’s none, we could parse the ini file once (instead of per instance) and provide a list of profiles from a class method, available to all instances

Trevor Rowe

@trevorrowe

@santana The purpose of supporting a path is to allow the instance profile credentials to be loaded in environments where a/the home directory is not available for the current user, but the file is available on disk.

Wouldn’t the following work?

Aws::InstanceProfileCredentials.new.each_profile do |profile_name, credentials|
  # yielded creds are Aws::Credentials
end

Ryan Romanchuk

@rromanchuk

Really need this functionality in V2 http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/SNS/Message.html

Crypto should be the first thing to get ported

Trevor Rowe

@trevorrowe

@rromanchuk Are you referencing the #authentic? helper method on AWS::SNS::Message?

Ryan Romanchuk

@rromanchuk

@trevorrowe yeah sorry, extremely timesaving method when dealing with https SNS delivery

Trevor Rowe

@trevorrowe

@rromanchuk Would you like to take a look at a pull-request I made to add sns message authentication?

aws/aws-sdk-ruby#741

Ryan Romanchuk

@rromanchuk

@trevorrowe amazing!

kevin-staiger

@kevin-staiger

I have noticed that sqs .poll and receive_message both automatically delete the message after receiving, is there any way to not delete it?

Erik Straub

@brkattk

Hey everyone. I’m attempting to get a stubbed Aws::AssumeRoleCredentials instance with the following

        aws_sts_client = Aws::STS::Client.new( credentials: credentials,
                                               region: DEFAULT_AWS_REGION)

        Aws::AssumeRoleCredentials.new( client: aws_sts_client,
                                        role_arn: profile[:role_arn],
                                        role_session_name: "#{profile[:name]}-session",
                                        duration_seconds: DEFAULT_AWS_TOKEN_DURATION)

credentials is either an instance of Aws::SharedCredentials or Aws::Credentials

My issue is I keep seeing this error:

NoMethodError:
       undefined method `access_key_id' for nil:NilClass
# /Users/brkattk/.rbenv/versions/2.2.2/gemsets/project-name/gems/aws-sdk-core-2.0.37/lib/aws-sdk-core/assume_role_credentials.rb:39:in `refresh'
# /Users/brkattk/.rbenv/versions/2.2.2/gemsets/project-name/gems/aws-sdk-core-2.0.37/lib/aws-sdk-core/refreshing_credentials.rb:20:in `initialize'
# /Users/brkattk/.rbenv/versions/2.2.2/gemsets/project-name/gems/aws-sdk-core-2.0.37/lib/aws-sdk-core/assume_role_credentials.rb:29:in `initialize'
# ./lib/project-name/configuration/aws_parser.rb:106:in `new'
# ./lib/project-name/configuration/aws_parser.rb:106:in `assume_role_credentials'
# ./lib/project-name/configuration/aws_parser.rb:83:in `build_credentials'
# ./lib/project-name/configuration/aws_parser.rb:63:in `block in validate_profiles'
# ./lib/project-name/configuration/aws_parser.rb:60:in `each'
# ./lib/project-name/configuration/aws_parser.rb:60:in `validate_profiles'
# ./lib/project-name/configuration/aws_parser.rb:49:in `validate!'
# ./lib/project-name/configuration/aws_parser.rb:29:in `parse!'
# ./spec/lib/project-name/configuration/aws_parser_spec.rb:186:in `block (9 levels) in <top (required)>'
# ./spec/lib/project-name/configuration/aws_parser_spec.rb:189:in `block (9 levels) in <top (required)>'

I’m assuming the general Aws.config[:stub_responses] = true does not construct the actual responses you need, just simply stops the HTTP call from happening, right?

Trevor Rowe

@trevorrowe

Thats correct. You can specify the data a stub should return, but by default it returns sample empty responses.

Erik Straub

@brkattk

ok. I guess I might as well use webmock and do it myself then. do you happen to know where I might find an example response? edit: http://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html

Trevor Rowe

@trevorrowe

If you can send an actual live request, then the SDK can provide you the http response. For example:

resp = aws_sts_client.assume_role(
  role_arn: profile[:role_arn],
  role_session_name: "#{profile[:name]}-session",
  duration_seconds: DEFAULT_AWS_TOKEN_DURATION)

resp.context.http_response.status_code
resp.context.http_response.headers
resp.context.http_response.body_contents

Erik Straub

@brkattk

I have an app that uses multiple different AWS credentials to do various things and eventually upload a file to S3. I’ve got an array of environments like so

[
  {
    :name=>"default”,
    :bucket_name=>”my.bucket”,
    :bucket_region=>”us-east-1”,
     :credentials=>#<Aws::SharedCredentials profile_name="default" path="/Users/brkattk/.aws/credentials”>
  },
  {
    :name=>”secondary",
    :bucket_name=>”my.bucket”,
    :bucket_region=>”us-east-1”,
    :credentials=>#<Aws::SharedCredentials profile_name=“secondary" path="/Users/brkattk/.aws/credentials”>
  }
]

I’ve got a Report class that will upload a CSV to an environment’s designated bucket:

class Report
  att_reader :csv, :environment
  def initialize(environment, csv)
    @environment = environment
    @csv = csv
  end

  def upload!
    s3_bucket.put_object(
          body: csv.to_csv,
          key: csv.to_filename,
          content_type: 'text/csv')
  end

  private

    def s3_resource
      ::Aws::S3::Resource.new(client: s3_client)
    end

    def s3_bucket
      s3_resource.bucket environment[:bucket_name]
    end

    def s3_client
      ::Aws::S3::Client.new(credentials: environment[:credentials], region: environment[:bucket_region])
    end
end

I’m looping each environment and producing a report, however, it is uploading both files to the last environment’s bucket.

Am I going insane?

Erik Straub

@brkattk

is Aws::S3::Client meant to be a singleton?

Trevor Rowe

@trevorrowe

No, you can construct as many instances of Aws::S3::Client as you require.

This should not affect your issue, but you can reduce some of that code by not constructing the Resource and bucket:

S3::Object.new(bucket_name, key).put(body:’…’, content_type: ‘…')

Also, your issue is that it is “uploading both files to the last environment’s bucket” — in the example you gave above, they list the same bucket name.

Erik Straub

@brkattk

Aws::SharedCredentials.new profile_name: ‘blah’

does not do what it advertises @trevorrowe

regardless of bucket name, it should tell which account by the credentials passed to Aws::S3::Client, right?

my issue is that the default doesn’t even have a bucket named my.bucket

I’ve got a test for this:

Where communities thrive

People

Repo info

Activity