Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Ryan Doherty
@ryandoherty
docs don't show anything either
Stevenson Jean-Pierre
@Sjeanpierre
I don't think the Aws::SQS::Resource object has that capability
Trevor Rowe
@trevorrowe
@ryandoherty The resource classes were removed from the last preview release of the v2 SDK before it went stable 2.0 -- The SQS interfaces were not finalized and will be revisited at some point.
gudstrand
@gudstrand
I am trying to upload a file to amazon S3 via the aws-sdk gem. I am able to connect but I don't know how to upload a file. Does anyone have a simple example?
Trevor Rowe
@trevorrowe
@gudstrand Sure, try the following, using version 2 of the SDK:
require 'aws-sdk'

s3 = Aws::S3::Resource.new(
  region: ‘us-east-1’, # this should be the region your bucket was created in
  credentials: Aws::Credentials.new(‘YOUR-ACESS-KEY’, ‘YOUR-SECRET-ACCESS-KEY’)
)
s3.bucket(’bucket-name’).object(’target-filename’).upload_file(‘/path/to/source/file’)
gudstrand
@gudstrand
thank you!
gudstrand
@gudstrand
I don't get any errors but I don't get a file either
not sure what i am doing wrong
gudstrand
@gudstrand
Thank you! I had an error (using Client instead of Resource). Once I fixed that it works a charm
I owe you a virtual beer
:-)
Peter Sankauskas
@pas256
Hey @trevorrowe, I've been lurking here for a while and am enjoying the conversations
Would be great to get the same thing set up for aws-flow-ruby
Trevor Rowe
@trevorrowe
@gudstrand You can use Aws::S3::Client to upload files, but you have the manage the upload more. You have to choose between using #put_object or using the multipart file APIs. Also, you need to open the source file for reading, etc. The #upload_file will intelligently switch to a manged multipart upload using multiple threads for performance on large objects.
Trevor Rowe
@trevorrowe
@pas256 Hey, good to see you here! Hosting a gitter channel has been an experiment, not all of the SDK teams are doing it. I’ve found it very useful though. I can certainly forward your request onto to the SWF team.
gudstrand
@gudstrand

I am back again, still missing something. This seems like it should work but it doesn't, I get an AccessDenied. I am trying to just list the objects in a bucket, these are objects that I have successfully put there using my ruby script (Thanks Trevor!) . This is the script which I am

code
credentials = Aws::Credentials.new(@aws_access_key_id, @aws_secret_access_key)
client = Aws::S3::Client.new(
region: 'us-east-1', # this should be the region your bucket was created in
credentials: credentials)
client.list_objects(bucket: "<bucket-name>")

It is the last line that throws an error,

<Error><Code>AccessDenied</Code><Message>Access Denied</Message>

I know my credentials are correct and I verified the region. My bucket is US standard and the docs say to use us-east-1
which I am. I did find another suggestion to add rescue statement and get the error, but that did not provide me with any additional error
Trevor Rowe
@trevorrowe
@gudstrand Do you know if your bucket has a different ACL or Policy applied that might limit the ability to list objects? That is my best guess.
abalakersky
@abalakersky
@trevorrowe Thank you very much for providing a sample above on the S3 upload using #upload_file. I was hoping to find more information regarding that in the v2 SDK documentation but was unable to. Could you point me to the correct place? Also, what would be the proper way to use Aws::S3::Client and multipart_upload? We've been struggling with with how to manage create and manage parts for the object.
Peter Sankauskas
@pas256
@trevorrowe Yes please, that would be great
Daniel Berlinger
@danielberlinger
Hi, can someone tell me if v2 uses SSL by default?
I don't (in my quick look) see the equivalent of :use_ssl
Trevor Rowe
@trevorrowe
@danielberlinger It forces SSL everywhere by default. The only mechanism to disable SSL is to provide a url as :endpoint that uses the http scheme instead of https.
flushentitypacket
@flushentitypacket
Anyone know if batch sending SQS via #batch_send charges per message, or if that counts as a single request?
I'm also curious to know how it works internally, but I'm having trouble viewing the source for #send_message_batch.
Nitin
@nitinmohan87
i am noticing minor discrepancies between EC2.api.json and the AWS EC2 API docs
For example, when creating a SecurityGroup the AWS docs shows GroupName and GroupDescription as the required parameters. But the EC2.api.json metadata and therefore the ruby client expects GroupName and Description (instead of GroupDescription)
Why do we see such differences between the docs and the metadata?
Nitin
@nitinmohan87
For CreatePlacementGroup call should the identifier target be Name instead of Id? https://github.com/aws/aws-sdk-ruby/blob/master/aws-sdk-core/apis/EC2.resources.json#L68
I think the identifiers for the PlacementGroups are Name
Nitin
@nitinmohan87
I also find that the RouteTable is missing the DeleteRouteTable action in the EC2.resources.json
 "actions": {
  "Delete": {
    "request": {
      "operation": "DeleteRouteTable",
        "params": [
          { "target": "RouteTableId", "source": "identifier", "name": "Id" }
        ]
     }
  }
}
if you think the CreatePlacementGroup and the missing RouteTable Delete action are valid bugs I can definitely put in a pull request to fix them
Trevor Rowe
@trevorrowe
@flushentitypacket The Amazon SQS pricing page indicates you are billed per reqeust, not per message: http://aws.amazon.com/sqs/pricing/ - The Aws::SQS::Client#send_message_batch method allows you to send up to 10 messages in a single request. The 10 limit is imposed by the service, not the client. There is also a maximum payload size of 256KB per request. So if your messages are larger, you may not be able to send 10 in a single request without exceeding that size limit.

@nitinmohan87 I’ll try to answer these in order. The AWS EC2 documentation is documenting the wire-protocol, the format of the query string parameters as they are marshalled onto an HTTP request as a GET querystring or via a POST body. That representation is an implementation detail. The AWS SDKs, all of the, share a model of the API that gives friendlier, context sensitive names for many things. Some of these names will provide a pluralized name for a list where the wire procotol uses a singular name. Some will removed prefixes where it can be inferred from the context.

When using the Ruby SDK, you should be referencing its API documentation, not service protocol API docs.

Trevor Rowe
@trevorrowe
You are correct, the create placement group call should use the “Name” for the identifier, not “Id”. That is a bug and should be changed. If you could please submit a PR, that would be helpful.
I wouldn’t call a missing action a bug, just a missing feature. :) Yes, please do submit a pull request for adding Delete to RouteTable.
I have a work-in-progress model validator that should catch bugs like the incorrect identifier. My intention is to wire this up to our model builders so these are caught by an automated tool, instead of by users.
Nitin
@nitinmohan87
thanks for the explanation..
i will send a PR for to fix the CreatePlacementGroup and add a new feature to add Delete action to RouteTable ;)
is it ok if i send one PR that includes both changes?
Trevor Rowe
@trevorrowe
@nitinmohan87 One is fine.
Nitin
@nitinmohan87
Here it is - aws/aws-sdk-ruby#737 .. Thanks
Daniel Berlinger
@danielberlinger
@trevorrowe Thanks. And thanks for all the work on on this gem.
Brent Wheeldon
@BrentWheeldon
Hi there. We're using the aws-sdk ruby gem and have just upgraded from v1 to v2. I know that this is a breaking change, but I can't seem to find any documentation on what the breaking changes are, and how to fix them. Does such a doc exist, or is there one in the works? Thanks!
Trevor Rowe
@trevorrowe

@BrentWheeldon There is not an upgrading document currently. I hope to put together an upgrading guide soon. You can use v1 and v2 in the same application.

# in your Gemfile
gem 'aws-sdk’, '~> 2'
gem 'aws-sdk-v1'

# in your app
require 'aws-sdk'
require 'aws-sdk-v1’

AWS # v1 namespace
Aws # v2 namespace

This allows you to upgrade gradually.

Brent Wheeldon
@BrentWheeldon
Thanks, @trevorrowe. We'll just lock to 1 now. We don't have much code using it, so it won't take us long to upgrade once we know where to start.
Trevor Rowe
@trevorrowe
@BrentWheeldon I can definetly answer specific questions. If you are using v1 clients, almost nothing has changed. Most of what is different is basic configuration and some of the objet-oriented interfcaes on services.
Brent Wheeldon
@BrentWheeldon

@trevorrowe our main use case is something like:

s3 = AWS::S3.new(access_key_id: key_id, secret_access_key: access_key)
bucket = s3.buckets[bucket_name]
bucket.objects["file_name"].write(contents)

Seems like we now need to use AWS::S3::Client, specify the region as well as our credentials, and then use the put_object method rather than everything being a hash?

Trevor Rowe
@trevorrowe

@BrentWheeldon The client class is optional. You can use Aws::S3::Resource. That said, there is no longer a default region. The old default was ‘us-east-1’, so that should still work:

s3 = Aws::S3::Resource.new(
  access_key_id: key_id,
  secret_access_key: access_key,
  region: ‘us-east-1’)

Using the s3 resource object, you can upload an object in one of two ways. You can use the #put method, or you can use the #upload_file method. Using put will upload the entire object in a single request, but it is not recomended for large objects. If you prefer to upload a file from disk, use the #upload_file method.

obj = s3.bucket(bucket_name).object('file_name')

# single request, up to 5GB, contents can be a string or an IO object
obj.put(body:contents)

# managed upload, intelligently uses the multipart API for larger objects. Better reliability and performance
obj.upload_file('/path/to/source/file')
Brent Wheeldon
@BrentWheeldon
Nice, thanks for that info!
William Wilson
@millenniumbrain
Excuse me, I was wondering how to format the options in the upload_file method in order to grant public read access. I am having trouble finding the string it wants.
Trevor Rowe
@trevorrowe
Same options as Client#put_object, for example upload_file('source', acl: 'public-read')