Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Trevor Rowe
@trevorrowe
Out of curiosity, what are you consuming the files for?
lisabien
@lisabien
I'm working with @tve, our use case is a proxy that exposes a standardized json interface and also a restful interface. this will be used by a set of other services and we want to feed everything through the proxy for a variety of reasons.
Trevor Rowe
@trevorrowe
Sounds cool, keep the questions coming.
lisabien
@lisabien
Why does API metadata (e.g. EC2's DescribeKeyPairs) specify their parameter names in CamelCase, when the underlying call actually expects snake_case?
KeyNames vs. key_names
Trevor Rowe
@trevorrowe

EC2 is a special case. Every other AWS service uses a consistent case (upper or lower camel case typically). EC2 uses Upper camel case for most input values and lower camel case for most outputs. That said, there is still a desire to share the structures between input and output as their only difference is the serialized format of their member names.

To work around this, EC2 is not marked as a “query” protocol, instead it is marked as “ec2”. The made-up-by-the-sdk-team “ec2” protocol is defined a query with modifications. Most of the deltas are on how to marshal input. One of those being how to case them as they are sent, but other differences exist. Some of these are all lists are “flattened” on input (no .member prefix on list members) and there is an extra “queryName” that overrides the typical “locationName” (location name is inflected to UpperCamelCase, where queryName is taken as is).

I have some internal documentation I might be able to scrounge up that documents the differences.

Also, the Ruby SDK chooses to snake_case all input and outputs, reguardless of their protocol wire format, but I don’t think that is what you are referencing.
lisabien
@lisabien
Thank you. I'm sorry, I should have specified, yes we are seeing this with Ruby SDK.
Trevor Rowe
@trevorrowe
This is by-design, as Ruby methods are snake_case by convention.
lisabien
@lisabien
So, is Ruby SDK using the CamelCase metadata and converting to snake_case internally?
Trevor Rowe
@trevorrowe
There are two peices of information on every structure member, there is the chosen name of the references shape. I call this the member name. Then the actual reference can also provide a locationName. This is the name of the input/output value as it appears on the wire http request/response. The Ruby SDK will always use the snake_cased version of the member name. The locationName is an implementation detail for marshalling requests and un-marshalling responses.
lisabien
@lisabien
Cool. Thx!
lisabien
@lisabien
I believe in EC2.resources.json, that the "Volume" resource is missing the "Delete" action:
https://github.com/aws/aws-sdk-ruby/blob/master/aws-sdk-core/apis/EC2.resources.json#L1732-L1812
Should it have a "Delete" action defined for it?
Also, in EC2.resources.json, there is a "CreateInstances" action defined to map to "RunInstances". Would you consider adding a "Delete" action for the "Instance" resource to call "TerminateInstances"?
Trevor Rowe
@trevorrowe
The #create_instances call exists as primary mechanism for running a new instance. To delete an instance you can call #terminate on an instance or #terminate on a batch of instances:
ec2 = Aws::EC2::Resource.new

# single instance terminate
ec2.isntance(‘id’).terminate

# batch terminate
ec2.instances(filters: […]).terminate
You are correct, that Volume is missing a Delete action, an accidental omission. Please feel free to submit a PR.
lisabien
@lisabien

With regard to Instances, would you be opposed to exposing a "Delete" action on the Instance resource to also map to "TerminateInstances", in addition to the existing "Terminate" action that maps to "TerminateInstances"?

It just seems more CRUD-like and consistent that resources have both a CreateResource and a Delete, rather than instances having "CreateInstances" and "TerminateInstances".

For example, here: https://github.com/aws/aws-sdk-ruby/blob/master/aws-sdk-core/apis/EC2.resources.json#L14-L23

CreateInstances gets mapped to RunInstances. Just seems like having the complement for Delete would be nice. Something like this in the actions for an Instance resource:

        "Delete": {
          "request": {
            "operation": "TerminateInstances",
            "params": [
               { "target": "InstanceIds[0]", "source": "identifier", "name": "Id" }
            ]
          }
        }

If you're not opposed, I could submit a PR. If you are opposed, no worries, just thought we'd throw that idea out there. :)

Trevor Rowe
@trevorrowe
The reasoning behind not using delete was to disambiguate a call to terminate from a call to stop (StopInstances).
lisabien
@lisabien
Ok, thanks for the info!
Daniel Collis-Puro
@djcp
Gotta say, I was a little confused by the wait_until semantics. I have a PR in the works lighting up a few additional waiters, maybe I'll take a stab at clarifying the docs too.
I was expecting blocks I pass to wait_until to be run after the process was complete, and that's how I've seen other async waiters implemented.
I kinda like that pattern because it makes your intention clear: "run the code I'm passing you when what we're waiting for is done", rather than "block the main thread and then do the next thing)
And the confuse the matter, the blocks ran, but instantly. Anyway, now that I understand it it's fine, just not what I was expecting.
Daniel Collis-Puro
@djcp
The now deprecated Capybara.wait_until is probably what I was getting confused by, among other ruby interfaces.
Trevor Rowe
@trevorrowe
@djcp I agree the final interface ended up a bit akward. I would be willing to explore other interfaces. The functionality needs to remain consistent, but I’m open to deprecating the current method in favor of a better interface. The current interface would need to stick around for backwards compatability.
Josep M. Blanquer
@blanquer
@trevorrowe , re: "The reasoning behind not using delete was to disambiguate a call to terminate from a call to stop (StopInstances)" ... I'm not sure that is necessarily any more ambiguous than having a create and a start.
having a consistent create and delete is powerful, and in fact, you do it for most if not all the other resources, I would strongly consider not making Instance a weird one off that doesn't have one. I don't think that developers using the metadata would be confused about that at all.
Trevor Rowe
@trevorrowe
@blanquer To be fair, most every other resource delete method wraps a client method that contains the word “Delete” is its name. I understand the point with consistancy, but I am really not a fan of duplicating the definition of terminate simply to provide an alias. The resource definition model does not provide a mechanism to provide alternate names, but maybe it should. I’ll bring this up with a few other engineers that I work with that also consome these definitions and see what their opinion is.
Josep M. Blanquer
@blanquer
sure, but isn't this argument contrary to providing creates? those are aliases and exhibit the same issues you say you don't like no?
Trevor Rowe
@trevorrowe

Thats not what I meant. What I don’t like is having two entries in the resource definition for two actions that are exact copies of each other. One action should be an alias, not a straight up copy, of the other. As I pointed out, we don’t have aliases.

With regard to the create method being an alias, it is not, as it is the only action present that calls RunInstances. I would take up the same argument if the request was to add a “RunInstances” action that did the same thing as the current create.

Also, we’ve reviewed these APIs, especially for naming, with the service teams that they represent. It was not always clear which naming approach is best as there are trade-offs. Again, I’m open to exploring adding proper aliases where the definition is not duplicated.
Josep M. Blanquer
@blanquer
ok, I understand how you guys see it. That's fine. I was just trying to say that from a client perspective trying to provide some naming consistencies across resources (and services) goes a long way, especially when you're dealing with the sheer amount of resources and services you have. But, yes, everybody looks at these things through their own glasses. thanks for the prompt response.
Ryan Doherty
@ryandoherty
Hey folks, working with the Aws::SQS::Resource object and I'm confused. Does it have any methods for getting queues, pushing/pull messages?
calling .inspect and .methods on a resource object doesn't show anything I can use afaict
docs don't show anything either
Stevenson Jean-Pierre
@Sjeanpierre
I don't think the Aws::SQS::Resource object has that capability
Trevor Rowe
@trevorrowe
@ryandoherty The resource classes were removed from the last preview release of the v2 SDK before it went stable 2.0 -- The SQS interfaces were not finalized and will be revisited at some point.
gudstrand
@gudstrand
I am trying to upload a file to amazon S3 via the aws-sdk gem. I am able to connect but I don't know how to upload a file. Does anyone have a simple example?
Trevor Rowe
@trevorrowe
@gudstrand Sure, try the following, using version 2 of the SDK:
require 'aws-sdk'

s3 = Aws::S3::Resource.new(
  region: ‘us-east-1’, # this should be the region your bucket was created in
  credentials: Aws::Credentials.new(‘YOUR-ACESS-KEY’, ‘YOUR-SECRET-ACCESS-KEY’)
)
s3.bucket(’bucket-name’).object(’target-filename’).upload_file(‘/path/to/source/file’)
gudstrand
@gudstrand
thank you!
gudstrand
@gudstrand
I don't get any errors but I don't get a file either
not sure what i am doing wrong
gudstrand
@gudstrand
Thank you! I had an error (using Client instead of Resource). Once I fixed that it works a charm
I owe you a virtual beer
:-)
Peter Sankauskas
@pas256
Hey @trevorrowe, I've been lurking here for a while and am enjoying the conversations
Would be great to get the same thing set up for aws-flow-ruby
Trevor Rowe
@trevorrowe
@gudstrand You can use Aws::S3::Client to upload files, but you have the manage the upload more. You have to choose between using #put_object or using the multipart file APIs. Also, you need to open the source file for reading, etc. The #upload_file will intelligently switch to a manged multipart upload using multiple threads for performance on large objects.
Trevor Rowe
@trevorrowe
@pas256 Hey, good to see you here! Hosting a gitter channel has been an experiment, not all of the SDK teams are doing it. I’ve found it very useful though. I can certainly forward your request onto to the SWF team.
gudstrand
@gudstrand

I am back again, still missing something. This seems like it should work but it doesn't, I get an AccessDenied. I am trying to just list the objects in a bucket, these are objects that I have successfully put there using my ruby script (Thanks Trevor!) . This is the script which I am

code
credentials = Aws::Credentials.new(@aws_access_key_id, @aws_secret_access_key)
client = Aws::S3::Client.new(
region: 'us-east-1', # this should be the region your bucket was created in
credentials: credentials)
client.list_objects(bucket: "<bucket-name>")