Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Trevor Rowe
@trevorrowe
@djcp Absolutely.
Daniel Collis-Puro
@djcp
@trevorrowe - I'm attempting to add a waiter to the IAM client to ensure the Instance Profile is available but I am getting Aws::Waiters::Errors::NoSuchWaiterError: no such waiter :instance_profile_available; valid waiter names are: no matter what. Link: harvard-dce/aws-sdk-ruby@bd9510c
The list of valid waiter names is always empty. What am I missing?
Daniel Collis-Puro
@djcp
It's weird because what I'm doing looks very similar to this commit: aws/aws-sdk-ruby@0d054b6
Daniel Collis-Puro
@djcp
forget it - I've got it working. It was an issue with binstubs in my testing environment. We're good!
lisabien
@lisabien
Resource metadata seems inconsistent. For example, EC2.resources.json gives us something like this for resources:
    "DhcpOptions": {
      "identifiers": [
        {
          "name": "Id",
          "memberName": "DhcpOptionsId"
        }
      ],

This is great for determining the underlying resource ID name. However CloudFormation.resources.json looks like this:

    "Stack": {
      "identifiers": [
        { "name": "Name" }
      ],

There's no memberName to reveal the underlying name (e.g. "StackName"). Which example is the standard, and what is the best way to determine the underlying resource ID name?

Trevor Rowe
@trevorrowe
The memberName is only populated when the identifier is echoed as part of the named “shape” from the API model. If the resource has no shape (additional properites) or if the property does not repeat the identifier, it has no shape “memberName”.
Trevor Rowe
@trevorrowe
@lisabien I took a look at the cloudformation resource model, and it appears that some of the resources, like Stack, were missing memberName traits where they should have been applied. See: aws/aws-sdk-ruby@e5d5eae
lisabien
@lisabien
Cool, thx! It appears that other resource files, like OpsWorks (and maybe others, but didn't check) have the same missing memberName issue. Would it be possible to get the resource files updated with memberName, too?
Trevor Rowe
@trevorrowe
Its possible for the services where the value is present in the shape, which is not always. If you know of any service missing them, please let me know.
lisabien
@lisabien
Will do. I'll go back and take a look. Thx!
Trevor Rowe
@trevorrowe
Out of curiosity, what are you consuming the files for?
lisabien
@lisabien
I'm working with @tve, our use case is a proxy that exposes a standardized json interface and also a restful interface. this will be used by a set of other services and we want to feed everything through the proxy for a variety of reasons.
Trevor Rowe
@trevorrowe
Sounds cool, keep the questions coming.
lisabien
@lisabien
Why does API metadata (e.g. EC2's DescribeKeyPairs) specify their parameter names in CamelCase, when the underlying call actually expects snake_case?
KeyNames vs. key_names
Trevor Rowe
@trevorrowe

EC2 is a special case. Every other AWS service uses a consistent case (upper or lower camel case typically). EC2 uses Upper camel case for most input values and lower camel case for most outputs. That said, there is still a desire to share the structures between input and output as their only difference is the serialized format of their member names.

To work around this, EC2 is not marked as a “query” protocol, instead it is marked as “ec2”. The made-up-by-the-sdk-team “ec2” protocol is defined a query with modifications. Most of the deltas are on how to marshal input. One of those being how to case them as they are sent, but other differences exist. Some of these are all lists are “flattened” on input (no .member prefix on list members) and there is an extra “queryName” that overrides the typical “locationName” (location name is inflected to UpperCamelCase, where queryName is taken as is).

I have some internal documentation I might be able to scrounge up that documents the differences.

Also, the Ruby SDK chooses to snake_case all input and outputs, reguardless of their protocol wire format, but I don’t think that is what you are referencing.
lisabien
@lisabien
Thank you. I'm sorry, I should have specified, yes we are seeing this with Ruby SDK.
Trevor Rowe
@trevorrowe
This is by-design, as Ruby methods are snake_case by convention.
lisabien
@lisabien
So, is Ruby SDK using the CamelCase metadata and converting to snake_case internally?
Trevor Rowe
@trevorrowe
There are two peices of information on every structure member, there is the chosen name of the references shape. I call this the member name. Then the actual reference can also provide a locationName. This is the name of the input/output value as it appears on the wire http request/response. The Ruby SDK will always use the snake_cased version of the member name. The locationName is an implementation detail for marshalling requests and un-marshalling responses.
lisabien
@lisabien
Cool. Thx!
lisabien
@lisabien
I believe in EC2.resources.json, that the "Volume" resource is missing the "Delete" action:
https://github.com/aws/aws-sdk-ruby/blob/master/aws-sdk-core/apis/EC2.resources.json#L1732-L1812
Should it have a "Delete" action defined for it?
Also, in EC2.resources.json, there is a "CreateInstances" action defined to map to "RunInstances". Would you consider adding a "Delete" action for the "Instance" resource to call "TerminateInstances"?
Trevor Rowe
@trevorrowe
The #create_instances call exists as primary mechanism for running a new instance. To delete an instance you can call #terminate on an instance or #terminate on a batch of instances:
ec2 = Aws::EC2::Resource.new

# single instance terminate
ec2.isntance(‘id’).terminate

# batch terminate
ec2.instances(filters: […]).terminate
You are correct, that Volume is missing a Delete action, an accidental omission. Please feel free to submit a PR.
lisabien
@lisabien

With regard to Instances, would you be opposed to exposing a "Delete" action on the Instance resource to also map to "TerminateInstances", in addition to the existing "Terminate" action that maps to "TerminateInstances"?

It just seems more CRUD-like and consistent that resources have both a CreateResource and a Delete, rather than instances having "CreateInstances" and "TerminateInstances".

For example, here: https://github.com/aws/aws-sdk-ruby/blob/master/aws-sdk-core/apis/EC2.resources.json#L14-L23

CreateInstances gets mapped to RunInstances. Just seems like having the complement for Delete would be nice. Something like this in the actions for an Instance resource:

        "Delete": {
          "request": {
            "operation": "TerminateInstances",
            "params": [
               { "target": "InstanceIds[0]", "source": "identifier", "name": "Id" }
            ]
          }
        }

If you're not opposed, I could submit a PR. If you are opposed, no worries, just thought we'd throw that idea out there. :)

Trevor Rowe
@trevorrowe
The reasoning behind not using delete was to disambiguate a call to terminate from a call to stop (StopInstances).
lisabien
@lisabien
Ok, thanks for the info!
Daniel Collis-Puro
@djcp
Gotta say, I was a little confused by the wait_until semantics. I have a PR in the works lighting up a few additional waiters, maybe I'll take a stab at clarifying the docs too.
I was expecting blocks I pass to wait_until to be run after the process was complete, and that's how I've seen other async waiters implemented.
I kinda like that pattern because it makes your intention clear: "run the code I'm passing you when what we're waiting for is done", rather than "block the main thread and then do the next thing)
And the confuse the matter, the blocks ran, but instantly. Anyway, now that I understand it it's fine, just not what I was expecting.
Daniel Collis-Puro
@djcp
The now deprecated Capybara.wait_until is probably what I was getting confused by, among other ruby interfaces.
Trevor Rowe
@trevorrowe
@djcp I agree the final interface ended up a bit akward. I would be willing to explore other interfaces. The functionality needs to remain consistent, but I’m open to deprecating the current method in favor of a better interface. The current interface would need to stick around for backwards compatability.
Josep M. Blanquer
@blanquer
@trevorrowe , re: "The reasoning behind not using delete was to disambiguate a call to terminate from a call to stop (StopInstances)" ... I'm not sure that is necessarily any more ambiguous than having a create and a start.
having a consistent create and delete is powerful, and in fact, you do it for most if not all the other resources, I would strongly consider not making Instance a weird one off that doesn't have one. I don't think that developers using the metadata would be confused about that at all.
Trevor Rowe
@trevorrowe
@blanquer To be fair, most every other resource delete method wraps a client method that contains the word “Delete” is its name. I understand the point with consistancy, but I am really not a fan of duplicating the definition of terminate simply to provide an alias. The resource definition model does not provide a mechanism to provide alternate names, but maybe it should. I’ll bring this up with a few other engineers that I work with that also consome these definitions and see what their opinion is.
Josep M. Blanquer
@blanquer
sure, but isn't this argument contrary to providing creates? those are aliases and exhibit the same issues you say you don't like no?
Trevor Rowe
@trevorrowe

Thats not what I meant. What I don’t like is having two entries in the resource definition for two actions that are exact copies of each other. One action should be an alias, not a straight up copy, of the other. As I pointed out, we don’t have aliases.

With regard to the create method being an alias, it is not, as it is the only action present that calls RunInstances. I would take up the same argument if the request was to add a “RunInstances” action that did the same thing as the current create.

Also, we’ve reviewed these APIs, especially for naming, with the service teams that they represent. It was not always clear which naming approach is best as there are trade-offs. Again, I’m open to exploring adding proper aliases where the definition is not duplicated.
Josep M. Blanquer
@blanquer
ok, I understand how you guys see it. That's fine. I was just trying to say that from a client perspective trying to provide some naming consistencies across resources (and services) goes a long way, especially when you're dealing with the sheer amount of resources and services you have. But, yes, everybody looks at these things through their own glasses. thanks for the prompt response.
Ryan Doherty
@ryandoherty
Hey folks, working with the Aws::SQS::Resource object and I'm confused. Does it have any methods for getting queues, pushing/pull messages?
calling .inspect and .methods on a resource object doesn't show anything I can use afaict
docs don't show anything either
Stevenson Jean-Pierre
@Sjeanpierre
I don't think the Aws::SQS::Resource object has that capability
Trevor Rowe
@trevorrowe
@ryandoherty The resource classes were removed from the last preview release of the v2 SDK before it went stable 2.0 -- The SQS interfaces were not finalized and will be revisited at some point.
gudstrand
@gudstrand
I am trying to upload a file to amazon S3 via the aws-sdk gem. I am able to connect but I don't know how to upload a file. Does anyone have a simple example?