aws/aws-sdk-ruby

The official AWS SDK for Ruby.

Daniel Collis-Puro

@djcp

So I've scripted stack creation via the ruby API. One odd thing - when I create an instance_profile I don't see it in the web console but I do when I call iam_client.list_instance_profiles

and it's associated with the stack properly - but this makes me think I'm missing something.

associated with the stack in the web console, that is.

Andrew Frank

@sirmxanot

@trevorrowe Thank you! Also, sorry for the obvious question :-)

Trevor Rowe

@trevorrowe

@sirmxanot No problem. Your question is valuable feedback. I’ve considered making this table more prominent, and I should probably move it from #waiter_names to #wait_until where people are looking.

@djcp I think the console has some magic around the creation of instance profiles. If I recall correctly, you have to create a role policy, and a role, associate them, then create an instance profile and associate the policy to that. The console profiles a single wizard for creating them all together. I suspect, though I don’t know, that only displays the policies that it manages.

Daniel Collis-Puro

@djcp

@trevorrowe Huh, interesting. I'll keep that in mind, IIRC I've arrived pretty much at what you layed out above. If I'm doing something wrong, I bet it'll be apparent when I get to adding instances.

Trevor Rowe

@trevorrowe

@djcp If I recall correctly, the only tricky for me was setting the correct assume role policy document. Its a json document that grants EC2 the right to assume a role on behalf of your account. There is a boilerplate document that I used. I haven’t set one up in a while.

Daniel Collis-Puro

@djcp

It seems like a bunch of clients lack waiters - Aws::IAM::Client for example. Is implementing a new waiter as easy as creating a new json definition and registering it with a service via Aws.service() or is there more to it? I am driving some operations that definitely need waiters and I would rather get them into the SDK than implement the semantics myself by hand.

@trevorrowe

Trevor Rowe

@trevorrowe

@djcp It is simply a matter of creating the waiter definitions as a json document and then add it to the add_service call.

Daniel Collis-Puro

@djcp

So should I do that and send you a PR, @trevorrowe ?

Trevor Rowe

@trevorrowe

@djcp Absolutely.

Daniel Collis-Puro

@djcp

@trevorrowe - I'm attempting to add a waiter to the IAM client to ensure the Instance Profile is available but I am getting Aws::Waiters::Errors::NoSuchWaiterError: no such waiter :instance_profile_available; valid waiter names are: no matter what. Link: harvard-dce/aws-sdk-ruby@bd9510c

The list of valid waiter names is always empty. What am I missing?

Daniel Collis-Puro

@djcp

It's weird because what I'm doing looks very similar to this commit: aws/aws-sdk-ruby@0d054b6

Daniel Collis-Puro

@djcp

forget it - I've got it working. It was an issue with binstubs in my testing environment. We're good!

lisabien

@lisabien

Resource metadata seems inconsistent. For example, EC2.resources.json gives us something like this for resources:

    "DhcpOptions": {
      "identifiers": [
        {
          "name": "Id",
          "memberName": "DhcpOptionsId"
        }
      ],

This is great for determining the underlying resource ID name. However CloudFormation.resources.json looks like this:

    "Stack": {
      "identifiers": [
        { "name": "Name" }
      ],

There's no memberName to reveal the underlying name (e.g. "StackName"). Which example is the standard, and what is the best way to determine the underlying resource ID name?

Trevor Rowe

@trevorrowe

The memberName is only populated when the identifier is echoed as part of the named “shape” from the API model. If the resource has no shape (additional properites) or if the property does not repeat the identifier, it has no shape “memberName”.

Trevor Rowe

@trevorrowe

@lisabien I took a look at the cloudformation resource model, and it appears that some of the resources, like Stack, were missing memberName traits where they should have been applied. See: aws/aws-sdk-ruby@e5d5eae

lisabien

@lisabien

Cool, thx! It appears that other resource files, like OpsWorks (and maybe others, but didn't check) have the same missing memberName issue. Would it be possible to get the resource files updated with memberName, too?

Trevor Rowe

@trevorrowe

Its possible for the services where the value is present in the shape, which is not always. If you know of any service missing them, please let me know.

lisabien

@lisabien

Will do. I'll go back and take a look. Thx!

Trevor Rowe

@trevorrowe

Out of curiosity, what are you consuming the files for?

lisabien

@lisabien

I'm working with @tve, our use case is a proxy that exposes a standardized json interface and also a restful interface. this will be used by a set of other services and we want to feed everything through the proxy for a variety of reasons.

Trevor Rowe

@trevorrowe

Sounds cool, keep the questions coming.

lisabien

@lisabien

Why does API metadata (e.g. EC2's DescribeKeyPairs) specify their parameter names in CamelCase, when the underlying call actually expects snake_case?

KeyNames vs. key_names

Trevor Rowe

@trevorrowe

EC2 is a special case. Every other AWS service uses a consistent case (upper or lower camel case typically). EC2 uses Upper camel case for most input values and lower camel case for most outputs. That said, there is still a desire to share the structures between input and output as their only difference is the serialized format of their member names.

To work around this, EC2 is not marked as a “query” protocol, instead it is marked as “ec2”. The made-up-by-the-sdk-team “ec2” protocol is defined a query with modifications. Most of the deltas are on how to marshal input. One of those being how to case them as they are sent, but other differences exist. Some of these are all lists are “flattened” on input (no .member prefix on list members) and there is an extra “queryName” that overrides the typical “locationName” (location name is inflected to UpperCamelCase, where queryName is taken as is).

I have some internal documentation I might be able to scrounge up that documents the differences.

Also, the Ruby SDK chooses to snake_case all input and outputs, reguardless of their protocol wire format, but I don’t think that is what you are referencing.

lisabien

@lisabien

Thank you. I'm sorry, I should have specified, yes we are seeing this with Ruby SDK.

Trevor Rowe

@trevorrowe

This is by-design, as Ruby methods are snake_case by convention.

lisabien

@lisabien

So, is Ruby SDK using the CamelCase metadata and converting to snake_case internally?

Trevor Rowe

@trevorrowe

There are two peices of information on every structure member, there is the chosen name of the references shape. I call this the member name. Then the actual reference can also provide a locationName. This is the name of the input/output value as it appears on the wire http request/response. The Ruby SDK will always use the snake_cased version of the member name. The locationName is an implementation detail for marshalling requests and un-marshalling responses.

lisabien

@lisabien

Cool. Thx!

lisabien

@lisabien

I believe in EC2.resources.json, that the "Volume" resource is missing the "Delete" action:
https://github.com/aws/aws-sdk-ruby/blob/master/aws-sdk-core/apis/EC2.resources.json#L1732-L1812
Should it have a "Delete" action defined for it?

Also, in EC2.resources.json, there is a "CreateInstances" action defined to map to "RunInstances". Would you consider adding a "Delete" action for the "Instance" resource to call "TerminateInstances"?

Trevor Rowe

@trevorrowe

The #create_instances call exists as primary mechanism for running a new instance. To delete an instance you can call #terminate on an instance or #terminate on a batch of instances:

ec2 = Aws::EC2::Resource.new

# single instance terminate
ec2.isntance(‘id’).terminate

# batch terminate
ec2.instances(filters: […]).terminate

You are correct, that Volume is missing a Delete action, an accidental omission. Please feel free to submit a PR.

lisabien

@lisabien

With regard to Instances, would you be opposed to exposing a "Delete" action on the Instance resource to also map to "TerminateInstances", in addition to the existing "Terminate" action that maps to "TerminateInstances"?

It just seems more CRUD-like and consistent that resources have both a CreateResource and a Delete, rather than instances having "CreateInstances" and "TerminateInstances".

For example, here: https://github.com/aws/aws-sdk-ruby/blob/master/aws-sdk-core/apis/EC2.resources.json#L14-L23

CreateInstances gets mapped to RunInstances. Just seems like having the complement for Delete would be nice. Something like this in the actions for an Instance resource:

        "Delete": {
          "request": {
            "operation": "TerminateInstances",
            "params": [
               { "target": "InstanceIds[0]", "source": "identifier", "name": "Id" }
            ]
          }
        }

If you're not opposed, I could submit a PR. If you are opposed, no worries, just thought we'd throw that idea out there. :)

Trevor Rowe

@trevorrowe

The reasoning behind not using delete was to disambiguate a call to terminate from a call to stop (StopInstances).

lisabien

@lisabien

Ok, thanks for the info!

Daniel Collis-Puro

@djcp

Gotta say, I was a little confused by the wait_until semantics. I have a PR in the works lighting up a few additional waiters, maybe I'll take a stab at clarifying the docs too.

I was expecting blocks I pass to wait_until to be run after the process was complete, and that's how I've seen other async waiters implemented.

I kinda like that pattern because it makes your intention clear: "run the code I'm passing you when what we're waiting for is done", rather than "block the main thread and then do the next thing)

And the confuse the matter, the blocks ran, but instantly. Anyway, now that I understand it it's fine, just not what I was expecting.

Daniel Collis-Puro

@djcp

The now deprecated Capybara.wait_until is probably what I was getting confused by, among other ruby interfaces.

Trevor Rowe

@trevorrowe

@djcp I agree the final interface ended up a bit akward. I would be willing to explore other interfaces. The functionality needs to remain consistent, but I’m open to deprecating the current method in favor of a better interface. The current interface would need to stick around for backwards compatability.

Josep M. Blanquer

@blanquer

@trevorrowe , re: "The reasoning behind not using delete was to disambiguate a call to terminate from a call to stop (StopInstances)" ... I'm not sure that is necessarily any more ambiguous than having a create and a start.

Where communities thrive

People

Repo info

Activity