Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Henri Meltaus
    @hmeltaus
    Currently, the template is processed before the parameters, and that's why parameters are not available in templates. It should be possible to change this ordering and expose resolved parameters to the template.
    jonatcorus
    @jonatcorus
    Hi, cool project. Is there a larger example of something more complete then the examples in takomo-examples?
    17 replies
    Henri Meltaus
    @hmeltaus
    This message was deleted
    jonatcorus
    @jonatcorus
    Any opinions on pulumi?
    1 reply
    Mika Fonsén
    @mfonsen

    I tried takomo@3.0.0-alpha.5. The first thing I bumped into was the following error:
    2021-01-21 11:25:44 +0200 [debug] - Init resolver 'my-resolver' for stack: '/events/eu-north-1/a.yml', parameter: 'myParameter' with properties:

    resolver: my-resolver
    functionName: my-function
    role: 'arn:aws:iam::00000:role/b-eu-north-1-c-role'
    region: eu-north-1
    immutable: false

    ERROR

    TypeError: Cannot read property 'keys' of undefined
    at Object.schema (/project/aws/resolvers/upload.js:6:19)
    at ResolverRegistry.initResolver (/packages/takomo/3.0.0-alpha.5/node_modules/@takomo/stacks-resolvers/dist/resolver-registry.js:33:41)
    at initializeResolver (/packages/takomo/3.0.0-alpha.5/node_modules/@takomo/stacks-context/dist/config/parameters.js:17:45)
    at Object.buildParameters (/packages/takomo/3.0.0-alpha.5/node_modules/@takomo/stacks-context/dist/config/parameters.js:28:36)
    at async Object.buildStack (/packages/takomo/3.0.0-alpha.5/node_modules/@takomo/stacks-context/dist/config/build-stack.js:39:24)
    at async Promise.all (index 2)
    at async processStackGroupConfigNode (/packages/takomo/3.0.0-alpha.5/node_modules/@takomo/stacks-context/dist/config/process-config-tree.js:93:29)
    at async Promise.all (index 0)
    at async processStackGroupConfigNode (/packages/takomo/3.0.0-alpha.5/node_modules/@takomo/stacks-context/dist/config/process-config-tree.js:96:5)
    at async Promise.all (index 0)

    OTHER INFO

    Your environment:
    OS: darwin
    Node version: v14.15.4
    Takomo version: 3.0.0-alpha.5

    Get support:
    Docs: https://takomo.io
    Bugs: https://github.com/takomo-io/takomo/issues

    Related code:
    module.exports = {
    name: "my-resolver",
    schema: (joi, schema) => {
    return schema.keys({
    functionName: joi.string().required(),
    region: joi.string().required(),
    role: joi.string().required(),
    })
    },
    init: async props => {
    ...

    No issues with takomo@2.12.0

    7 replies
    Mika Fonsén
    @mfonsen
    Bumped into a problem when reviewing stack changes which deletes an Elastic Container Registry. Happened on 3.0.1 and did not try with the latest version so I did not create a issue. Worked fine without reviewing the changes:
    2021-03-17 08:42:51 +0200 [debug] - /app/eu-north-1/stack.yml/eu-north-1 - Begin step 'review-change-set'
    
    Review deployment plan for a stack:
    -----------------------------------
    A stack deployment plan has been created and is shown below.
    
      stack path:                    /app/eu-north-1/stack.yml/eu-north-1
      stack name:                    stack
      stack region:                  eu-north-1
      operation:                     UPDATE
    
    Parameters:
    
      + ParameterA:          (parameter will be created)
          current value:             <undefined>
          new value:                 valueA
    
      + ParameterB:      (parameter will be created)
          current value:             <undefined>
          new value:                 valueB
    
      - ParameterC:       (parameter will be removed)
          current value:             valueC
          new value:                 <undefined>
    2021-03-17 08:42:51 +0200 [error] - /app/eu-north-1/stack.yml/eu-north-1 - An error occurred Error: Unsupported parameter operation: 'delete'
        at /Users/user/projects/xmevents/node_modules/@takomo/cli-io/dist/stacks/deploy-stacks/parameters.js:134:23
        at Array.map (<anonymous>)
        at Object.printParameters (/Users/user/projects/xmevents/node_modules/@takomo/cli-io/dist/stacks/deploy-stacks/parameters.js:125:10)
        at Object.confirmStackDeploy (/Users/user/projects/xmevents/node_modules/@takomo/cli-io/dist/stacks/deploy-stacks/deploy-stacks-io.js:160:48)
        at reviewChangeSet (/Users/user/projects/xmevents/node_modules/@takomo/stacks-commands/dist/stacks/deploy/steps/review-change-set.js:13:29)
        at /Users/user/projects/xmevents/node_modules/@takomo/stacks-commands/dist/stacks/deploy/transitions.js:60:22
        at executeStep (/Users/user/projects/xmevents/node_modules/@takomo/stacks-commands/dist/stacks/common/steps.js:30:22)
        at Object.executeSteps (/Users/user/projects/xmevents/node_modules/@takomo/stacks-commands/dist/stacks/common/steps.js:56:24)
        at processTicksAndRejections (internal/process/task_queues.js:93:5)
        at async Object.executeDeployContext (/Users/user/projects/xmevents/node_modules/@takomo/stacks-commands/dist/stacks/deploy/execute-deploy-context.js:90:27)
    7 replies
    Mika Fonsén
    @mfonsen
    Thank you for the latest releases. Logs are no more filled with throttling. Also the possibility to relay aws credentials to hooks is nice. So is the ability to refer to hook output from a resolver.
    Some first impressions on these: It would be nice if AWS region would also be included with the credentials but that is easy to implement in a stack config. The output reading resolver was a bit surprising as it reads all output. My expectation was that it would only read last line of stdout, not all lines or stderr. But also this was easy to get around.
    Henri Meltaus
    @hmeltaus
    Thanks for the feedback! I'll add a new option "exposeStackRegion" to control if stack's region should be exposed to command hook, would that work for you?
    Mika Fonsén
    @mfonsen
    That would be great. But this is by no means critical. It is just a thing you learn when you try the feature for the first time.
    Henri Meltaus
    @hmeltaus
    Yeah but it's easy to implement and useful feature so I'll do it like that.
    You're right that getting all output from a command hook isn't always what is needed. One option would be to add an option to specify that only the last line should be collected. There could also be an option to specify to collect everything that is printed after some marker string until an optional end marker is found.
    for example, collect everything after line "--- start ---" until output ends or end marker "--- end ---" is encountered
    Mika Fonsén
    @mfonsen
    I was also wondering that does Cloudformation support multiline parametets but maybe there’s some other usecase that benefits from multiline output
    My use case was to prepare a container image and return it’s ID in ECR. I had to forward output from docker build to a file to avoid it being included. A marker or last line would work well
    Mika Fonsén
    @mfonsen
    Took latest Takomo into use. Used region expose and last line parameter features of the command resolver. They work very well. Thank you
    Henri Meltaus
    @hmeltaus
    Excellent!
    jonatcorus
    @jonatcorus
    Wow you've been quite productive in the last couple months. I might be blind, but I do not see the AWS organizations feature in the docs, was it removed?
    Actually I see it in the code, so I guess the docs are missing is all...
    In case you haven't seen this project https://github.com/org-formation/org-formation-cli
    jonatcorus
    @jonatcorus
    How does takomo handle moving OUs and accounts around and can you rename accounts? I played around with using terraform and changing the name of an account makes TF want to delete/recreate the account...
    Henri Meltaus
    @hmeltaus
    Hi, I haven't seen org-formation-cli. Looks interesting, I'll definitely have to check it out. Have you used it yourself?
    I moved the docs to gitbook some time ago and haven't included org management features there yet. I should take the time to finalize the docs.
    Henri Meltaus
    @hmeltaus
    Takomo lets you move OUs and accounts around. You can also rename accounts but that you have to do manually from AWS console. I believe there's no APIs for that.
    Henri Meltaus
    @hmeltaus
    You can check org management docs from the old site https://takomo.io/docs/release/v3-4-0/docs/organizations/introduction
    Mika Fonsén
    @mfonsen
    Does Takomo expose assumed roles or base credentials? With Takomo 3.11.1 I tried commandRole for the first time but the command resolver I have fails. By requesting caller identity from the command resolver it seems that I got the base credentials, not the temporary credentials from commandRole. Not 100% sure I did everything right, but I have checked the setup couple times already. The stacks that don't have such hooks/resolvers get deployed.
    Mika Fonsén
    @mfonsen
    Oh, I got two things mixed up there. I checked the identity from a custom resolver that does:
    const aws = require("aws-sdk")
    const sts = new aws.STS()
    ...
    const credentialManager =
    await input.ctx.credentialManager.createCredentialManagerForRole(
    props.role,
    )
    const credentials = await credentialManager.getCredentials()
    console.log(
    credentials,
    await sts.getCallerIdentity().promise(),
    )
    command hook got the correct role
    Maybe there's a better way of requesting credentials from a custom resolver?
    Mika Fonsén
    @mfonsen
    My goal is to make Takomo run all parts of the deployment including build, data seeding and others through one command. This should speed up the deployment. Currently there are scripts commanding Takomo but that feels unnecessary
    Mika Fonsén
    @mfonsen
    Maybe the iamRoles array the resolver can ask would be a clean solution?
    Henri Meltaus
    @hmeltaus
    Hi, so are you implementing a custom resolver and wonder how to use credentials there?
    Mika Fonsén
    @mfonsen
    Yes. I would like to use the assumed role the stack has (commandRole) to assume another role to push files to a bucket.
    The custom resolver works without the commandRole if the base credentials given to Takomo are correct
    Henri Meltaus
    @hmeltaus
    custom resolvers's resolve function gets one argument of type ResolverInput. The current stack is in stack property and from it you can get credential manager that has credentials that are bound to the stack (base credentials or credentials from commandRole).
    So, in resolve function you can get stack credentials like so:
        resolve: async (input) => {
    
          const credentials = await input.stack.credentialManager.getCredentials()
    
          // ...rest of the resolve function
        }
    Mika Fonsén
    @mfonsen
    Thank you! I will try that
    Henri Meltaus
    @hmeltaus
    Built-in command resolver exposes the current stack's credentials if exposeStackCredentials is set to true. You can find the code here https://github.com/takomo-io/takomo/blob/ad91d05b3a3d5b9cb2fa86909d4ca501d1721ee9/packages/stacks-resolvers/src/cmd-resolver.ts#L53
    Mika Fonsén
    @mfonsen
    Thank you. That worked great. I also got the resolver working that assumes subsequent roles:
            const credentialManager = await input.stack.credentialManager.createCredentialManagerForRole(
              props.role,
            )
            const credentials = await credentialManager.getCredentials()
    Mika Fonsén
    @mfonsen
    Another challenge with the commandRole approach is MFA. I have a setup similar to the documentation (https://docs.takomo.io/configuration/aws-credentials#assuming-roles-that-require-mfa). Instead of starting Takomo for each role separately with --profile switch I would like to run all stacks in one go. This would require Takomo or some earlier tool to ask for the MFA once. Is there a way to get this use case working?
    My first idea was to use profile (as account-a-admin in the docs) in stack group configuration as in Sceptre but that option does not seem to be supported. I could then let Takomo to ask for MFA once or write temporary credentials to ~/.aws/credentials using a tool such as awsume -o to avoid interactive prompts.
    I then tried what happens if I give just the main account (manager in the docs) using the profile switch. This fails as Takomo will not ask MFA. Probably there's no way it could as there's no way of telling what mfa_serial should be used. Maybe there could be an option to specify the MFA serial. This seems to be supported in AWS SDK/STS.
    Henri Meltaus
    @hmeltaus
    Good to hear that you were able to use credentials with resolvers.
    Henri Meltaus
    @hmeltaus
    I'll check how to achieve your use case with MFA and get back to you later
    Mika Fonsén
    @mfonsen
    Maybe if I'll give takomo a role that was already assumed and thus the MFA check had been completed. Using that role I Takomo would be able to assume other roles as they would not need to check for MFA anymore. There's one extra level role in the chain but that probably works.
    Mika Fonsén
    @mfonsen

    Got it working with that one additional assume role layer.

    This worked: awsume manager + takomo ... (no profile). Awsume takes care of the MFA. Takomo assumes roles as defined with multiple commandRoles. These do not require further MFAs.

    This did not work: takomo ... --profile manager --> ProcessCredentialsProviderFailure: Profile manager did not include credential process. Takomo does ask MFA first before failing. Aws cli works when an target profile (account-a-admin) was defined.

    The complexity of this solution goes into IAM and the developer experience is still good so I guess I'll go with this approach.

    Henri Meltaus
    @hmeltaus
    hmm, I'm just about to do some testing myself. I'll see if there's something I can do to make it simpler.
    Mika Fonsén
    @mfonsen
    The error I mentioned on Friday was probably about something different. Takomo started working with mfa after I removed unused profiles from ~/.aws/config
    Mika Fonsén
    @mfonsen
    I had defined credential_process set in ~/.aws/config for unused profiles. These seemed to confuse Takomo (or aws-sdk). The error message ("Profile manager did not include credential process") would hint this was the case. Anyway now it works great.
    Henri Meltaus
    @hmeltaus
    ok, the way AWS SDK handles credentials is quite complex. I should probably do some more digging and at least document the most common use cases and problems
    Mika Fonsén
    @mfonsen
    That's true. Personally I wouldn't regard this as a problem in Takomo so I didn't verify this finding or try to replicate this with trivial aws-sdk example.