Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
    Good morning... when I run a copilot app ls there are (2) apps listed. The cloudformation stacks for them were manually deleted. How do I cleanup this list?
    3 replies
    is there a --force switch or something that can force it to remove the apps regardless?
    Kevin O'Neill
    what's the recommended way to spin up a version of an application in a second account ... in my case I have the co-pilot directory with my services defined and I want to install it in an entirely new account set
    Austin Ely
    copilot env init β€”profile myotheraccountcreds! You can create a new env in any account and region to which you can deploy all your services and share a domain name as long as you dont mind the new account being associated with the previous one via a cloudformation stackset.
    Kevin O'Neill

    Hello, n00b question, I'm trying to use copilot to deploy a simple container and call a python script in that container, but couldn't debug why it won't work. Any tips? I have a hunch I'm not using the Backend Service type properly

    I did copilot init, created "Backend Service", and ran copilot task run --command "python manage.py runscript send_deal_email --script-args debug save"

    and it finishes but I didn't get an email the script was supposed to send.

    02c055ef67f5: Layer already exists
    latest: digest: sha256:94ba6ef7a0f386af628039e41ba5fe113caf1453ecaed6cb6a69f4bcbdf804fe size: 3049
    βœ” Proposing infrastructure changes for stack task-flueth
    - Updating the infrastructure for stack task-flueth      [update complete]  [11.6s]
      - An ECS task definition to run your container on ECS  [create complete]  [3.5s]
    βœ” Task flueth is running.
    The public IP associated with the task is:
    - (for 78f286ba)
    (django) markx@Marks-MacBook-Pro ~/flueth (main)> copilot svc status                                                                                                                                                                                                    01:27:12
    Found only one deployed service flueth-scripts in environment test
    Service Status
      ACTIVE 0 / 1 running tasks (1 pending)
    Last Deployment
      Updated At         15 seconds ago
      Task Definition    arn:aws:ecs:us-west-2:915518705299:task-definition/flueth-scripts-test-flueth-scripts:2
    Task Status
      ID                Image Digest        Last Status         Started At          Stopped At          Capacity Provider    Health Status
      --                ------------        -----------         ----------          ----------          -----------------    -------------
      c307a0f          -                   PENDING             -                   -                   -                    UNKNOWN
      Name              Condition           Last Updated        Health
      ----              ---------           ------------        ------
    I also tried copilot svc exec to start bash in the container to debug but couldnt figure out how to do it for a Backend service which doesnt have a running task
    (django) markx@Marks-MacBook-Pro ~/flueth (main)> copilot svc exec                                                                                                                                                                                                      01:28:44
    Found only one deployed service flueth-scripts in environment test
    ✘ found no running task for service flueth-scripts in environment test
    Austin Ely
    Hey @Ycationtime_twitter, it seems like your container has failed to start. If you just need to run a one-off task, i'd recommend using copilot task run --dockerfile ./path/to/your/Dockerfile --command "python manage.py runscript send_deal_email --script-args debug save". This will deploy a one-off task using an image built from your dockerfile and the command overridden.
    Austin Ely
    if you need your container running as a service at all times, though, then svc init/svc deploy is what you need. once all the containers are running, you can do copilot svc exec to execute your email script.
    Can you post the Dockerfile and manifest.yml you used to deploy this, and post the output of copilot svc deploy?
    Sam Jones

    Hi all! I have a service with an addon containing a AWS::ElastiCache::CacheCluster resource. I would like to change this to an AWS::ElastiCache::ReplicationGroup. When I run copilot svc deploy I get the following

    deploy service: wait for creation of change set copilot-xxx for stack xxx: ResourceNotReady: failed waiting for successful resource state: Update of resource type is not permitted. The new template modifies resource type of the following resources: [Redis]: Health check interval must be greater than the timeout

    I would have expected Cloud Formation to delete the CacheCluster and create a new ReplicationGroup. I there a workaround for this? Do I need to do something like delete the Environment or Service and recreate it?

    4 replies
    Kevin O'Neill
    so I have a copilot directory with service manifests defined ... how do I add them to the current application ?
    7 replies
    Hi all, quick question: How can I maintain multiple copilot apps (and associated pipelines) in the same repo? E.g. I have one set of services (ie core-app) running the prod system and one set of services (ie admin-app) that run admin services (with higher update frequency than the core-app). I would like to deploy these apps independently from each other. There is no need to deploy core-app if the admin-app is updated. Both these applications share the same repo. Is that related to workspaces? Maybe I missed the docs on that? Thanks for any pointers!
    3 replies

    Hi all -- I'm having an issue getting my first Copilot for a Bitbucket repository to work... I'm confused by what the value should by for source.properties.repository in the pipeline.yml file. An example for what is generated by the CLI is: https://git@bitbucket.org:<account-name>@bitbucket.org/git@bitbucket.org:<account-name>/<repository-name>

    This doesn't seem right. When I manually edit the pipeline in the console and change the repository value to <account-name>/<repository-name>, the SRC download step works, but then the build fails. Anybody else seen this issue?

    11 replies
    Brian Rogers
    Hi all, I'm trying to mob with some developers. Where is the clearest possible instructions on making sure we init correctly and see the same environment?
    Brian Rogers
    Basically I did a bunch of work. Committed it to my repository. Now a new developer pulled down my work and wants to resume, but cannot do so.

    After much trial and error we found this to be copilot app init

    Man that could be A LOT CLEARER in your documentation.

    It's so critical I think y'all should surround it in red at the top of a page somewhere. It took way too much to figure that out.

    2 replies
    Brian Rogers
    So now I'm having an issue with Copilot Pipeline's. Particularly the Post-Build step is just bombing with no clear information.
    8 replies
    Brian Rogers


    So I just setup my first pipeline of my APP with 3 services. I just noticed that since they're all in the same repo, they all get deployed even if only the code/artifacts changed for one service.

    How would I go about enabling the ability to deploy separately? Would I just split this across three repos and init it into the same app?

    4 replies
    Anyone know if there is a clean way to run a command on all tasks in a service? I just need to manually clear some caches from time to time. If not I guess I'll just write a script to do it using the --task-id flag.
    2 replies
    Andre Gallo
    has anyone used used an ElasticCache Redis cluster as an add-on in their Copilot projects? What would be a good cloudformation template I could leverage with best practices baked in, particularly regarding VPC. Thanks!
    2 replies
    Brian Rogers

    Hi folks - I have some frontend devs looking at using AWS Amplify for mobile and front-end components - I realized that it does a lot of the same things as copilot, but it's built in Node.js rather than Typescript.

    Request: As a DevOps engineer, I wish I had a better way to compare these tools so I can better advise these developers on what they should be using. I really like Copilot, but if Amplify is more useful for what we're doing, then we should probably switch sooner rather than later. Googling "AWS Copilot vs AWS Amplify" doesn't really return much useful information.

    From the developer perspective, it looks like Amplify does more and covers more of our use case. I just wish I had a more cut and dry way of comparing these tools given the existing momentum on Copilot.

    Request 2: As a DevOps engineer, some guides about how to share resources between CoPilot/Amplify/SAM could help alleviate the need for Request 1. I don't want to have to care about the tools my developers decide to use. If I know how to comfortably share resources between the different verticals/tools then it won't matter what they decide to use. Conceptually, it makes sense to simply use parameter store to surface stack export values to other verticals, but perhaps there's a simpler way.

    I can totally open these up as feature requests somewhere, but the Copilot repo might not be the right place for this.

    1 reply
    Ronique Ricketts
    @brogers-propstream I’ve done this combination before but it’s hectic as I was using amplify (react) and hitting Copilot alb with a node server hosting a Frontend facing Apollo server connected to a dynamodb (which was an antipattern) I wish we could just run an amplify init and specify that we’ll be connecting to copilot containers and it spins up our amplify frontend and set up that container with our frontend in amplify but has that connection to copilot container as backend instead of using lambdas. 😏 Would be great to also configure amplify’s auth on the server side in copilot.
    That simple withAutjenticator() that amplify uses would do well with copilot when we are setting up vue, react or angular apps. 😏
    Nicolas Barrera

    Hello every one.

    I 've been looking through the docs for a way to customize/add an IAM role to the task definition that copilot creates for me.

    I can do that afterwards with aws-cli or at the aws web, but then that turns into a problem where whenever I want to clean the copilot app I need to hunt and delete those first or the copilot clean won't finish.

    Do anyone knows if attaching IAM roles is possible?

    thanks in advance!

    2 replies

    Hello everyone. I was experimenting with addons, beginning with copilot storage init for an S3 bucket. I noticed that the addon resources are provisioned "per-service". I was particularly looking for a way to provision one S3 bucket and share it between two different services. I ended up provisioning it outside of copilot using Terraform, and then adding the necessary policies to Copilot's generated task execution role, which was a little painful. I have two questions:

    1) Do you have a preferred solution for sharing addon resources between multiple services?
    2) Would you consider exposing more of the resources in the "service" CloudFormation stacks as outputs (particularly the task execution ARN) so it's easier to hook into them with Terraform or other external tools?

    1 reply
    Sam Jones
    Hi again! Is it possible to access CodeBuild exported variables? I have dynamic values that I would like to pass to later stages.
    17 replies

    Hi Everyone,

    I am trying to deploy WordPress using https://github.com/bvtujo/copilot-wordpress/ and I am getting "/bin/sh: 1: /opt/copilot/scripts/startup.sh: not found" error.

    Can you please help?

    3 replies
    Paul Murage
    Hi Everyone, (Curious question)
    Why are copilot scheduled Jobs setup using Step Functions state machines with ECS Fargate as the target for EventBridge /Cloudwatch Events rather than just use ECS Fargate without Step Functions state machines?
    3 replies
    Hi All,
    Copilot 1.7.1 creates cloudformation resource for post deployment tests as BuildTestCommands<ENV>
    When environment name contains '-' in its name, resulting resource name ends up being something like BuildTestCommandsandbox-dev which conflicts with cloudformation resource id naming pattern (A-Za-z0-9).
    Error ValidationError: Template format error: Resource name BuildTestCommandssandbox-dev is non alphanumeric. is thrown in our case.
    Any ideas to overcome this other than renaming our ENVs?
    Should copilot environments names be alpha numeric as well?
    4 replies
    Brian Rogers

    Anyone else seeing an AWS outage at the moment? I cannot auth/assume-role at the moment and it definitely isn't a convenient time for that.

    All my colleagues report similar issues and DownDetector shows a hockey-stick in reports.

    1 reply
    Error: ECR repository not found for application api-prod in region us-west-1 and account 219450226639 anyone familiar with this error when trying to deploy? my pipelines were working for more than a year I think and just got this error. Note that this issue is when deploying, I am sure that the ECR repo actually exists.
    5 replies
    Sam Jones

    Hi again! Is it possible to get a reference to the load balancer's ARN within a CloudFormation template used to create additional resources. I have a need to create WAF Web ACLs. The CloudFormation template requires an ARN to the associated resource (i.e. the load balancer). Here is an example:

        Type: 'AWS::WAFv2::WebACLAssociation'
          WebACLArn: ExampleARNForWebACL
          ResourceArn: ExampleARNForRegionalResource

    Is this possible with Copilot?

    7 replies
    Efe Karakus

    Hi folks! Copilot v1.8.0 is out :tada: ! https://github.com/aws/copilot-cli/releases/tag/v1.8.0

    There are lots of features and bug fixes in this release, if you have any feedback please let us know! πŸ˜ƒ

    Nayan Hajratwala

    Hi all -- i'm trying to deploy hydra using copilot, so i need a service and DB. Using the following commands:

    copilot app init my-app
    copilot svc init \
        --name auth-server \
        --svc-type "Load Balanced Web Service" \
        --image oryd/hydra:v1.10.2 \
        --port 4444
    copilot storage init \
        --name my-app-cluster \
        --storage-type Aurora \
        --workload auth-server \
        --engine PostgreSQL \
        --initial-db my_db
    copilot env init --name test
    copilot deploy --env test --app my_app

    during deployment is where i run into issues. The hydra server requires a DSN environment variable, which I put in the manifest.yml as so:

      DSN : postgres://<username>:<password>@<hostname>:5432/hydra?sslmode=disable

    but I'm not sure how to get the username/password/hostname values here. I see that they're supposed to be in the MYAPPCLUSTER_SECRET environment variable as a json object, but how can i extract & use them

    Any suggestions?

    Austin Ely
    @nhajratw hey Nayan! We dont have a great way to do this except for suggesting that you wrap your hydra service in a docker entrypoint script that parses the environment variables from the secret. Would something like this work for you? https://github.com/bvtujo/copilot-wordpress/blob/main/startup.sh
    Nayan Hajratwala
    yes, that will probably work. Will try it out in the AM. Thanks!
    Nayan Hajratwala

    @bvtujo well, in theory that works, but i'm having trouble getting it to work.

    The service is this one: https://github.com/ory/hydra/blob/v1.9.0/.docker/Dockerfile-alpine

    I try to wrap it with a Dockerfile like this:

    FROM oryd/hydra:v1.10.2
    RUN apk -U --no-cache add jq
    ENTRYPOINT ["./launch-hydra.sh"] # <-- contains the secret parsing and DSN setting

    but it doesn't even build, complaining that it is unable to find user 1000: invalid argument

    19 replies
    Has anyone run into this issue? I only have one copilot application with one service and two environments. Test environment has 1 task instances, Prod environment has 2 task instances. Google is not helpful and I don't see such low limits here https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-quotas.html
    service $SERVICE_NAME was unable to place a task. Reason: You've reached the limit on the number of                                   
            tasks you can run concurrently. For more information, see the Troubles                                  
            hooting section of the Amazon ECS Developer Guide.
    3 replies
    Hi Everyone. Really liking the new tool!
    Quick question, can I add an existing RDS addon to a new service? I have one service providing an API using a MySQL DB. I need to grant access to the same DB instance on another service within the same application. The new services does not have access to the DB by default, unless I add or amend security groups after creating the new service. Thanks.
    2 replies
    Miklos Szots
    Hi all,
    Great tool! One question I have -> when can I expect the next patch release to be released? There's an important bug fix with regards to the environment section not getting picked up for job deploys. Wouldn't wanna look into alternative solutions if it's just around the corner :)
    2 replies
    Nicolas Barrera

    Hi there...
    I need to run database migration (on mongo atlas) before deploying. I thought of doing that from the CI pipeline, but... as it's on Atlas I need to know the public IP of the incoming connection to whitelist it.

    At the app's vpc I got a private subnet with a NAT gw for the main app to be whitelisted into Mongo Atlas.

    Is it possible to run a one-off task specifically to run the migrations? I would like to run it on that same private subnet so it's already whitelisted

    I thought copilot task run was what I needed, tried it.. but didn't work

    % copilot task run  
    In which application would you like to run this task? my-app
    In which environment would you like to run this task? test
    ✘ provision resources for task my-app: read template for task stack: read template task/cf.yml: stat /Users/xxx/git/my-app/task/cf.yml: no such file or directory

    my-app is a Load Balanced Web Service and was created by importing it into an existing vpc. (also, I 'm using copilot 1.8.0)

    Do you think it would be possible to achieve this with tasks? Why do you think the command failed?

    thanks in advance

    2 replies
    Camilo Santana

    i have an environment deployed with copilot 1.4.0 ... i ran copilot svc deploy ... from a different machine using copilot 1.8.0. it expectedly went to upgrade infrastructure. However, I've added a certificate to the load balancer and a few custom rules (to accommodate a friendly URL instead of the auto-generated ones).

    i'm currently in an UPDATE_ROLLBACK_FAILED state on the cloudformation stacks. I've removed the custom certificate and tried to continue the rollback but i get the following error:

    Resource handler returned message: 
    "Uploaded file must be a non-empty zip (Service: Lambda, Status Code: 400,
    Request ID: ##########, Extended Request ID: null)"
    (RequestToken: ##########, HandlerErrorCode: InvalidRequest)

    any general idea on how to clear this up and move forward so we can deploy again?

    15 replies
    Question - if I change environment scaling settings in my mainfiest (https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/) on an app that has an existing pipeline and an existing deployment, what do I need to do to get them to apply? Should the pipeline automatically apply them?
    8 replies
    Sam Jones

    I am trying to use Copilot to deploy a CloudFront distribution that uses my application's load balancer as an origin. I am using the load balancer's domain name to specify the origin within the CF template:

              - DomainName:
                    !Sub "${App}-${Env}-PublicLoadBalancerDNS"

    The distribution deploys successfully, but when I try to access a resource that exists on the origin, I get a 502 error. According to the CloudFront docs this is usually because of SSL errors.

    I think the problem is that the certificate created by Copilot is for my application's domain (environment.application.domain.com), and CloudFront sees that as a host name mismatch. I can think to define the origin as ${Env}.${App}.domain.com in the template. But is there a better way to do this?

    3 replies
    Fred Smith
    heh, even with 100k requests/sec, I can't get it to scale past 3 containers :-/ https://files.derf.us/auberon-2021-06-24-5.22.35-PM-oot43E0V.png
    2 replies
    Fred Smith
    Nah, I can't drive enough load without bring ridiculous
    It's fine, though, I can see the alarms in 'svc status' so I know they're being applied. Thanks for your help