by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 05:30
    sujata-sharma labeled #6145
  • 05:30
    sujata-sharma opened #6145
  • 00:39
    marcoceppi commented #6142
  • 00:38
    marcoceppi commented #6142
  • Sep 21 23:07
    jcttrll opened #6144
  • Sep 21 23:07
    jcttrll labeled #6144
  • Sep 21 22:09
    tomjones2a opened #6143
  • Sep 21 21:00
    marcoceppi edited #6142
  • Sep 21 20:17
    marcoceppi synchronize #6142
  • Sep 21 20:17
    marcoceppi review_requested #6142
  • Sep 21 20:17
    marcoceppi opened #6142
  • Sep 21 20:05
  • Sep 21 20:05
  • Sep 21 18:21
    williambrady starred cloud-custodian/cloud-custodian
  • Sep 21 17:14
    howbazaar synchronize #6133
  • Sep 21 15:32
    nomecks commented #5326
  • Sep 21 14:29

    kapilt on master

    aws - fix phd mode error for re… (compare)

  • Sep 21 14:29
    kapilt closed #6141
  • Sep 21 14:29
    kapilt closed #6135
  • Sep 21 13:58

    kapilt on gh-pages

    Updated generated Sphinx docume… (compare)

Allison D
@allisonis
Any thoughts on where to look for how subnet resources are iterated over per region?
DigeratiDad
@digeratidad
Has anyone seen this issue before with ELB
[WARNING]    2020-05-14T21:43:54.197Z    da8bfff2-799d-453c-8e82-313b6edd3046    event ids not resolved: ['arn:aws:elasticloadbalancing:us-west-2:########:loadbalancer/app/cctagtesting/0ae4bc40b82b22c9'] error:An error occurred (ValidationError) when calling the DescribeLoadBalancers operation: LoadBalancer name cannot be longer than 32 characters
Here is my policy:
policies:
  - name: elb-classic-auto-tag-creator
    resource: elb
    mode:
      type: cloudtrail
      role: arn:aws:iam::{account_id}:role/CloudEng-Cloud_Custodian
      events:
        - source: elasticloadbalancing.amazonaws.com
          event: CreateLoadBalancer
          ids: "responseElements.loadBalancers[].loadBalancerArn"
    filters:
      - "tag:Creator": absent
    actions:
      - type: auto-tag-user
        tag: Creator
29 replies
Ravindra babu
@Ravindrababu99_twitter
@jtroberts83 Hi, old custodian mail alerts also getting delivered after ugrading the mailer to 3.7 from 2.7 old version with a new msg-template, I've been using same sqs-mailer, so can you help me understand the scenario why am I getting older msgs getting delivered now and how can I stop that.
6 replies
Ravindra babu
@Ravindrababu99_twitter
@jtroberts83 getting below error in cloudwatch logs, assuming because of this mails are getting re-delivered.
error: An error occurred (InvalidParameterValue) when calling the SendRawEmail operation: Missing final '@domain'
1 reply
satvan23
@satvan23

@satvan23 It could look something like this:

- name: ec2-using-unapproved-ami
  resource: ec2
  filters:
      - type: value
        key: "ImageId"
        op: not-in
        value_from:
           url: s3://yours3bucket/ApprovedAMIs.csv
           format: csv2dict

@jtroberts83 Thanks !

1 reply
Mike
@mikejgray
Hey folks, I'm becoming enamored of this c7n-org report functionality...can it read from S3 logs directly? They're in a different tree structure than local logs so wondering if it does that
2 replies
amolkk1980
@amolkk1980_twitter
Hi Guys.. I have a question. Can Cloud Custodian be configured/supported with Oracle OCI?
2 replies
Alicia Steen
@aliciasteen
Hi, does the flag --region all work, I have tried to use it but come across the error botocore.exceptions.NoRegionError: You must specify a region.
10 replies
Ravindra babu
@Ravindrababu99_twitter

@kapilt , while three different custodian policies triggered on cloudtrail (CreateBucket) event to add tags, getting below error in one of the policies while defining mark-for-op tag. Due to this, mailer is not sending bucket information in the alert...Can you please help how to overcome this.

"errorMessage": "A conflicting conditional operation is currently in progress against this resource. Please try again."

7 replies
KVInventoR
@KVInventoR

How does or operator work?
Hi all, I tried run CC in cloudtrail mode and create a simple filter, I would like to filter any ec2 from elastimapreduce, datapipeline or if ec2 has specific tag
But I got many reports with ec2 which were part of emr cluster.
I checked the events in cloudwatch logs for lambda: ec2_tags_validate_mode_cloudtrail

        "eventTime": "2020-05-18T19:00:34Z",
        "eventSource": "ec2.amazonaws.com",
        "eventName": "RunInstances",
        "awsRegion": "us-west-2",
        "sourceIPAddress": "elasticmapreduce.amazonaws.com",
        "userAgent": "elasticmapreduce.amazonaws.com",

for it looks like something wrong with my filter or CC just ignored filters section.

policies:
  - name: ec2_tags_validate_mode_cloudtrail
    resource: ec2
    description: |
      EC2 validate tags on new started instances.
    mode:
      type: cloudtrail
      role: *lambda_role
      events:
        - RunInstances
      tags:
        Name: *lambda_name
        Team: "TEAMNAME"
    filters:
      - or:
        - "tag:TestTag": absent
        - type: event
          key: sourceIPAddress
          op: ne
          value: "elasticmapreduce.amazonaws.com"
        - type: event
          key: sourceIPAddress
          op: ne
          value: "datapipeline.amazonaws.com"

    actions:
      - <<: *notify-var
        subject: "CControl - EC2:TAGS validation in cloudtrail mode"

Could you please help to find, what can be wrong?

7 replies
aakshaik2
@aakifshaikh
what filter should i use to check on unencrypted snapshots: type: value key: Snapshots.Encrypted value: false
4 replies
Dan Parsons
@danparsons
Who here actually uses c7n-org? Care to answer a few questions for me? Like how am I supposed to give it a AWS_SESSION_TOKEN env var? To my knowledge, you only get a AWS_SESSION_TOKEN when you do a sts:AssumeRole, however c7n-org itself is going to be doing sts:AssumeRole to get into all my accounts.... so I can't feed it creds to start with (to then access all accounts) because all I have are the access key and secret key for the top level c7n account
46 replies
Eri Wahyudi
@ewahyudi_gitlab
Greetings all ! Cloud-Custodian newby here; I'm starting out with simple script to apply simple tag key/value pair. I'm trying to add tag value in json format as follows:
{"tz": "US/Eastern", "sched": "2-6:0800-1730"}, so far I have added single quotes like so -
  • type: tag
    key: EC2_Start_Stop
    value: '{"tz": "US/Eastern", "schedule": "2-6:0800-1730"}'
    When I ran the "custodian validate script.yml" command, this returned INFO Configuration valid but when I tried running, it failed. Does anyone have any insight to what is it in the syntax that may be wrong?
14 replies
tomarv2
@tomarv2
I am trying to find unused VPC, as VPC can have many resources, any suggestion?, look for unused ENI and then find VPC
8 replies
Jamison Roberts
@jtroberts83
This message was deleted
3 replies
Jamison Roberts
@jtroberts83
Question - When should I expect the Lambda policy functions 'LastModified' date to change? Should that be updated each time I deploy the lambda based policy or only when something changes? I see most of my custodian lambda functions LastModified date gets updated daily but then some lambda policy functions are showing as last modified several days or months ago. Is that to be expected??
4 replies
Barak Schoster Goihman
@schosterbarak
Question - I want to use c7n-org i see it can take role as a parameter for cross-account access. can it take external id too?
1 reply
Kapil Thangavelu
@kapilt
fwiw, there's a proposal up for being able to use custodian policies on iaac assets (terraform, cfn) in DevOps pipelines, comments welcome cloud-custodian/cloud-custodian#5782
1 reply
Amarankit Srivastava
@amarankit-srivastava
@all, I am struggling with c7n_mailer setup... have python36, have c7n and then installed successfully c7n-mailer (i.e. without any error via https://cloudcustodian.io/docs/tools/c7n-mailer.html#developer-install-os-x-el-capitan)... but when I am trying any command (even -h/help), its giving error ..................... from c7n_mailer import deploy, utils
File "/root/cloud-custodian/tools/c7n_mailer/c7n_mailer/deploy.py", line 19, in <module>
from c7n.mu import (
ModuleNotFoundError: No module named 'c7n'......ANY CLUES WHERE I AM GOING WRONG...
22 replies
Barak Schoster Goihman
@schosterbarak
i'm trying to connect c7n to cloudtrail but i didn't find where do i point c7n to the cloudtrail bucket. is there a good tutorial for it?
Karl de Castro Fonseca
@KarlCF

I'm trying to tag (action) some container instances using the resource aws.ecs-container-instance without filters. Using both the new arn model and the old one.
I'm getting the following error in both situations. I got no error just listing the container instances, just when i'm trying to tag (action) the resource.

2020-05-19 16:53:59,740: custodian.policy:INFO policy:ecs-verify-container-instances-ami resource:aws.ecs-container-instance region:sa-east-1 count:1time:0.37
2020-05-19 16:53:59,746: custodian.actions:ERROR Exception with tags: [{'Key': 'teste', 'Value': 'instance1'}] 'containerInstance'
2020-05-19 16:53:59,747: custodian.output:ERROR Error while executing policy
Traceback (most recent call last):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/policy.py", line 323, in run
results = a.process(resources)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 426, in process
self.process_resource_set, self.id_key, resources, tags, self.log)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 133, in _common_tag_processer
raise error
File "/usr/lib64/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(self.args, **self.kwargs)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 664, in process_resource_set
if not ecs_taggable(self.manager.resource_type, r):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 44, in ecs_taggable
path_parts = r[model.id].rsplit(':', 1)[-1].split('/')
KeyError: 'containerInstance'
2020-05-19 16:53:59,749: custodian.commands:ERROR Error while executing policy ecs-verify-container-instances-ami, continuing
Traceback (most recent call last):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/commands.py", line 281, in run
policy()
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/policy.py", line 1062, in call
resources = mode.run()
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/policy.py", line 323, in run
results = a.process(resources)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 426, in process
self.process_resource_set, self.id_key, resources, tags, self.log)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 133, in _common_tag_processer
raise error
File "/usr/lib64/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(
self.args, **self.kwargs)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 664, in process_resource_set
if not ecs_taggable(self.manager.resource_type, r):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 44, in ecs_taggable
path_parts = r[model.id].rsplit(':', 1)[-1].split('/')
KeyError: 'containerInstance'

policies:

  • name: ecs-verify-container-instances-ami
    resource: aws.ecs-container-instance
    actions:
    • type: tag
      key: teste
      value: "instance1"
7 replies
pendyalal
@pendyalal

Hi all, we have multip;e cloudtrails in account, I want to see if a specific name trail is enabled or not. Here is my policy.

policies:
  - name: "cloudtrail-enable-trail"
    resource: cloudtrail
        - type: value
          key: Name
          value: Log-Trail
        - type: value
          key: Name
          value: Trails

Am I missing anything here. I only want the accounts that these specific trails doesn't exist

18 replies
Dan Parsons
@danparsons
Anyone here using c7n-org in production? I'd love to ask you a question or two
1 reply
Mike
@mikejgray
Happy to help with questions, @danparsons
Jamison Roberts
@jtroberts83
Maybe my brain just isn't working today but I am trying to find a way to do a regex check on each dict value without knowing the dict key names. For example in the following snippet how would I check all those variables values against a regex checking for Access Keys?
"Environment": {
      "Variables": {
        "Bucketname": "some-input",
        "TopicArn": "arn:aws:sns:us-east-1:12345678910:testSNS",
        "TopicId": "OutSNS"
       "SomeOtherVar": "AKHH34HSF8YS748"
      }
    },
45 replies
Dan Parsons
@danparsons
Is there a way in c7n to report on "all aws resources in this account should be in us-east-1. Report on any resources that are in another region"?
8 replies
@mikejgray thanks for the response, I figured it out!
Naggappan
@naggappan
Hi team, I am trying to use cloud0custodian to crawl aws cloud trial logs. Like lets say if some one create or upload a new AMI i should get notify, some one delete any existing ami i should get notify etc..
12 replies
Amarankit Srivastava
@amarankit-srivastava
I am wondering, how does c7n-mailer finds/gets - event-owner & - resource-owner (as in 'to')?
3 replies
Jason Roth
@jsnrth

Hi, I'm working on setting up cloud custodian against our test account in AWS. I'd like to have it clean up EC2 instances after some period of time, but I'd like that timing to be somewhat granular.

Ideally, I would like to tag EC2 instances with tag:cloud-custodian and values like 30m, 1h, 6h, 3d, etc., and use that as a policy to terminate the EC2 instances after 30 minutes, 1 hour, 6 hours, 3 days, etc.

Is that a supported use-case for this tool?

Jamison Roberts
@jtroberts83
@jsnrth Yes you can do that with the mark-for-op action and marked-for-op filters
Jason Roth
@jsnrth
@jro
@jtroberts83 great, thanks! i'll take a look at those docs
Jon Gilmore
@JonGilmore_gitlab

think I'm not quite understanding the conditions correctly. i'm trying to get a policy to only execute with accounts defined with c7n-org with tag type:prod

policies:
  - name: elb-no-instances
    resource: elb
    conditions:
      - type: value
        key: type
        value: "prod"
        op: eq
    filters:
      - Instances: []

and here's a snip from my org yml

  - account_id: "123123123"
    name: prod
    tags:
      - type:prod
    regions:
      - us-east-1
      - us-east-2
      - eu-west-1
      - ap-northeast-1
      - ap-northeast-2
    role: arn:aws:iam::123123123:role/c7n-role
27 replies
when I then run c7n-org, it comes back with zeros instantly. if I remove the condition, elbs return, so I know that its not just returning zero
Edward Moon
@edwardmoon

is there a way to detect vpc network acls which allow wide open access (eg like the default vpc rules shown here: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html)?

looked at the vpc network acl docs but didn't see anything useful there.

1 reply
Charles Roberts
@charles545587

@kapilt

I am working on a project where we are using AWS Config Managed rules for compliance visibility and because they cover a lot of the use cases we want.

Because there is no native integration between config and Security Hub and so we can build in some automated responses we are using c7n policies.

At the moment, there is no functionality within c7n to enable the managed config rules. This is easy to achieve in cfn or terraform. Would the ability to enable a named Config managed rule from within a c7n policy be a valid use case.

It would allow users to define everything within their policy and if there was no managed config rule they can define a custom config rule using c7n.

What would be your view?

3 replies
Jamison Roberts
@jtroberts83

@all - I got the my script posted which parses your current custodian schema to extract all resource types, then runs generic scan policies against all those resource types it discovers and then generates csv and json reports for all resources as well as an expanded schema and uploads all the report files to your S3 bucket. These JSON reports are EXTREMELY useful for writing policies as you can see the keys you can filter on and what data is available.

https://github.com/jtroberts83/Cloud-Custodian/blob/master/schema_expander/ScanAndReportALLResourceTypes.sh

To run this you would copy the script to your custodian environment, run a chmod +x sciptname.sh to make it executable, edit the S3 bucket variable in the script and then run it. Assumes you are using c7n-org. If you are not using c7n-org you can modify the c7n-org commands to use custodian run with your parameters. Let me know if you have any questions and enjoy! Cheers! :beers:

3 replies
manitmalik
@manitmalik

Hi, Running into an issue regarding IAM users with 2 access Keys
Filter :

- and:
    - type: credential
      key: access_keys.last_used_date
      value_type: age
      value: 80
      op: greater-than
    - type: credential
      key: access_keys.active
      value: true

The resource.json is bringing result as

"access_keys": [
        {
          "active": true,
          "last_rotated": "2019-03-07T15:28:09+00:00",
          "last_used_date": "2020-04-23T02:53:00+00:00",
          "last_used_region": "us-west-2",
          "last_used_service": "s3",
          "c7n:match-type": "credential"
        },
        {
          "active": false,
          "last_rotated": "2019-09-10T09:41:02+00:00",
          "last_used_date": "2019-10-03T15:00:00+00:00",
          "last_used_region": "us-west-2",
          "last_used_service": "s3",
          "c7n:match-type": "credential"
        }

Query is that why i am getting this resource as none of the access key satisfy both conditions. Is it working as "or" condition for both of the access keys
@jtroberts83 @kapilt Please share your thoughts

38 replies
Clayton
@hammer2j2
Are there any plans to support an un-deploy feature in CC? Sometimes we have cases where we may have more than a single lambda to enforce say MFA on a console user, and one lambda tags then after 7 days of the tag age another lambda removes access. We want to downgrade that policy to only keep the notification/tagging lambda but not have the removal lambda in some cases where it's been decided not to do that part.
14 replies
crickyyy1
@crickyyy1
Hello, I'm having a bit of trouble figuring out how to change the number of async retries when creating/running a lambda with periodic mode in AWS, does anyone have any experience changing it from the default of 2 to something else?
15 replies
Todd Stansell
@tjstansell
just posted a starting PR for refactoring how actions are handled ... essentially moving from a policy-level set of actions to resource-level ... where we actually track individual resource success/fail and proceed with only ones that succeed. this also allows us to have some actions like notify that can look at all resources, even previously failed ones, so we can report on failures. The PR is going to be pretty large, as I'm refactoring every action in every resource ... using common code for many things, etc. if you're interested, please review ... comments welcome as i continue to walk through all the resources and update their actions. :) cloud-custodian/cloud-custodian#5790
2 replies
Kapil Thangavelu
@kapilt
1 reply
Dan Parsons
@danparsons
Is there a way to get c7n (via c7n-org) to NOT attempt to write to output/ ?
20 replies
Dan Parsons
@danparsons
LOL this is what happens when you test c7n too many times in one day, from aws sqs error: An error occurred (Throttling) when calling the SendRawEmail operation (reached max retries: 4): Daily message quota exceeded.
4 replies
Jamison Roberts
@jtroberts83
wallabyies
@wallabyies

Schema Diff for c7n 0.9.1.0 - 0.9.2.0 Has been generated here - https://github.com/jtroberts83/Cloud-Custodian/blob/master/schema_expander/AWS-0.9.1.0-0.9.2.0-Changes.md

A new version, awesome!

Jamison Roberts
@jtroberts83
@kapil Having an issue where I am trying to tag the Creation Time of a lambda via the CreatFunction cloudtrail event and tagging it with the {now} UTC value but it is tagging the date that the policy was created, not when the lambda CreateFunction was triggered. I was told there is an issue open on this but didn't find it. Any plans to get that fixed as it will solve some big problems for us with enforcing on resources which don't have a CreationDate in their metadata
2 replies
Is there a way to extract the LastModified time from the cloudtrail event and put it's value as the CreatedDate tag value?