Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • May 09 07:45
  • May 08 20:48
  • May 08 05:12
    tjstansell synchronize #6610
  • May 08 05:07
    ranjanashish starred cloud-custodian/cloud-custodian
  • May 08 02:15
    tjstansell synchronize #6610
  • May 08 01:37
    ajkerrigan edited #6486
  • May 08 01:37
    ajkerrigan edited #6486
  • May 08 01:37
    ajkerrigan edited #6486
  • May 08 01:36
    ajkerrigan commented #6666
  • May 07 21:16
    kylejohnson514 synchronize #6469
  • May 07 21:08
    kylejohnson514 synchronize #6469
  • May 07 20:46
    kylejohnson514 synchronize #6469
  • May 07 20:27
    kylejohnson514 synchronize #6469
  • May 07 20:13
    kylejohnson514 synchronize #6469
  • May 07 20:09
    kylejohnson514 synchronize #6469
  • May 07 19:55
    kapilt closed #6669
  • May 07 19:55
    kapilt commented #6669
  • May 07 19:52
    kylejohnson514 labeled #6669
  • May 07 19:52
    kylejohnson514 opened #6669
  • May 07 17:53
    kylejohnson514 synchronize #6469
Ravindra babu
@Ravindrababu99_twitter

@jtroberts83 can you please help how to filter roles that has only ec2 service in trusted entity, "type: used" filters all the roles that has Lambda, EC2 and ECS services in trusted entities.

policies:

name: iam-role-InUse-and-have-no-ssmmanaged-notify
resource: iam-role
mode:
type: periodic
role: arn:aws:iam::account-name:role/role-name
schedule: "cron(21 6 ? *)"
filters:
type: used
state: true
type: no-specific-managed-policy
value: AmazonSSMManagedInstanceCore
type: value
key: AssumeRolePolicyDocument.Statement[].Principal.Service
op: in
value:
ec2.amazonaws.com
actions:
type: notify

5 replies
mohinder6
@mohinder6
Hello. Does ECS services support offhours like we have for ec2? https://cloudcustodian.io/docs/aws/examples/offhours.html#scheduleparser-time-specifications
2 replies
Morgan McEntire
@mmcenti
Does aws.iam-role.actions.set-policy support dynamic IAM policy names? Looks like no but I could be missing something. I am trying to create a custodian policy to make sure an IAM role is unable to be modified and should therefore detach any IAM policies attached to it
14 replies
satvan23
@satvan23
Guys. So we have AWS only and was running c7n-org with no issues. But after a yum update, it asks for c7n-azure/cn7-gcp ?
4 replies
Dan Parsons
@danparsons
Hey everyone! I've got a single c7n policy here that, when i run c7n-mailer on the entries in SQS, it sends 2 emails for the 1 policy. Is there any way to control that? I'd prefer it all be in one email
In case it matters, the policy in question is simple, it's just looking for 0.0.0.0/0 or ::/0 in security groups
Maybe it's trying to avoid sending emails above a certain size?
Dan Parsons
@danparsons
Log looks like this:
2020-05-13 21:19:31,503 - custodian-mailer - INFO - Downloading messages from the SQS queue.
2020-05-13 21:19:33,088 - custodian-mailer - INFO - Sending account:xxxx-team-se policy:wide-open-sg security-group:35 email:default1.html to ['foo@xxxx3.com']
2020-05-13 21:19:34,750 - custodian-mailer - INFO - Sending account:xxxx-team-se policy:wide-open-sg security-group:250 email:default1.html to ['foo@xxxx3.com']
9 replies
brunoeustaquio
@brunoeustaquio

Hey guys, I'm just starting with c7n. Ran a few simple policies locally in my machine and now want to build a poc for the company. I have two questions:

1 - I ran locally following the example using STS credentials, but I want to use roles on AWS. Reading the documentation it's supported (https://cloudcustodian.io/docs/deployment.html#iam-setup) but I'm getting PermissionError: [Errno 13] Permission denied all the way. I added a role to the server giving read and list to all EC2 and policy is super simple, it just check if there's servers in regions which isn't supposed to:

  - name: ec2-invalid-regions
    resource: aws.ec2
    filters:
      - not:
        - type: value
          key: Placement.AvailabilityZone
          value: ap-southeast-2
          op: contains

Do I need to specify some specific flag when using roles instead of credentials?

2 - I see c7n can provision lambdas to execute some actions, but can c7n itself run from a lambda?

thanks!

13 replies
Amarankit Srivastava
@amarankit-srivastava

Hello experts, I have simple policies as below and it is working for a set of pre-created users BUT weirdly not working for a new user I created... any expert comments here?
Working for (1)
filters:

  - type: value
    key: UserName
    value: "sys_admin"
    op: eq
  - type: credential
    key: mfa_active
    value: false

Got result- [
{
"Path": "/",
"UserName": "sys_admin",
"UserId": "xxxxxxxxxx",
"Arn": "arn:aws:iam::xxxxxxxxx:user/sys_admin",
"CreateDate": "2018-03-10T21:01:09+00:00",
"PasswordLastUsed": "2018-03-17T10:46:05+00:00",
"c7n:MatchedFilters": [
"UserName"
],
"c7n:credential-report": {
"user": "sys_admin",
"arn": "arn:aws:iam::xxxxxxxxx:user/sys_admin",
"user_creation_time": "2018-03-10T21:01:09+00:00",
"password_enabled": true,
"password_last_used": "2018-03-17T10:46:05+00:00",
"password_last_changed": "2018-03-17T10:45:39+00:00",
"password_next_rotation": "2018-06-15T10:45:39+00:00",
"mfa_active": false,
"access_keys": [
{
"active": true,
"last_rotated": "2018-03-10T21:01:11+00:00",
"last_used_date": null,
"last_used_region": null,
"last_used_service": null
}
]
}
}
]
Not working for (2) filters:

  - type: value
    key: UserName
    value: "sys_abc"
    op: eq
  - type: credential
    key: mfa_active
    value: false

Got result as - [] ------ AND it is only working when I am filtering ONLY with 'UserName' (i.e. after removing 'credential' filter and in that case it off-course doesn't shows "c7n:credential-report" in output).

11 replies
pendyalal
@pendyalal
@kapilt , enabling encryption for new s3 buckets ,especially If say I create-bucket, Custodian looks and doesn't see encryption while at the same time I'm execute a put-encryption with a kms key and custodian overwrites that with . how can we solve this?
6 replies
DigeratiDad
@digeratidad
Hey everyone. Is there a way to update policies or do you just have to delete them and re-deploy? I don’t see a —update flag so I assume I can’t update an exisiting policy but just want to make sure.
11 replies
manitmalik
@manitmalik
Is there a way to parameterize values in custodian policy ?
18 replies
DigeratiDad
@digeratidad
Hello, does anyone have a good example of a auto-tag-user policy for EBS volumes. Mine is not working. I keep getting event ids not resolved in my cloudwatch logs. Here is my policy:
policies:
  - name: ebs-auto-tag-creator
    resource: ec2
    mode:
      type: cloudtrail
      role: arn:aws:iam::{account_id}:role/CloudEng-Cloud_Custodian
      events:
        - source: ec2.amazonaws.com
          event: CreateVolume
          ids: "requestParameters.volumeId"
    filters:
      - "tag:Creator": absent
    actions:
      - type: auto-tag-user
        tag: Creator
10 replies
I’ve also tried ids: “responseElements.volumeId” and got the same error.
DigeratiDad
@digeratidad
Here is the exact error I’m seeing:
error:An error occurred (InvalidInstanceID.Malformed) when calling the DescribeInstances operation:
3 replies
KVInventoR
@KVInventoR
Hi All,
@kapilt
Do you have any example to filter all aws managed policies for iam role?
probably filter by arn: arn:aws:iam::aws:policy/aws-service-role/*.
Mike
@mikejgray
How do I disable the caching on a c7n-org report? I see a --cache-path parameter but I'm not clear on how it works or what it's doing.
23 replies
Allison D
@allisonis
I noticied recently have c7n_gcp enable flow logs is not enabling flow logs for a partiuclar region.
24 replies
Any thoughts on where to look for how subnet resources are iterated over per region?
DigeratiDad
@digeratidad
Has anyone seen this issue before with ELB
[WARNING]    2020-05-14T21:43:54.197Z    da8bfff2-799d-453c-8e82-313b6edd3046    event ids not resolved: ['arn:aws:elasticloadbalancing:us-west-2:########:loadbalancer/app/cctagtesting/0ae4bc40b82b22c9'] error:An error occurred (ValidationError) when calling the DescribeLoadBalancers operation: LoadBalancer name cannot be longer than 32 characters
Here is my policy:
policies:
  - name: elb-classic-auto-tag-creator
    resource: elb
    mode:
      type: cloudtrail
      role: arn:aws:iam::{account_id}:role/CloudEng-Cloud_Custodian
      events:
        - source: elasticloadbalancing.amazonaws.com
          event: CreateLoadBalancer
          ids: "responseElements.loadBalancers[].loadBalancerArn"
    filters:
      - "tag:Creator": absent
    actions:
      - type: auto-tag-user
        tag: Creator
29 replies
Ravindra babu
@Ravindrababu99_twitter
@jtroberts83 Hi, old custodian mail alerts also getting delivered after ugrading the mailer to 3.7 from 2.7 old version with a new msg-template, I've been using same sqs-mailer, so can you help me understand the scenario why am I getting older msgs getting delivered now and how can I stop that.
6 replies
Ravindra babu
@Ravindrababu99_twitter
@jtroberts83 getting below error in cloudwatch logs, assuming because of this mails are getting re-delivered.
error: An error occurred (InvalidParameterValue) when calling the SendRawEmail operation: Missing final '@domain'
1 reply
satvan23
@satvan23

@satvan23 It could look something like this:

- name: ec2-using-unapproved-ami
  resource: ec2
  filters:
      - type: value
        key: "ImageId"
        op: not-in
        value_from:
           url: s3://yours3bucket/ApprovedAMIs.csv
           format: csv2dict

@jtroberts83 Thanks !

1 reply
Mike
@mikejgray
Hey folks, I'm becoming enamored of this c7n-org report functionality...can it read from S3 logs directly? They're in a different tree structure than local logs so wondering if it does that
2 replies
amolkk1980
@amolkk1980_twitter
Hi Guys.. I have a question. Can Cloud Custodian be configured/supported with Oracle OCI?
2 replies
Alicia Steen
@aliciasteen
Hi, does the flag --region all work, I have tried to use it but come across the error botocore.exceptions.NoRegionError: You must specify a region.
10 replies
Ravindra babu
@Ravindrababu99_twitter

@kapilt , while three different custodian policies triggered on cloudtrail (CreateBucket) event to add tags, getting below error in one of the policies while defining mark-for-op tag. Due to this, mailer is not sending bucket information in the alert...Can you please help how to overcome this.

"errorMessage": "A conflicting conditional operation is currently in progress against this resource. Please try again."

7 replies
KVInventoR
@KVInventoR

How does or operator work?
Hi all, I tried run CC in cloudtrail mode and create a simple filter, I would like to filter any ec2 from elastimapreduce, datapipeline or if ec2 has specific tag
But I got many reports with ec2 which were part of emr cluster.
I checked the events in cloudwatch logs for lambda: ec2_tags_validate_mode_cloudtrail

        "eventTime": "2020-05-18T19:00:34Z",
        "eventSource": "ec2.amazonaws.com",
        "eventName": "RunInstances",
        "awsRegion": "us-west-2",
        "sourceIPAddress": "elasticmapreduce.amazonaws.com",
        "userAgent": "elasticmapreduce.amazonaws.com",

for it looks like something wrong with my filter or CC just ignored filters section.

policies:
  - name: ec2_tags_validate_mode_cloudtrail
    resource: ec2
    description: |
      EC2 validate tags on new started instances.
    mode:
      type: cloudtrail
      role: *lambda_role
      events:
        - RunInstances
      tags:
        Name: *lambda_name
        Team: "TEAMNAME"
    filters:
      - or:
        - "tag:TestTag": absent
        - type: event
          key: sourceIPAddress
          op: ne
          value: "elasticmapreduce.amazonaws.com"
        - type: event
          key: sourceIPAddress
          op: ne
          value: "datapipeline.amazonaws.com"

    actions:
      - <<: *notify-var
        subject: "CControl - EC2:TAGS validation in cloudtrail mode"

Could you please help to find, what can be wrong?

7 replies
aakshaik2
@aakifshaikh
what filter should i use to check on unencrypted snapshots: type: value key: Snapshots.Encrypted value: false
4 replies
Dan Parsons
@danparsons
Who here actually uses c7n-org? Care to answer a few questions for me? Like how am I supposed to give it a AWS_SESSION_TOKEN env var? To my knowledge, you only get a AWS_SESSION_TOKEN when you do a sts:AssumeRole, however c7n-org itself is going to be doing sts:AssumeRole to get into all my accounts.... so I can't feed it creds to start with (to then access all accounts) because all I have are the access key and secret key for the top level c7n account
46 replies
Eri Wahyudi
@ewahyudi_gitlab
Greetings all ! Cloud-Custodian newby here; I'm starting out with simple script to apply simple tag key/value pair. I'm trying to add tag value in json format as follows:
{"tz": "US/Eastern", "sched": "2-6:0800-1730"}, so far I have added single quotes like so -
  • type: tag
    key: EC2_Start_Stop
    value: '{"tz": "US/Eastern", "schedule": "2-6:0800-1730"}'
    When I ran the "custodian validate script.yml" command, this returned INFO Configuration valid but when I tried running, it failed. Does anyone have any insight to what is it in the syntax that may be wrong?
14 replies
tomarv2
@tomarv2
I am trying to find unused VPC, as VPC can have many resources, any suggestion?, look for unused ENI and then find VPC
8 replies
Jamison Roberts
@jtroberts83
This message was deleted
3 replies
Jamison Roberts
@jtroberts83
Question - When should I expect the Lambda policy functions 'LastModified' date to change? Should that be updated each time I deploy the lambda based policy or only when something changes? I see most of my custodian lambda functions LastModified date gets updated daily but then some lambda policy functions are showing as last modified several days or months ago. Is that to be expected??
4 replies
Barak Schoster Goihman
@schosterbarak
Question - I want to use c7n-org i see it can take role as a parameter for cross-account access. can it take external id too?
1 reply
Kapil Thangavelu
@kapilt
fwiw, there's a proposal up for being able to use custodian policies on iaac assets (terraform, cfn) in DevOps pipelines, comments welcome cloud-custodian/cloud-custodian#5782
1 reply
Amarankit Srivastava
@amarankit-srivastava
@all, I am struggling with c7n_mailer setup... have python36, have c7n and then installed successfully c7n-mailer (i.e. without any error via https://cloudcustodian.io/docs/tools/c7n-mailer.html#developer-install-os-x-el-capitan)... but when I am trying any command (even -h/help), its giving error ..................... from c7n_mailer import deploy, utils
File "/root/cloud-custodian/tools/c7n_mailer/c7n_mailer/deploy.py", line 19, in <module>
from c7n.mu import (
ModuleNotFoundError: No module named 'c7n'......ANY CLUES WHERE I AM GOING WRONG...
22 replies
Barak Schoster Goihman
@schosterbarak
i'm trying to connect c7n to cloudtrail but i didn't find where do i point c7n to the cloudtrail bucket. is there a good tutorial for it?
Karl de Castro Fonseca
@KarlCF

I'm trying to tag (action) some container instances using the resource aws.ecs-container-instance without filters. Using both the new arn model and the old one.
I'm getting the following error in both situations. I got no error just listing the container instances, just when i'm trying to tag (action) the resource.

2020-05-19 16:53:59,740: custodian.policy:INFO policy:ecs-verify-container-instances-ami resource:aws.ecs-container-instance region:sa-east-1 count:1time:0.37
2020-05-19 16:53:59,746: custodian.actions:ERROR Exception with tags: [{'Key': 'teste', 'Value': 'instance1'}] 'containerInstance'
2020-05-19 16:53:59,747: custodian.output:ERROR Error while executing policy
Traceback (most recent call last):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/policy.py", line 323, in run
results = a.process(resources)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 426, in process
self.process_resource_set, self.id_key, resources, tags, self.log)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 133, in _common_tag_processer
raise error
File "/usr/lib64/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(self.args, **self.kwargs)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 664, in process_resource_set
if not ecs_taggable(self.manager.resource_type, r):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 44, in ecs_taggable
path_parts = r[model.id].rsplit(':', 1)[-1].split('/')
KeyError: 'containerInstance'
2020-05-19 16:53:59,749: custodian.commands:ERROR Error while executing policy ecs-verify-container-instances-ami, continuing
Traceback (most recent call last):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/commands.py", line 281, in run
policy()
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/policy.py", line 1062, in call
resources = mode.run()
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/policy.py", line 323, in run
results = a.process(resources)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 426, in process
self.process_resource_set, self.id_key, resources, tags, self.log)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 133, in _common_tag_processer
raise error
File "/usr/lib64/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(
self.args, **self.kwargs)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 664, in process_resource_set
if not ecs_taggable(self.manager.resource_type, r):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 44, in ecs_taggable
path_parts = r[model.id].rsplit(':', 1)[-1].split('/')
KeyError: 'containerInstance'

policies:

  • name: ecs-verify-container-instances-ami
    resource: aws.ecs-container-instance
    actions:
    • type: tag
      key: teste
      value: "instance1"
7 replies
pendyalal
@pendyalal

Hi all, we have multip;e cloudtrails in account, I want to see if a specific name trail is enabled or not. Here is my policy.

policies:
  - name: "cloudtrail-enable-trail"
    resource: cloudtrail
        - type: value
          key: Name
          value: Log-Trail
        - type: value
          key: Name
          value: Trails

Am I missing anything here. I only want the accounts that these specific trails doesn't exist

18 replies
Dan Parsons
@danparsons
Anyone here using c7n-org in production? I'd love to ask you a question or two
1 reply
Mike
@mikejgray
Happy to help with questions, @danparsons
Jamison Roberts
@jtroberts83
Maybe my brain just isn't working today but I am trying to find a way to do a regex check on each dict value without knowing the dict key names. For example in the following snippet how would I check all those variables values against a regex checking for Access Keys?
"Environment": {
      "Variables": {
        "Bucketname": "some-input",
        "TopicArn": "arn:aws:sns:us-east-1:12345678910:testSNS",
        "TopicId": "OutSNS"
       "SomeOtherVar": "AKHH34HSF8YS748"
      }
    },
45 replies
Dan Parsons
@danparsons
Is there a way in c7n to report on "all aws resources in this account should be in us-east-1. Report on any resources that are in another region"?
8 replies
@mikejgray thanks for the response, I figured it out!
Naggappan
@naggappan
Hi team, I am trying to use cloud0custodian to crawl aws cloud trial logs. Like lets say if some one create or upload a new AMI i should get notify, some one delete any existing ami i should get notify etc..
12 replies
Amarankit Srivastava
@amarankit-srivastava
I am wondering, how does c7n-mailer finds/gets - event-owner & - resource-owner (as in 'to')?
3 replies
Jason Roth
@jsnrth

Hi, I'm working on setting up cloud custodian against our test account in AWS. I'd like to have it clean up EC2 instances after some period of time, but I'd like that timing to be somewhat granular.

Ideally, I would like to tag EC2 instances with tag:cloud-custodian and values like 30m, 1h, 6h, 3d, etc., and use that as a policy to terminate the EC2 instances after 30 minutes, 1 hour, 6 hours, 3 days, etc.

Is that a supported use-case for this tool?