Cloud Custodian/cloud-custodian

Rules engine for AWS management, DSL in yaml for query, filter, and actions on resources

Kapil Thangavelu

@kapilt

@jterlecki there's a now date time variable thats available when rendering tag values that supports https://pyformat.info/#datetime date time strings

it also supports some date time arithmetic (+/- n days)

vtomardb

@vtomardb

@pavana-gadde : cross-account will work

jterlecki

@jterlecki

@kapilt ty - that helped. Is there a list of default runtime variables that we can consult?

vtomardb

@vtomardb

@pavana-gadde my bad it was for rds-snapshot

check this:

policies:
  - name: disable-rds-public-accessibility
    resource: rds
    filters:
      - PubliclyAccessible: true
    actions:
      - type: set-public-access
        state: false

https://cloudcustodian.io/docs/aws/resources/rds.html#aws-rds-actions-set-public-access

Kapil Thangavelu

@kapilt

@jterlecki for tag formatting, its pretty limited account_id, region, now

pavana-gadde

@pavana-gadde

@vtomardb it worked thanks

vtomardb

@vtomardb

@kapilt when using c7n-mailer-replay where can I extract MESSAGE_FILE from..

Kapil Thangavelu

@kapilt

@vtomardb

MAILER_FILE input is a file containing the exact base64-encoded, gzipped                                                                                                     
data that's enqueued to SQS via :py:meth:`c7n.actions.Notify.send_sqs

so grab a message off the queue

vtomardb

@vtomardb

tried with python 3.7 as well..

pavana-gadde

@pavana-gadde

i am writing a policy for tag compilance

Tag-Compliance (Should contain owner, end date details)

Tag should be of the format Username + End Date, Example: sameer-2019-08-20

i am using the filter filters:

- type: value
  key: "tag:tag:aws:cloudformation:stack-name"
  op: regex
  value: '^[a-z]+-\d{4}-\d{1,2}-\d{1,2}$'

my output is like this

Traceback (most recent call last):
File "/Users/pgadde/custodian/bin/custodian", line 11, in <module>
load_entry_point('c7n', 'console_scripts', 'custodian')()
File "/Library/Python/2.7/site-packages/c7n/cli.py", line 373, in main
command(config)
File "/Library/Python/2.7/site-packages/c7n/commands.py", line 65, in _load_policies
collection = policy_load(options, fp, validate=validate, vars=vars)
File "/Library/Python/2.7/site-packages/c7n/policy.py", line 63, in load
errors[1], errors[0]))
c7n.exceptions.PolicyValidationError: Failed to validate policy Tag-Compliance Format
Error on policy:Tag-Compliance Format resource:security-group
'Tag-Compliance Format' does not match u'^[A-z][A-z0-9](-[A-z0-9]+)$'

Failed validating u'pattern' in schema[108][u'allOf'][0][u'properties'][u'name']:
{u'pattern': u'^[A-z][A-z0-9](-[A-z0-9]+)$', u'type': u'string'}

On instance[u'name']:
'Tag-Compliance Format'

Mirage288

@Mirage288

@kapilt - using c7n-trailcreator, i want to find users who created the ec2 instances..i'm able to successfully get records for s3, rds, security-group resources...but not for EC2...this is the resourcemap i'm using, what is that i'm doing wrong?

{
  "resource": "ec2",
  "shape": "User",
  "events": [
    {
      "event": "CreateInstance",
      "ids": "requestParameters.instanceId",
      "service": "ec2.amazonaws.com"
    }

i tried using userName for ids as well, no luck. Thank you

vtomardb

@vtomardb

@kapilt finally I figured it out..there was an issue with the subject not the template: subject: "{{account: account_id}}" it should have been: subject: "account: {{account_id}}"

ndesai15

@ndesai15

Best way to delete/remove/clean all the resources (i.e., cloudwatch Event Rules, Lambda functions) created by cloud custodian?

Kapil Thangavelu

@kapilt

@ndesai15 tools/ops/mugc.py

ndesai15

@ndesai15

dryrun of mugc.py returns all the custodian lambda functions even though config file is creating just one lambda function. Also, tried to filter by passing --prefix option which is not returning the exact function.

vtomardb

@vtomardb

@ndesai15 it will delete everything but the policies specified in the file.. please see chat from 19th there was a discussion around that..

Ian Hunt

@remainiac

@kapilt . thank you very much, will test that out. Does this only work when the resource type is set to lambda, ie if it was EC2 it wont work?

Ian Hunt

@remainiac

@kapilt Dont worry have tested and it works. As you say note its important its in array format. It can work on resource types other than lambda. I have put it under the "mode" section. Many thanks!

Ian Hunt

@remainiac

syntax
mode:
layers: ['arn:aws:lambda:{region}:{account_id}:layer:custodian_0843:1']

vtomardb

@vtomardb

@kapilt
i have a s3 policy:

      type: cloudtrail
      role: "arn:aws:iam::{account_id}:role/CustodianGuard"
      events:
        - event: PutBucketPolicy
          source: s3.amazonaws.com
          ids: "requestParameters.bucketName"
    filters:
      - type: global-grants
        allow_website: True

but it alerted me, i wanted it to ignore if there is any condition..
here is my bucket policy

"Principal": "*",
  "Action": "s3:*",
  "Resource": "arn:aws:s3:::xxxxxx/*",
  "Condition": {
      "IpAddress": {
          "aws:SourceIp": [
              "xxxxxxx",

any suggestion?

vtomardb

@vtomardb

this is what I was thinking.. i am looking for a broad condition, if they have any condition, i dont want alert:

      - type: has-statement
        statements:
          - Sid: *
            Effect: "Allow"
            Action: "s3:*"
            Resource:
                - "arn:aws:s3:::{bucket_name}/*"
                - "arn:aws:s3:::{bucket_name}"
            Condition:
              Null:

Philip Su

@philipsu522

Hey folks, are there service account-limits for s3? It doesn't look like it exists based on the schema, but AWS documentation does specify bucket limits https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

SrikanthSoma

@SrikanthSoma

@kapilt Is it possible to get the lambda list based on tags?

policies:
 - name: lambda-filter-tags
   description: get the list of lambdas which has key as stage and value as dev 
   resource: lambda 
   filters: 
     - type: value 
       key: "STAGE"
       op: eq 
       value: dev

The above policy don't seem to work

vtomardb

@vtomardb

@SrikanthSoma : key should be key: "tag:STAGE"

vtomardb

@vtomardb

@philipsu522 i see only these:

          'enum': ['EC2', 'ELB', 'VPC', 'AutoScaling',
                     'RDS', 'EBS', 'SES', 'IAM']}})

see the first example: https://cloudcustodian.io/docs/aws/examples/accountservicelimit.html

Philip Su

@philipsu522

Yep, does it make sense that we might want to support S3 as well?

given that there are AWS imposed limits there

ndesai15

@ndesai15

@vtomardb It will delete everything but from that file means, it will delete all the lambdas & cloudwatch event rules that created by that file only? Then why --dryRun shows all the custodian lambdas from account?

vtomardb

@vtomardb

it will delete everything else.. what ever is not there is that file..

ndesai15

@ndesai15

@vtomardb that means even to change retention policy on log-group you need a policy. I was expecting a CLI option while deploying policies

it will delete everything else.. what ever is not there is that file..

Sorry this is bit confusing. So it will delete everything related to policy.yml file & it won't touch other policies file resources. Is that something you mean?

Kapil Thangavelu

@kapilt

You need that policy anyways for other log groups, I don’t see value as a cli flag

Mugc there was a discussion two days ago that went into detail. Nutshell it deletes the policy artifacts for policies not in the passed in config files.

vtomardb

@vtomardb

@kapilt any suggestion on my question..

Kapil Thangavelu

@kapilt

@vtomardb global grants is about bucket acl for embedded iam policy you want cross-account

vtomardb

@vtomardb

thankyou..

vtomardb

@vtomardb

@kapilt : here are my s3 policies:

policies:      
  - name: cross-account-s3
    resource: s3
    mode: 
      type: cloudtrail
      role: "arn:aws:iam::{account_id}:role/Custodian"
      events:
        - event: PutBucketPolicy
          source: s3.amazonaws.com
          ids: "requestParameters.bucketName"
    filters:
      - type: cross-account
        whitelist: *accounts 

  - name: cross-account-s3-acl
    resource: s3
    mode: 
      type: cloudtrail
      role: "arn:aws:iam::{account_id}:role/Custodian"
      events:
        - event: PutBucketAcl
          source: s3.amazonaws.com
          ids: "requestParameters.bucketName"
    filters:
      - type: global-grants
        allow_website: True

If I put this policy in bucket:

"Principal": "*",
  "Action": "s3:*",
  "Resource": "arn:aws:s3:::xxxxxx/*",
  "Condition": {
      "IpAddress": {
          "aws:SourceIp": [
              "xxxxxxx",

its hitting 1st policy, i dont want it to do that.. I want it to ignore it...

ndesai15

@ndesai15

Mugc there was a discussion two days ago that went into detail. Nutshell it deletes the policy artifacts for policies not in the passed in config files.

Ok Thanks @kapilt now it makes sense.

Kapil Thangavelu

@kapilt

@vtomardb see the configuration options on cross-account you can white list individual conditions

vtomardb

@vtomardb

@kapilt i am just trying to check if there is any condition just ignore.. else i will have to handle hundreds of conditions..

let me try something..

this is what I did for SQS, i am trying to do same for s3:

      - not:
        - type: value
          key: Policy
          value: ".*\"Condition.*\".*"
          op: regex

Where communities thrive

People

Repo info

Activity