Cloud Custodian/cloud-custodian

Rules engine for AWS management, DSL in yaml for query, filter, and actions on resources

mfrost-uptic

@mfrost-uptic

Failed validating 'type' in schema['properties']['vars']:
{'type': 'object'}

1 reply

cant seem to get it to work

Pradeep Reddy

@prareed_twitter

Hi, Is there a way to run multiple policies with one Lambda?

1 reply

Ramprasath26

@Ramprasath26

Can someone help on finding the default security group
custodian schema aws.security-group.filters.value.

2 replies

aakshaik2

@aakifshaikh

By default cloud custodian created the cloudwatch metrics for every single policy- How do I find the cost for all metrics created by custodian?

6 replies

smithjamiej

@smithjamiej

Is it possible to have a policy similar to the following example: (https://cloudcustodian.io/docs/aws/examples/eipgarbagecollect.html) do a reminder notification every day, such a 5 days left, 4 days left, etc. (coundown)?

7 replies

Rahul Vinod Sharma

@rahulsharma0810_twitter

is it possible to check if there is the same key: value in the resource JSON, For example, check if multiple lambdas have the same role associated?

1 reply

Cloud14

@AWSCloud21

Hey! Has anyone created a policy to delete an overly permissive iam policy from an iam role? Ex: Have a role named TestRole which has allow "*" and I want to delete the policy but keep the role

1 reply

udit sidana

@udit_sidana_twitter

Hello, I want to implement c7n-org . What would be best practice in terms of infrastructure?

2 replies

mihait

@mihait:matrix.org

[m]

hi, any chance I can run this https://cloudcustodian.io/docs/usecases/s3globalgrants.html to filter out and find s3 buckets that allow any users to view files in my bucket?

filters:

          - type: global-grants
            grantees:
              - "http://acs.amazonaws.com/groups/global/AllUsers"

thinking of something like this but it's not exactly working

2 replies

Alexander Qiu

@aq17

Hi! Does anyone know if CC supports granular pattern matching? Specifically, for a GCP resource's set-iam-policy action, instead of listing all users in the remove-bindings list, being able to just say something like user:* ?

aakshaik2

@aakifshaikh

In order to use c7n-trailcreator, is there an instruction on how to create a sqlite db? Does the script creates by itself or no?

1 reply

jvoeller

@jvoeller

Hello,
I was trying to create policy that checks if a VNet subnet has a security group attached to it. I'd also settle for a filter that just outputs the VNet that has a subnet missing a security group.
Using a value filter on the vnet resource it shows all the subnets, but I found no way to iterate over them to check for an existing properties.subnets[].properties.networkSecurityGroup key. Using the present value does not work without a number in the brackets. Anyone got an idea how to get around that?

1 reply

Abel

@Abikjose

I'm getting this error

Error: Invalid base64-encoded string: number of data characters (53) cannot be 1 more than a multiple of 4 Unable to base64 decode slack_token, will assume plaintext.

at lambda

This is my mailer.

queue_url: https://sqs.ap-southeast-1.amazonaws.com/123456789075/-custodian role: arn:aws:iam::123456789075:role/custodian-lambda-sqs-readonly slack_token: xoxb-123456789432-1234567890000-BhfNXkkVnXfeJ49ypb5kUH4C from_address: abc@abc.com region: ap-southeast-1

1 reply

Used slack template as slack_template: slack_default

Abel

@Abikjose

I'm sending notification to both email and slack.

Any idea?

Abel

@Abikjose

@LykinsN @thisisshi @kapilt

paulc75-sco

@paulc75-sco

Hello all. I keep seeing the error bellow when creating policies for cloud trail events. I have been able to create it with EC2 and S3 no bother but on all other modules i see this error with different events. File "e:\venv\custodian\lib\site-packages\c7n\policy.py", line 618, in validate
assert e in CloudWatchEvents.trail_events, "event shortcut not defined: %s" % e
AssertionError: event shortcut not defined: CreateNatGateway

is it an authoring error on my part.

Sample code below.

policies:

name: VPC-Tag-Compliance
resource: vpc
mode:
type: cloudtrail
events:
```
- CreateNatGateway
- CreateNetworkAcl
- CreateNetworkInterface
- CreateRouteTable
- CreateSecurityGroup
- CreateVpc
- CreateVpnGateway
```
role: xxx
timeout: 900
actions:
- type: tag
  tags:
  application-name: test 1
  business-unit: test 2
  contact-email: test 3
  environment: test 4
  group-project: test 5
  operating-centre: test 6
  owner: test 7
  short-description: test 8
  use-context: test 9

6 replies

smithjamiej

@smithjamiej

Is there a way to put a AWS tag value into a violation_desc, action_desc, or email subject for reference on the alert

Ananth Balasubramanian

@linuxananth1976

Hey, I have a query
whether I'm missing anything or the behaviour is like that only?
in c7n_mailer aws lambda i don't see any logging in cloudwatch functions
logs except lambda standard logs i.e, start, end and report logging.
when I modified the code as below it works.

logging.getLogger('botocore').setLevel(logging.WARNING) ==> logger.setLevel(logging.INFO)

Can you please confirm the same if i.e, the case, can we have it as an argument to not disturb the code.

10 replies

Atul Jadhav

@tulJadhav_twitter

policies:

name: ec2-ssm-check
resource: ec2
filters:
- type: ssm
  key: PlatformName
  op: ne
  value_from:
  url: file:file.txt
  format : txt

have been using this policy to count the number of Ubuntu instances, file.txt contains 'Ubuntu' on the first line, the policy does not read value from the given file to match for value, hence also counts all other instances as well, what could be the mistake here

3 replies

Sonia Gurdian

@PendragonDay

Help, I'm trying to write a policy to detect if an rds-snapshot has been made public, then the action will be to delete it. However when looking at the output of the describe-db-snapshot cli command, there is one option that could be used for a generic filter which is "SnapshotType": however even when I have set the snapshot to public, the "SnapshotType" remains as "manual" there is nothing in the output that indicates the snapshot is public. How can I make Cloud Custodian detect this? This is my policy:

  - name: aws-rds-snapshot-PubliclyAccesible-rem
    resource: rds-snapshot
    description: Deletes RDS public snapshots.
    filters:
      - type: value
        key: SnapshotType
        op: eq
        value: public
    actions:
      - delete

The filter on the value: public is not detecting, because it remains manual even when I have made the snapshot public. Cross-account option is not equivalent because I can still make an rds-snapshot public without giving cross-account access to another account. deleting based on cross-account will delete snapshots that are not public and will not delete those that are public. Any help will be appreciate it!

4 replies

Pradeep Reddy

@prareed_twitter

Hello all,
When I run the policy and redirect the output to S3, it is saving as .gz. Is there a way to save the output in json or csv?
custodian run --output-dir s3://devops-bucket instance-type-stop-action.yml

2 replies

hamzazai2021

@hamzazai2021

Hi, is there any instruction on how to provide secure string for Smtp_password in mailer ?

8 replies

myoung34

@myoung34

am i missing something?

  - name: engine-admin-assume-role-detected
    resource: account
    description: A Team Engine admin sso assume role has occurred
    mode:
      type: cloudtrail
      role: arn:aws:iam::{account_id}:role/cloud-custodian
      events:
        - source: sts.amazonaws.com
          event: AssumeRoleWithSAML
          ids: "requestParameters.roleSessionName"
    filters:
       - type: event
         key: "detail.responseElements.assumedRoleUser.arn"
         op: regex
         value: "AWSReservedSSO_engine-.+?-admin_.*"

the lambda fired as expected, i took the payload and put it in payload.json:

± cat payload.json | jq .detail.responseElements.assumedRoleUser.arn
"arn:aws:sts::redact:assumed-role/AWSReservedSSO_engine-production-admin_4e122ecb4/marcus.young@redact.com"

so i know the regex is right:

but:

Filter #1 applied 1->0 filter: {"type": "event", "key": "detail.responseElements.assumedRoleUse
r.arn", "op": "regex", "value": "AWSReservedSSO_engine-.+?-admin_.*"}

Filtered from 1 to 0 account

4 replies

Ananth Balasubramanian

@linuxananth1976

Hello, I have couple of questions as below:

I want to know about whether c7n org execution logs in ec2 can be logged in CloudWatch log groups?
SQS Trigger instead of Scheduler can we have as Real time events trigger? the reason is i don't want to run c7n mailer Lambda every 5 mins to reduce the cost instead of it having trigger as events.

2 replies

Ajay Misra

@ajmsra

hello I have a policy for ec2-off hours and have configured slack notification when the instances are stopped or started

    actions:
      - start
      - type: notify
        to:
        {% if sns=='True' %}
          - arn:aws:sns:us-east-1:{{ aws_account_id }}:c7n-mailer
        {% endif %}
        {% if slack_channel %}
          - slack://#{{ slack_channel }}
        {% endif %}
        action_desc: The above EC2 instances are Stopped
        transport:
          type: sqs
          queue: https://sqs.us-east-1.amazonaws.com/{{ aws_account_id }}/c7n-mailer

I have not defined violation_desc but still when the notification comes on slack violation_desc is marked as blank

How can I make sure that a blank violation_desc is not posted as this is not a violation of any rule we are just stopping and starting instances

2 replies

@kapilt

Sean Benton

@seanmac5_twitter

Hi, we're just getting started on GCP and trying to understand what capability exists to filter on compute instances metrics. We're getting the error:

2021-06-07 14:11:56,859: custodian.commands:ERROR invalid policy file: test.yml error: instance.filters Invalid filter type {'type': 'metrics'}

Oddly the example in the doc refers to gcp.firewall (not gcp.instance) so am I just trying to do something that's not supported?

5 replies

pentagonal-proboscis

@pentagonal-proboscis

is there a way with the aws.rrset resource to only pull back records which are in a public hosted zone? I have managed to filter down to just A records and CNAME, but also want to filter on public zone.

5 replies

Abel

@Abikjose

Getting this error from function app.

Result: Failure Exception: ModuleNotFoundError: No module named 'azure.identity'. Troubleshooting Guide: https://aka.ms/functions-modulenotfound Stack: File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/dispatcher.py", line 305, in _handle__function_load_request func_request.metadata.entry_point) File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 42, in call raise extend_exception_message(e, message) File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 40, in call return func(*args, **kwargs) File "/azure-functions-host/workers/python/3.6/LINUX/X64/azure_functions_worker/loader.py", line 83, in load_function mod = importlib.import_module(fullmodname) File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/site/wwwroot/mailer_697dc9ec-d696-4962-94d9-c4e1fadb0455/function.py", line 11, in <module> from c7n_mailer.azure_mailer import handle File "/home/site/wwwroot/c7n_mailer/azure_mailer/handle.py", line 6, in <module> from c7n_azure.session import Session File "/home/site/wwwroot/c7n_azure/session.py", line 14, in <module> from azure.identity import (AzureCliCredential, ClientSecretCredential,

1 reply

I have done it exactly mentioned in the documentation but I am not able to receive email through sendgrid

I tried sending email to the sendgrid directly and it's working.

CurtisAndersenSysdig

@CurtisAndersenSysdig

Does anyone know if I am doing anything wrong?

 - type: notify
      slack_template: slack-custom-template
      slack_msg_color: '#00F0F0'
      violation_desc: 'test vi'
      to:
        - slack://#custodian-all
        - https://hooks.slack.com/services/T/B/lasdf
        - slack://foo@bar.com
      transport:
        type: sqs

The issue is that both the slack channel and the web hook send the correct notify message to their respective channels. However, the email doesn't seem to start a slack message with the user, but slack does notify the user through an email that they have messages waiting for them. When the email is opened however there is no message. Is there a special way that I need to preface direct messages?

8 replies

Ravindra babu

@Ravindrababu99_twitter

@kapilt

Below event based policy on redshift, policy shows successfully triggered in lambda monitoring but there is no log-stream generated and policy is not adding the user in tags(creatorid).

name: redshift-auto-tag-user
resource: redshift
mode:
type: cloudtrail
role: arn:aws:iam::account-number:role/CloudCustodian
events:
```
- source: redshift.amazonaws.com
  event: CreateCluster
  ids: "requestParameters.ClusterIdentifier"
```
filters:
- "tag:owner": absent
- "tag:creatorid": absent
  actions:
- type: auto-tag-user
  tag: creatorid

5 replies

Shivanjan Chakravorty

@Glitchfix

Hi everyone, Glitchfix here I am looking for some help. Is there any open source repositories to refer policies for any well known compliance?

3 replies

CurtisAndersenSysdig

@CurtisAndersenSysdig

Hi everyone, does any one know if there is a way to have a policy that takes all of the EC2 instances missing particular tags. Then using these EC2 instances could individual notify the owners that their particular instance is missing the specified tag.
Getting the EC2 instances missing the desired tag filtered out is repetitively simple. But, from what I can tell these filtered out instances are then sent over to actions of notify and notify sends all of them at once.

7 replies

Pradeep Reddy

@prareed_twitter

Hello, Is there any plan to support OCI integration?

2 replies

CurtisAndersenSysdig

@CurtisAndersenSysdig

I'm now working on making custom templates for slack notifications. Does anyone have any references of to help in learning how to format these properly>

3 replies

Ray Henson

@RayHenson_twitter

Hello, I am just starting to set up Cloud Custodian for PoC, and of course I need to show something more difficult than just enforcing tagging etc. I was asked to show if I could report when (AWS) an RDS instance is not part of a particular "Parameter Group" != "ssl-group". tried the following:
policies:

name: rds-pg
resource: rds
filters:
- type: value
  key: "DBParameterGroupName"
  op: eq
  value: "ssl-group"

but doesn't work, not sure how to get the JSMEPath expression right for :

        "DBParameterGroups": [
            {
                "DBParameterGroupName": "ssl-group",
                "ParameterApplyStatus": "in-sync"
            }
        ],

6 replies

Shawn L

@slaphitter

Hey folks. Trying to restrict checking the tags of S3 buckets to only run out of AWS us-east-1. When I deploy the policy to a lambda the conditions stanza seems to disappear. Is that expected behavior?

23 replies

    conditions:
      - type: value
        key: region
        op: equal
        value: "us-east-1"

Where communities thrive

People

Repo info

Activity