Rules engine for AWS management, DSL in yaml for query, filter, and actions on resources
kapilt on master
gcp - metrics filter (#6595) (compare)
@kapilt , while three different custodian policies triggered on cloudtrail (CreateBucket) event to add tags, getting below error in one of the policies while defining mark-for-op tag. Due to this, mailer is not sending bucket information in the alert...Can you please help how to overcome this.
"errorMessage": "A conflicting conditional operation is currently in progress against this resource. Please try again."
How does or
operator work?
Hi all, I tried run CC in cloudtrail mode and create a simple filter, I would like to filter any ec2 from elastimapreduce, datapipeline or if ec2 has specific tag
But I got many reports with ec2 which were part of emr cluster.
I checked the events in cloudwatch logs for lambda: ec2_tags_validate_mode_cloudtrail
"eventTime": "2020-05-18T19:00:34Z",
"eventSource": "ec2.amazonaws.com",
"eventName": "RunInstances",
"awsRegion": "us-west-2",
"sourceIPAddress": "elasticmapreduce.amazonaws.com",
"userAgent": "elasticmapreduce.amazonaws.com",
for it looks like something wrong with my filter or CC just ignored filters section.
policies:
- name: ec2_tags_validate_mode_cloudtrail
resource: ec2
description: |
EC2 validate tags on new started instances.
mode:
type: cloudtrail
role: *lambda_role
events:
- RunInstances
tags:
Name: *lambda_name
Team: "TEAMNAME"
filters:
- or:
- "tag:TestTag": absent
- type: event
key: sourceIPAddress
op: ne
value: "elasticmapreduce.amazonaws.com"
- type: event
key: sourceIPAddress
op: ne
value: "datapipeline.amazonaws.com"
actions:
- <<: *notify-var
subject: "CControl - EC2:TAGS validation in cloudtrail mode"
Could you please help to find, what can be wrong?
I'm trying to tag (action) some container instances using the resource aws.ecs-container-instance without filters. Using both the new arn model and the old one.
I'm getting the following error in both situations. I got no error just listing the container instances, just when i'm trying to tag (action) the resource.
2020-05-19 16:53:59,740: custodian.policy:INFO policy:ecs-verify-container-instances-ami resource:aws.ecs-container-instance region:sa-east-1 count:1time:0.37
2020-05-19 16:53:59,746: custodian.actions:ERROR Exception with tags: [{'Key': 'teste', 'Value': 'instance1'}] 'containerInstance'
2020-05-19 16:53:59,747: custodian.output:ERROR Error while executing policy
Traceback (most recent call last):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/policy.py", line 323, in run
results = a.process(resources)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 426, in process
self.process_resource_set, self.id_key, resources, tags, self.log)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 133, in _common_tag_processer
raise error
File "/usr/lib64/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(self.args, **self.kwargs)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 664, in process_resource_set
if not ecs_taggable(self.manager.resource_type, r):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 44, in ecs_taggable
path_parts = r[model.id].rsplit(':', 1)[-1].split('/')
KeyError: 'containerInstance'
2020-05-19 16:53:59,749: custodian.commands:ERROR Error while executing policy ecs-verify-container-instances-ami, continuing
Traceback (most recent call last):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/commands.py", line 281, in run
policy()
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/policy.py", line 1062, in call
resources = mode.run()
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/policy.py", line 323, in run
results = a.process(resources)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 426, in process
self.process_resource_set, self.id_key, resources, tags, self.log)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/tags.py", line 133, in _common_tag_processer
raise error
File "/usr/lib64/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(self.args, **self.kwargs)
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 664, in process_resource_set
if not ecs_taggable(self.manager.resource_type, r):
File "/home/<user>/custodian/lib64/python3.7/site-packages/c7n/resources/ecs.py", line 44, in ecs_taggable
path_parts = r[model.id].rsplit(':', 1)[-1].split('/')
KeyError: 'containerInstance'
policies:
Hi all, we have multip;e cloudtrails in account, I want to see if a specific name trail is enabled or not. Here is my policy.
policies:
- name: "cloudtrail-enable-trail"
resource: cloudtrail
- type: value
key: Name
value: Log-Trail
- type: value
key: Name
value: Trails
Am I missing anything here. I only want the accounts that these specific trails doesn't exist
"Environment": {
"Variables": {
"Bucketname": "some-input",
"TopicArn": "arn:aws:sns:us-east-1:12345678910:testSNS",
"TopicId": "OutSNS"
"SomeOtherVar": "AKHH34HSF8YS748"
}
},
Hi, I'm working on setting up cloud custodian against our test account in AWS. I'd like to have it clean up EC2 instances after some period of time, but I'd like that timing to be somewhat granular.
Ideally, I would like to tag EC2 instances with tag:cloud-custodian
and values like 30m
, 1h
, 6h
, 3d
, etc., and use that as a policy to terminate the EC2 instances after 30 minutes, 1 hour, 6 hours, 3 days, etc.
Is that a supported use-case for this tool?
think I'm not quite understanding the conditions
correctly. i'm trying to get a policy to only execute with accounts defined with c7n-org
with tag type:prod
policies:
- name: elb-no-instances
resource: elb
conditions:
- type: value
key: type
value: "prod"
op: eq
filters:
- Instances: []
and here's a snip from my org yml
- account_id: "123123123"
name: prod
tags:
- type:prod
regions:
- us-east-1
- us-east-2
- eu-west-1
- ap-northeast-1
- ap-northeast-2
role: arn:aws:iam::123123123:role/c7n-role
is there a way to detect vpc network acls which allow wide open access (eg like the default vpc rules shown here: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html)?
looked at the vpc network acl docs but didn't see anything useful there.
@kapilt
I am working on a project where we are using AWS Config Managed rules for compliance visibility and because they cover a lot of the use cases we want.
Because there is no native integration between config and Security Hub and so we can build in some automated responses we are using c7n policies.
At the moment, there is no functionality within c7n to enable the managed config rules. This is easy to achieve in cfn or terraform. Would the ability to enable a named Config managed rule from within a c7n policy be a valid use case.
It would allow users to define everything within their policy and if there was no managed config rule they can define a custom config rule using c7n.
What would be your view?
@all - I got the my script posted which parses your current custodian schema to extract all resource types, then runs generic scan policies against all those resource types it discovers and then generates csv and json reports for all resources as well as an expanded schema and uploads all the report files to your S3 bucket. These JSON reports are EXTREMELY useful for writing policies as you can see the keys you can filter on and what data is available.
To run this you would copy the script to your custodian environment, run a chmod +x sciptname.sh to make it executable, edit the S3 bucket variable in the script and then run it. Assumes you are using c7n-org. If you are not using c7n-org you can modify the c7n-org commands to use custodian run with your parameters. Let me know if you have any questions and enjoy! Cheers! :beers:
Hi, Running into an issue regarding IAM users with 2 access Keys
Filter :
- and:
- type: credential
key: access_keys.last_used_date
value_type: age
value: 80
op: greater-than
- type: credential
key: access_keys.active
value: true
The resource.json is bringing result as
"access_keys": [
{
"active": true,
"last_rotated": "2019-03-07T15:28:09+00:00",
"last_used_date": "2020-04-23T02:53:00+00:00",
"last_used_region": "us-west-2",
"last_used_service": "s3",
"c7n:match-type": "credential"
},
{
"active": false,
"last_rotated": "2019-09-10T09:41:02+00:00",
"last_used_date": "2019-10-03T15:00:00+00:00",
"last_used_region": "us-west-2",
"last_used_service": "s3",
"c7n:match-type": "credential"
}
Query is that why i am getting this resource as none of the access key satisfy both conditions. Is it working as "or" condition for both of the access keys
@jtroberts83 @kapilt Please share your thoughts
notify
that can look at all resources, even previously failed ones, so we can report on failures. The PR is going to be pretty large, as I'm refactoring every action in every resource ... using common code for many things, etc. if you're interested, please review ... comments welcome as i continue to walk through all the resources and update their actions. :) cloud-custodian/cloud-custodian#5790
Schema Diff for c7n 0.9.1.0 - 0.9.2.0 Has been generated here - https://github.com/jtroberts83/Cloud-Custodian/blob/master/schema_expander/AWS-0.9.1.0-0.9.2.0-Changes.md
A new version, awesome!
{now} UTC
value but it is tagging the date that the policy was created, not when the lambda CreateFunction was triggered. I was told there is an issue open on this but didn't find it. Any plans to get that fixed as it will solve some big problems for us with enforcing on resources which don't have a CreationDate in their metadata
error:An error occurred (ValidationException) when calling the DescribeElasticsearchDomains operation: Please provide a maximum of 5 Elasticsearch domain names to describe.