Rules engine for AWS management, DSL in yaml for query, filter, and actions on resources
thisisshi on event-rule-deletion-target-filter
thisisshi on master
aws.event-rule - add delete wit… (compare)
thisisshi on event-rule-deletion-target-filter
aws - kinesis video streams res… azure - resolve arm tagging iss… core - enable richer policy met… and 1 more (compare)
thisisshi on event-rule-deletion-target-filter
drive by for #6494 (compare)
Hi, I'm working on setting up cloud custodian against our test account in AWS. I'd like to have it clean up EC2 instances after some period of time, but I'd like that timing to be somewhat granular.
Ideally, I would like to tag EC2 instances with tag:cloud-custodian
and values like 30m
, 1h
, 6h
, 3d
, etc., and use that as a policy to terminate the EC2 instances after 30 minutes, 1 hour, 6 hours, 3 days, etc.
Is that a supported use-case for this tool?
think I'm not quite understanding the conditions
correctly. i'm trying to get a policy to only execute with accounts defined with c7n-org
with tag type:prod
policies:
- name: elb-no-instances
resource: elb
conditions:
- type: value
key: type
value: "prod"
op: eq
filters:
- Instances: []
and here's a snip from my org yml
- account_id: "123123123"
name: prod
tags:
- type:prod
regions:
- us-east-1
- us-east-2
- eu-west-1
- ap-northeast-1
- ap-northeast-2
role: arn:aws:iam::123123123:role/c7n-role
is there a way to detect vpc network acls which allow wide open access (eg like the default vpc rules shown here: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html)?
looked at the vpc network acl docs but didn't see anything useful there.
@kapilt
I am working on a project where we are using AWS Config Managed rules for compliance visibility and because they cover a lot of the use cases we want.
Because there is no native integration between config and Security Hub and so we can build in some automated responses we are using c7n policies.
At the moment, there is no functionality within c7n to enable the managed config rules. This is easy to achieve in cfn or terraform. Would the ability to enable a named Config managed rule from within a c7n policy be a valid use case.
It would allow users to define everything within their policy and if there was no managed config rule they can define a custom config rule using c7n.
What would be your view?
@all - I got the my script posted which parses your current custodian schema to extract all resource types, then runs generic scan policies against all those resource types it discovers and then generates csv and json reports for all resources as well as an expanded schema and uploads all the report files to your S3 bucket. These JSON reports are EXTREMELY useful for writing policies as you can see the keys you can filter on and what data is available.
To run this you would copy the script to your custodian environment, run a chmod +x sciptname.sh to make it executable, edit the S3 bucket variable in the script and then run it. Assumes you are using c7n-org. If you are not using c7n-org you can modify the c7n-org commands to use custodian run with your parameters. Let me know if you have any questions and enjoy! Cheers! :beers:
Hi, Running into an issue regarding IAM users with 2 access Keys
Filter :
- and:
- type: credential
key: access_keys.last_used_date
value_type: age
value: 80
op: greater-than
- type: credential
key: access_keys.active
value: true
The resource.json is bringing result as
"access_keys": [
{
"active": true,
"last_rotated": "2019-03-07T15:28:09+00:00",
"last_used_date": "2020-04-23T02:53:00+00:00",
"last_used_region": "us-west-2",
"last_used_service": "s3",
"c7n:match-type": "credential"
},
{
"active": false,
"last_rotated": "2019-09-10T09:41:02+00:00",
"last_used_date": "2019-10-03T15:00:00+00:00",
"last_used_region": "us-west-2",
"last_used_service": "s3",
"c7n:match-type": "credential"
}
Query is that why i am getting this resource as none of the access key satisfy both conditions. Is it working as "or" condition for both of the access keys
@jtroberts83 @kapilt Please share your thoughts
notify
that can look at all resources, even previously failed ones, so we can report on failures. The PR is going to be pretty large, as I'm refactoring every action in every resource ... using common code for many things, etc. if you're interested, please review ... comments welcome as i continue to walk through all the resources and update their actions. :) cloud-custodian/cloud-custodian#5790
Schema Diff for c7n 0.9.1.0 - 0.9.2.0 Has been generated here - https://github.com/jtroberts83/Cloud-Custodian/blob/master/schema_expander/AWS-0.9.1.0-0.9.2.0-Changes.md
A new version, awesome!
{now} UTC
value but it is tagging the date that the policy was created, not when the lambda CreateFunction was triggered. I was told there is an issue open on this but didn't find it. Any plans to get that fixed as it will solve some big problems for us with enforcing on resources which don't have a CreationDate in their metadata
error:An error occurred (ValidationException) when calling the DescribeElasticsearchDomains operation: Please provide a maximum of 5 Elasticsearch domain names to describe.
set-attributes
it would be nice to have the bucket prefix be something like this: Prefix: '{resource.DomainName}'
so that all the logs aren't dumped into the same directory and instead it pulls the DomainName value from the resource metadata and puts it as the bucket logging prefix'{resource.LastModifedTime}'
as exact creation time would be nice as well, I know the {now} option currently doesn't work for lambda run policies so this would be one solution. We have to have the creation date for enforcement as we do monthly release cycles and our policies apply to all gear created AFTER that monthly release cycle date so we required to verify a resource was created after 'X' date, otherwise we are not allowed to enforce on it at this time.Hi all,
Guys, as you know a couple weeks ago EC2 key pairs started supporting tags.
Our issue that we had ton of keys for a long time and I got an idea to clean up them, as example:
every 6 hours run CC job to get over all EC2 instances and if EC2 has a key, find this key and update tags for key: LastUsedTime.
Keys, which doesn't have a tag: LastUsedTime or LastUsedTime more than 30 days, should be removed.
Is it possible to write all this steps with CC?
https://awsapichanges.info/archive/changes/20d670-ec2.html
hey guys, howdy?
I'm trying to run a policy over all my regions using the keyword all but it's not working:
% docker run -it \
-v $(pwd)/output:/home/custodian/output \
-v $(pwd)/ec2.yml:/home/custodian/ec2.yml \
cloudcustodian/c7n run -v -s /home/custodian/output ec2.yml --region=all
2020-05-25 17:22:44,902: custodian.commands:DEBUG Loaded file ec2.yml. Contains 2 policies
Traceback (most recent call last):
File "/usr/local/bin/custodian", line 11, in <module>
load_entry_point('c7n', 'console_scripts', 'custodian')()
File "/src/c7n/cli.py", line 359, in main
command(config)
File "/src/c7n/commands.py", line 107, in _load_policies
policies += provider.initialize_policies(
File "/src/c7n/resources/aws.py", line 608, in initialize_policies
get_profile_session(options).client('ec2').describe_regions(
File "/usr/local/lib/python3.8/site-packages/boto3/session.py", line 258, in client
return self._session.create_client(
File "/usr/local/lib/python3.8/site-packages/botocore/session.py", line 831, in create_client
client = client_creator.create_client(
File "/usr/local/lib/python3.8/site-packages/botocore/client.py", line 83, in create_client
client_args = self._get_client_args(
File "/usr/local/lib/python3.8/site-packages/botocore/client.py", line 285, in _get_client_args
return args_creator.get_client_args(
File "/usr/local/lib/python3.8/site-packages/botocore/args.py", line 71, in get_client_args
final_args = self.compute_client_args(
File "/usr/local/lib/python3.8/site-packages/botocore/args.py", line 147, in compute_client_args
endpoint_config = self._compute_endpoint_config(
File "/usr/local/lib/python3.8/site-packages/botocore/args.py", line 218, in _compute_endpoint_config
return self._resolve_endpoint(**resolve_endpoint_kwargs)
File "/usr/local/lib/python3.8/site-packages/botocore/args.py", line 300, in _resolve_endpoint
return endpoint_bridge.resolve(
File "/usr/local/lib/python3.8/site-packages/botocore/client.py", line 360, in resolve
resolved = self.endpoint_resolver.construct_endpoint(
File "/usr/local/lib/python3.8/site-packages/botocore/regions.py", line 121, in construct_endpoint
result = self._endpoint_for_partition(
File "/usr/local/lib/python3.8/site-packages/botocore/regions.py", line 135, in _endpoint_for_partition
raise NoRegionError()
botocore.exceptions.NoRegionError: You must specify a region.
running the latest img version
any hint on what am I missing here?
- name: iam-user-remediate-non-compliant
resource: aws.iam-user
filters:
- type: config-compliance
rules: [securityhub-iam-user-unused-credentials-check-nv6rb]
states: [NON_COMPLIANT]
- name: disable-rds-deletion-protection
resource: rds
mode:
role: arn:aws:iam::{account_id}:role/ABCD
type: periodic
schedule: "rate(1 day)"
execution-options:
output_dir: s3://ABCD
filters:
- type: marked-for-op
op: modify-db
actions:
- type: modify-db
update:
- property: 'DeletionProtection'
value: false
- property: 'PubliclyAccessible'
value: false
policies:
- name: no_ssh_from_internet
resource: gcp.firewall
mode:
type: gcp-audit
methods:
- v1.compute.firewalls.insert
- v1.compute.firewalls.patch
filters:
- type: value
key: allowed[?IPProtocol=='tcp'].ports[]
value: "22"
op: contains
- or:
- type: value
key: "sourceRanges | {range: join(', ', @)}.range"
value: 0.0.0.0
op: contains
- type: value
key: "sourceRanges | {range: join(', ', @)}.range"
value: /0
op: contains
actions:
- type: notify
subject: no_ssh_from_internet
to:
- test@test.com
format: json
transport:
type: pubsub
topic: projects/root-cortex-265110/topics/custodian
policies:
- name: Security Hub Non compliant resources
resource: aws.iam-user
filters:
- type: finding
query:
ComplianceStatus:
- Comparison: EQUALS
Value: FAILED
Title:
- Comparison: EQUALS
Value: "1.4 Ensure access keys are rotated every 90 days or less"
RecordState:
- Comparison: EQUALS
Value: ACTIVE