Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 07:45
  • May 08 20:48
  • May 08 05:12
    tjstansell synchronize #6610
  • May 08 05:07
    ranjanashish starred cloud-custodian/cloud-custodian
  • May 08 02:15
    tjstansell synchronize #6610
  • May 08 01:37
    ajkerrigan edited #6486
  • May 08 01:37
    ajkerrigan edited #6486
  • May 08 01:37
    ajkerrigan edited #6486
  • May 08 01:36
    ajkerrigan commented #6666
  • May 07 21:16
    kylejohnson514 synchronize #6469
  • May 07 21:08
    kylejohnson514 synchronize #6469
  • May 07 20:46
    kylejohnson514 synchronize #6469
  • May 07 20:27
    kylejohnson514 synchronize #6469
  • May 07 20:13
    kylejohnson514 synchronize #6469
  • May 07 20:09
    kylejohnson514 synchronize #6469
  • May 07 19:55
    kapilt closed #6669
  • May 07 19:55
    kapilt commented #6669
  • May 07 19:52
    kylejohnson514 labeled #6669
  • May 07 19:52
    kylejohnson514 opened #6669
  • May 07 17:53
    kylejohnson514 synchronize #6469
srikanthcs
@srikanthcs
Does custodian support policies for apigatewayv2 (http and websocket api) ?
The docs have filters/actions for rest-api resource, and rest-api resource doesn't filter out http api.
indrajeet singh
@indraindrajit71:matrix.org
[m]

Hi team. Just started exploring custodian and testing on aws.
Created a simple yaml for stop and start an ec2 instance with a tag applied on the instance. Problem is this that if I stop an instance it works fine, but when I try to start it again, it gives error. To add: Before stopping the instance it was made sure that all checks has been passed in Status checks. However, after an interval of like 28 minutes approx, the same yaml gets executed successfully and the instance gets stopped or started. can someone please let me know if something wrong. I hope cusotdian checks the instances status checks post executing. Mnay thanks !

policies:
  - name: my-first-policy
    resource: aws.ec2
    filters:
      - "tag:custodian": present
    actions:    
      - stop

Error/warning it gives post starting the instance and using the same yaml with "start" in action

2021-04-02 19:47:57,891: custodian.policy:INFO policy:my-first-policy resource:aws.ec2 region:us-east-1 count:1 time:0.00
2021-04-02 19:47:57,892: custodian.actions:WARNING stop implicitly filtered 0 of 1 resources key:State.Name on running
2021-04-02 19:47:57,892: custodian.policy:INFO policy:my-first-policy action:stop resources:1 execution_time:0.00

After 28 minutes the same command gets executed successfully

custodian run  -s out custodian.yml --region=us-east-1
2021-04-02 20:11:03,541: custodian.policy:INFO policy:my-first-policy resource:aws.ec2 region:us-east-1 count:1 time:63.05
2021-04-02 20:12:05,418: custodian.policy:INFO policy:my-first-policy action:stop resources:1 execution_time:61.88
myoung34
@myoung34

Im hitting a mental wall…..

Im testing out c7n-org

Basically I have a role cloud-custodian in all my org accounts that can be assumed by a single one of them via stacksets

Thats working.
The stackset that creates the arn:aws:iam::{acct id}:role/cloud-custodian role has a policy:

            "Action": [
                "securityhub:BatchImportFindings"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }

But Im getting the error
2021-04-02 15:16:14,308: c7n_org:ERROR Exception running policy:iam-users-with-active-keys account:users region:us-east-1 error:An error occurred (AccessDeniedException) when calling the BatchImportFindings operation: User: arn:aws:sts::{acct id}:assumed-role/cloud-custodian/CloudCustodian is not authorized to perform: securityhub:BatchImportFindings on resource: arn:aws:securityhub:us-east-1::product/cloud-custodian/cloud-custodian

Is assumed-role different from my role permissions? Im not sure why its not picking up that permission

7 replies
Derek Egel
@egelnoteagle
I was wondering how I would be able to use filters to create a resources.json file that only lists s3 buckets (as well as any other resource such as ec2 instances, asg, etc...) with no tags at all
3 replies
Derek Egel
@egelnoteagle
policies:
  - name: tag-compliance
    resource: s3
    comment: | 
      Report on total count of non compliant buckets
    filters:
      - or:
        - tag:"*": absent
This returns all s3 buckets in the account
myoung34
@myoung34

@kapilt New feature request: cloud-custodian/cloud-custodian#6602

Open to feedback

Donato Azevedo
@donatoaz
Hi everyone! new to cloud-custodian here. I was wondering if there is a "pretty-print" solution that summarizes somehow the results of filters. I still have no actions set up, and am using it with a ReadOnly role. So before I start rolling my own solution to parse the resulting resources.json files, I just wanted to check with y'all
5 replies
aakshaik2
@aakifshaikh
aws.insight-rule is equivalent to AWS Cloudwatch Event Rules? Do we have aws.event-rule as separate resource types?
Donato Azevedo
@donatoaz
Is there a way I can have a policy for an ebs volume to make sure it's tagged with the same tag key and value as that of the ec2 instance it is attached to?
2 replies
Donato Azevedo
@donatoaz
Does custodian support notifying to a plain sns topic? I saw there is a generic webhook target (and probably I could do an API GW - SNS service integration easily) but just wanted to ask first
2 replies
Donato Azevedo
@donatoaz
Honest question from non-native english speaker: what's the deal with "maid_" as a prefix on tags?
6 replies
Donato Azevedo
@donatoaz
Me again... is there a way to put_metric using the value of tag as a dimension (or as a metric name for that matter)?
4 replies
Donato Azevedo
@donatoaz
Does anyone have any tips on how to manage DRYness? I'm seeing myself repeating a lot (I am implementing the tagging policy, and having to repeat many policies for each relevant resource type)
- name: ec2-instance-tagging-policy
  resource: ec2
  filter:
    - <<: *invalid_circle_tag

-name: ebs-volume-tagging-policy
  resource: ebs
  filter:
    - <<: *invalid_circle_tag

...
2 replies
cleo2525
@cleo2525

I'm trying to filter for s3 bucket policies that contain certain actions. I know I can use the 'has-statement' filter with 'action' option. But I was wondering if I could use the generic 'value' filter with 'op: regex' so I don't have to list out every action I am filtering for.

I pulled the available keys from the 's3' resource and I see the bucket policy is contained in the "Policy" key. I want to narrow my filter down to just look at the Action section of the policy when applying the regex. But I'm not sure how to specify that in the filter. The following filter isn't working.

policies:
  - name: s3-filter-test
    resource: s3
    filters:
      - type: value
        key: Policy.Statement.Action
princeshalem
@princeshalem
I'm trying to write a policy, which will automatically create a CloudWatch Event Rule triggered Lambda function in my account. But my policy is validate successfully . but it does not trigger or mail to me.
princeshalem
@princeshalem
olicies:
  • name: invalid-ip-address-login-detected
    resource: account
    description: |
    Notifies on invalid external IP console logins
    mode:
    type: cloudtrail
    role: CloudCustodian-QuickStart
    events:
      - ConsoleLogin
    filters:
    • not:
      • type: event
        key: 'detail.sourceIPAddress'
        value: |
        '^((158.103.|142.179.|187.39.)([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])
        .([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5]))|(12.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])
        .([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5]).([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5]))$'
        op: regex
        actions:
    • type: notify
      template: default.html
      priority_header: 1
      subject: "Login From Invalid IP Detected - [custodian {{ account }} - {{ region }}]"
      violation_desc: "A User Has Logged In Externally From A Invalid IP Address Outside The Company's Range:"
      action_desc: |
      "Please investigate and revoke the invalid session along
      with any other restrictive actions if appropriate"
      to:
3 replies
pentagonal-proboscis
@pentagonal-proboscis

Hi there - I have created a policy in an attempt to return all AWS iam users + their credentials as follows:

policies:
- name: iam-users-inventory
  resource: iam-user
  region: us-east-1
  filters: 
    - or:
      - type: credential
        key: password_enabled
        value: false
      - type: credential
        key: password_enabled
        value: true

However, this doesn't return every user in my account - is there something I am obviously doing wrong here?

5 replies
Dave Van
@dvan1_gitlab

Hi, has anyone tried out azure multi-cloud support in the most recent release? Is it only supported in custodian or does c7n_org have support for it as well? I'm attempting to call ./c7n-org run -c azure_gov_subscription.yml -s output -u my_policy.yml --dryrun --region=AzureUSGovernment --verbose and it seems to produce an error correlating to commercial authentication endpoints login.microsoftonline.com instead of gov ones.

azure_gov_subscription.yml has the subscription name and ID that is located in Azure US Gov Cloud
my_policy.yml is a simple policy that should return the # of virtual machines under the specified subscription in azure_gov_subscription.yml

9 replies
cleo2525
@cleo2525

I'm having trouble getting the cross-account whitelist_conditions filter to work on my s3 policies. I have a test policy like this

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "OrgReadAccess",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": [
                "arn:aws:s3:::devprivbucket-#######",
                "arn:aws:s3:::devprivbucket-#######/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:PrincipalOrgID": "o-##########"
                }
            }
        }
    ]
}

And my filter looks like this

policies:
  - name: public-access-policy-s3-mark
    resource: s3
    description: |
      tag bucket with public read access policy and send notifications
    filters:
      - type: check-public-block
      - type: cross-account
        everyone_only: true
      - type: cross-account
        whitelist_conditions: ["StringEquals"]

When I run this filter, it matches on buckets with the whitelisted condition. My understanding is that 'whitelisted_conditions" should make the filter skip the bucket policy

10 replies
Ben Khmelnytskiy
@benkhmelnytskiy

hey,
does anybody know why it happen?

2021-04-06 17:31:21,716: c7n.policies:WARNING policy:SecCCKmsRule resources:kms-key not available in region: us-east-1
2021-04-06 17:31:21,716: custodian.commands:WARNING Empty policy file(s). Nothing to do.

I'm trying to run custodian in lambda container and facing the above error; but when running the same policy from command line with exactly the same parameters - it works without any errors

1 reply
pentagonal-proboscis
@pentagonal-proboscis
Is it possible to specify which order c7n policies run in? Or to specify dependencies between them? e.g. run X after Y has finished?
10 replies
Jimmyd84
@Jimmyd84

For sqs custodian policy has anyone tried to used a cross account key for encryption? I tried and it didn't work.
actions:

   - type: set-bucket-encryption
     crypto: aws:kms
     key: xxxx

I added the full cross account alias and it appended the local account number in front of it.

3 replies
Jothibasu
@jblinuxclicks
Hi All.. i am new to Cloud-Custodian
Any one could you please help me to find out the AMi and Parents AMI to filter out the ec2 instance and stop action for the same
So we can enforce users should use the new AMI on every 60days, Please help me out
3 replies
rkalva
@rkalva

Hi All. I have a requirement where am trying to filter the data for nested JSON values. The value am trying to filter is "Type" and "Protocol" under the DefaultActions. Can anyone help. Below is the JSON and the syntax am trying which is unfortunately not working.

"c7n:MatchedListeners": [
      {
        "ListenerArn": "arn:aws:elasticloadbalancing:us-west-2:181167:listener/app/testhttpredirect/e2af8aa22/1c9c2c9c",
        "LoadBalancerArn": "arn:aws:elasticloadbalancing:us-west-2:181167:loadbalancer/app/testhttpredirect/e2af8aa22",
        "Port": 80,
        "Protocol": "HTTP",
        "DefaultActions": [
          {
            "Type": "redirect",
            "Order": 1,
            "RedirectConfig": {
              "Protocol": "HTTPS",
              "Port": "443",
              "Host": "#{host}",
              "Path": "/#{path}",
              "Query": "#{query}",
              "StatusCode": "HTTP_301"
            }
          }
        ]

Filter:

- type: listener
  key: DefaultActions.Type
  value: "redirect"
  op: eq
4 replies
Mike
@mikejgray
Value filter question...kind of random, but has anyone identified a way to do something like a csv2dict from a file but use the values as RegEx filters? Not that we need more sub-filters...but this would be handy for a particular use case we have.
2 replies
mogmismo
@mogmismo_twitter
with the aws.ec2 resource, I would like to use cloudtrail execution mode on the RunInstances event, but be able to use the reduce filter on the total number of instances in the account, not on the number of instances in the event itself (as found by the ids). Is this possible?
8 replies
Deborshi Choudhury
@choudhde
API gateway question... I need to implement a PRIVATE api gateway which should have a specific policy document, it should only allow traffic from a specific centralized VPC endpoint . Need to figure out if it's possible with cloud custodian. Any help would be much appreciated. Attaching the describe api ..
  {
    "id": "xxxxxxxxx",
    "name": "ssm_support_api",
    "description": "API supporting centralizaed SSM EC2 registration.",
    "createdDate": "2021-03-22T16:14:13+00:00",
    "apiKeySource": "HEADER",
    "endpointConfiguration": {
      "types": [
        "PRIVATE"
      ]
    },
    "policy": "{\\\"Version\\\":\\\"2012-10-17\\\",\\\"Statement\\\":[{\\\"Effect\\\":\\\"Deny\\\",\\\"Principal\\\":\\\"*\\\",\\\"Action\\\":\\\"execute-api:Invoke\\\",\\\"Resource\\\":\\\"arn:aws:execute-api:us-east-1:xxxxxxxxx:xxxxxxxxx\\/*\\\",\\\"Condition\\\":{\\\"StringNotEquals\\\":{\\\"aws:SourceVpce\\\":\\\"vpce-xxxxxxxxxxxxxx\\\"}}},{\\\"Effect\\\":\\\"Allow\\\",\\\"Principal\\\":\\\"*\\\",\\\"Action\\\":\\\"execute-api:Invoke\\\",\\\"Resource\\\":\\\"arn:aws:execute-api:us-east-1:xxxxxxxxx:xxxxxxxxxx\\/*\\\"}]}",
    "Tags": [],
    "c7n:MatchedFilters": [
      "policy"
    ]
  }
]
3 replies
jmahowald-slalom
@jmahowald-slalom
I'm not seeing anywhere if we could within the same policy specify both a config rule that is triggered on both config changes as well as periodic. This makes it hard from what I can see to update the policy for instance for IAM roles. We wanted to exclude service based roles and updated the policy. But I haven't successfully figured out how to rerun config for those items already marked non compliant without something like that ability.
Christopher Pitstick
@cpitstick-argo
Is there a way to get Cloud Custodian to consume Prometheus metrics on AWS instead of Amazon CloudWatch? We're using Node Exporter from Kubernetes clusters and we'd like to use Prometheus metrics to avoid the high cost of large quantities of AWS CloudWatch metrics.
1 reply
jfricioni
@jfricioni
Is there a document/video/etc on deploy custodian policies through github and github actions or the like? Trying to build out an actual pipeline to automate everything for my company and IT users who want to deploy policies out without having a dozen people ssh to an ec2 instance to deploy.
Kristina Trump
@KristinaTrump_twitter
@kapilt , can we tag the custodian policies while they getting deployed
8 replies
myoung34
@myoung34

Is there a way (c7n-org) to not duplicate some things by region?
the rule:

  policies:
    - name: IAM User with active access key
      resource: iam-user
      filters:
        - type: access-key
          key: Status
          value: Active
      actions:
        - type: post-finding
          severity_normalized: 5
          types:
            - "Software and Configuration Checks/AWS Security Best Practices"

duplicates as it runs across each region even though IAM is global

c7n_org:INFO Ran account:sandbox region:us-east-1 policy:IAM User with active access key matched:3 time:2.32
c7n_org:INFO Ran account:sandbox region:us-west-2 policy:IAM User with active access key matched:3 time:3.86
c7n_org:INFO Policy resource counts Counter({'root-user-login-detected': 396, 'IAM User with active access key': 6})
5 replies
smithjamiej
@smithjamiej
quick question I can't seem to find the answer for: I have setup c7n-org and can run reports across accounts, however I have a AWS SQS queue (which works without encryption), however when I add a KMS key, c7n-org tries to use the default KMS Key for the account where the SQS and KMS key live. Is there a place in accounts.yml or mailer.yml where I can specify which KMS key to choose, instead of the default? (I can't seem to find the answer anywhere) (I am using the SQS queue to send messages to slack)
4 replies
myoung34
@myoung34

Ive stumbled on some old school stuff that says that AssumeRoleWithSAML cant be used as a cloudtrail trigger, is this still true?

cc @jtroberts83 https://gitter.im/cloud-custodian/cloud-custodian?at=5eb193d3a9de3d01b1ecae4a

1 reply
aakshaik2
@aakifshaikh
Quick Question- I have deployed the policies that are 2 years old and running just fine. But they are running on runtime python 2.7. How do i update the runtime for those lambda functions? Do I have to make minor changes to the policies and redploy again? will that change the runtime to newer version? Because all latest policies that got deployed have latest runtime python 3.8.
8 replies
Deborshi Choudhury
@choudhde
@kapilt is there any roadmap for supporting "has-statement" for services that support resource policies, such as api gateway, sqs, sns and such?
5 replies
aakshaik2
@aakifshaikh
Question- For AZURE Cloud Provider - We are using the SumoLogic as SIEM solution and we want to ingest the custodian output results from azure blob into Sumo. We recently found that it is not possible to ingest gzip file. here is the below response-
I am extremely sorry to say that .gzip files is currently not supported. Sumo Logic support only .csv, .log, .blob and .json as mention in the following
doc and as result you are seeing this error.
It is assumed that:

- The Azure service updates the blob (adding new blocks) in small chunks and has been tested with block blobs.


- Log files have a file extension of .csv, .json, .blob, or .log.
---
** In .csv files, it is assumed the delimiting character is a comma (,). The .csv files are converted to JSON and sent to Sumo.
--- ** If the file is .json, the JSON objects are extracted and sent to Sumo.
--- ** If the file is .blob, the JSON objects are extracted and sent to Sumo.
--- ** If the file is .log, log lines are sent to Sumo as is.
Any advise or alternate approach? Anyone?
aakshaik2
@aakifshaikh
Can someone please verify if my policy is correct to identify the acm certificates thats expiring in 60days. When I check the results - NotAfter value also includes the past date (means march). Does it mean it also identify the onces that already got expired? I think I want to have 2 separate policies- 1) expired certificates [value: 0 in the below filters] and 2) expiring in 60 days [value: 60 in the below filters]
5 replies
policies:

- name: sec-n-acm-certificate-expiring-in-60-days
  resource: aws.acm-certificate
  comment: |
    Notify of any ACM Certificates that will be expiring within 60 days.
  filters:
    - type: value
      key: NotAfter
      value_type: expiration
      op: lt
      value: 60
  mode:
    schedule: "rate(3 days)"
    type: periodic
    execution-options:
      output_dir: s3://xxxxx/xxxxx/xxxxx/{account_id}/
    tags: *sec-tags
    runtime: python3.8
myoung34
@myoung34

Has anyone ever done any policies to look for specific things in trust policies for roles?

Id like to flag roles that have a certain account in the trust policy

4 replies
Purushotham Reddy
@purushothamkdr453
Hi All, I am novice to Cloud custodian, I am working on a use case i.e writing a cloud custodian policy to look for a security groups that have broader access port range i.e difference between starting port and ending port should be less than 1000, does jmespath support arithmetic operaors, if so can someone provide me an example?
1 reply
Callum Hibbert
@CallumHibbert

I'm trying to write a policy to remove rules for permissive network Security Groups, except for those SGs efined in a whitelist.

I was following this article: https://stackoverflow.com/questions/59375009/restrict-only-specific-ports-in-specific-security-groups-using-cloud-custodian

This is what I have so far but it is not working, it is ignoring the whitelist and removing any permissive SG rule.

- name: policy-security-groups-ingress-risk
  description: ...
  resource: security-group
  filters:
  - and:
    - type: ingress
      OnlyPorts:
      - 80
      - 443
      Cidr:
        value: 0.0.0.0/0
    - type: value
      key: GroupId
      op: in
      value_from:
        url: s3:/...csv
        format: csv
        expr: 0 # column index in CSV
  actions:
  ...
  mode:
    events:
    - source: ec2.amazonaws.com
      event: AuthorizeSecurityGroupIngress
      ids: "requestParameters.groupId"
    - source: ec2.amazonaws.com
      event: AuthorizeSecurityGroupEgress
      ids: "requestParameters.groupId"
    - source: ec2.amazonaws.com
      event: RevokeSecurityGroupEgress
      ids: "requestParameters.groupId"
    - source: ec2.amazonaws.com
      event: RevokeSecurityGroupIngress
      ids: "requestParameters.groupId"
    ...
    type: cloudtrail

Any pointers? Thanks.

2 replies
Saurav Patar
@sauravpatar

Hi Guys,
I am trying to enable cloudtrail logging to one of my aws account using custodian policy. Currently the bucket is already created using cloudformation and will be used to trail logs. Below is my policy.

policies:
  - name: enable-cloudtrail-logging
    comment: This policy is used to create cloudtrail logging for the account 
    resource: account
    actions:
    - type: enable-cloudtrail
      trail: mytrail
      bucket: <bucket-name>

When I am trying to run it, I am getting exceptions as this.
botocore.exceptions.ClientError: An error occurred (InvalidLocationConstraint) when calling the CreateBucket operation: The specified location-constraint is not valid
2021-04-14 12:58:16,743: custodian.commands:ERROR The following policies had errors while executing
enable-cloudtrail-logging
Any help will be appreciated. Thanks

6 replies
Caio Henrique Aviz de Souza
@caiohasouza

Hello guys, i'm trying create a s3 tag compliance but i don't understand why not works, my configs

name: s3-tag-compliance
comments: S3 Tag Compliance
resource: aws.s3
description: |
  Find all S3 buckets that are not conformant to tagging policies.
filters:
  - or:
  - type: value
    key: "tag:product"
    op: ni
    value:
      - "test1"
      - "test2"

There are something wrong?

9 replies
aakshaik2
@aakifshaikh
I made some minor changes to all the policies by introducing the statement of runtime python3.8. Merged the PR and this triggered the concourse pipeline to deploy the changes. The pipeline finished in GREEN since it completed all its tasks. However, when I checked the details- A lot of policies shows “skipping policy due to execution conditions”. Not sure why? Seeing this for the first time. I noticed when i went into the console that the lambda functions are NOT reflecting the runtime as mentioned in the policies. There are no conditions defined in the policy as such. The same policies just got deployed fine into other accounts via same pipeline
9 replies
myoung34
@myoung34

I want to delete cloudwatch log groups by prefix and not seeing any examples

All the examples use last-write, etc all defined here: https://github.com/cloud-custodian/cloud-custodian/blob/002fa72de3cb3faf6309b0608a4f4216c2f1d571/c7n/resources/cw.py

The problem here is that the only thing I see to filter by thats obvious to me are tags, but in this scenario none of the log groups are tagged

(I want to clear out log groups for cloudcustodian log groups /aws/lambda/custodian-*, and cloud-custodian lambda does not tag the log groups it creates)

12 replies
aakshaik2
@aakifshaikh
Question- As we keep writing more custodian policies, we found that we have exceeded the quota limit on event-rules (from the default number 300). Is there a way by which we can share the same event rule to trigger with multiple lambda functions? any advise.
3 replies
Ben Khmelnytskiy
@benkhmelnytskiy

hello guys,
trying to run c7n-org tool in lambda, but getting next error:
OSError: [Errno 30] Read-only file system: '/home/sbx_user1051'

I know that in lambda we can write only to /tmp/ folder, but c7n-org for some reason is trying to write unknown data for me to other directories

13 replies