by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 11:58
    babitha-mathew edited #5938
  • 11:55
    babitha-mathew opened #5938
  • 11:55
    babitha-mathew labeled #5938
  • Jul 08 21:26
    anovis commented #5934
  • Jul 08 20:50
    JohnHillegass commented #5934
  • Jul 08 20:49
    JohnHillegass commented #5934
  • Jul 08 20:49
    JohnHillegass commented #5934
  • Jul 08 20:40
    KVInventoR opened #5937
  • Jul 08 20:40
    KVInventoR labeled #5937
  • Jul 08 20:39
    kylejohnson514 labeled #5936
  • Jul 08 20:39
    kylejohnson514 opened #5936
  • Jul 08 20:36
    kapilt opened #5935
  • Jul 08 20:15
    PratMis commented #5934
  • Jul 08 20:03
    PratMis commented #5934
  • Jul 08 19:52
    todd-davenport edited #5930
  • Jul 08 18:50
    KVInventoR commented #5917
  • Jul 08 18:32
    todd-davenport synchronize #5930
  • Jul 08 18:06
    anovis synchronize #5934
  • Jul 08 17:53
    anovis synchronize #5934
  • Jul 08 17:19
    anovis synchronize #5934
Kristina Trump
@KristinaTrump_twitter
@kapilt , is there a way to take an EC2 AMI backup before running below type of action, actions:
  - type: terminate
    force: true
3 replies
sl805
@sl805
Does anybody know if cloud-custodian supports ignore_errors: yes or is it possible to implement such feature ? It might be useful for periodic garbage-collection tasks. Idea is to allow lambda to delete some portion of a stack until all target resources are eventually deleted
Antoni Gierczak
@antoni-g

hey all, I must be doing something wrong, but is there anything different about the syntax for resources that are non ec2 for filtering for missing tags?

policies:
  - name: ec2-tag-compliance
    resource: ec2
    comment: |
      Report on total count of non compliant instances
    filters:
      - or:
....

gives me everything I want

policies:
  - name: sqs-tag-compliance
    resource: sqs
    comment: |
      Report on total count of non compliant instances
    filters:
      - or:
....

gives me nothing

vikas1055
@vikas1055

Hi All, I have one policy for security group manual edit event this policy works fine. but I want to ignore event for our pipeline users. I added our pipeline users in filters with - not option but custodian still capturing event for pipeline users and sending alerts.
policies:

  • name: get-ec2-manual-sg-creation
    resource: security-group
    description: |
    Trap events wherein a Security Group is not created via one of the prescribed
    jenkins automation job.

    mode:
    type: cloudtrail

    events:

    - source: ec2.amazonaws.com
      event: CreateSecurityGroup
      ids: "responseElements.groupId"

    filters:

    • not:
      • type: event
        key: "detail.userIdentity.arn"
        value: -my-user
        op: contains
      • type: event
        key: "detail.userIdentity.principalId"
        value: 'ABCO'
        op: contains
      • type: event
        key: "detail.userIdentity.arn"
        value: -shared-test-cdk
        op: contains

    actions:

Can someone suggest what change can ignore my jenkins pipeline user to track by custodian here?
sl805
@sl805
@kapilt Hi. Can I somehow extract list of resources-supported by cloud-custodian programmatically ? I'm working on a policy generator, so IO want to ensure that I'll allways will loop through supported list of resources
Kapil Thangavelu
@kapilt
@sl805 custodian schema —outline —json
sl805
@sl805
@kapilt would it be possible to do the same programmatically from a script? I mean with importing c7n, and then using corresponding method ?
@kapilt Thanks, that also works perfectly.
KarlCF
@KarlCF
the command to run c7n-org only on specific accounts is c7n-org run -a (insert account here) ?
Harish Navnit
@tinvaan
hey, is there a definitive guide for authoring cloudtrail policies? basically, I always have trouble looking up the (source, event, ids) when defining a cloudtrail evnet for c7n.
4 replies
^any takers ?
KVInventoR
@KVInventoR
Guys, when are you going to release a new version?
1 reply
tomarv2
@tomarv2
anyone, what am I missing:
policies:           
  - name: rds-offhours-stop
    resource: rds
    #mode: *hourly-mode
    region: us-west-2
    tags:
      - cost:downtime
    filters:
      - "tag:c7n_do_not_shut_down": absent
      - type: offhour
        weekends: true
        default_tz: "pst"
        offhour: 17
        tag: c7n_off_hours
custodian.policy:INFO Skipping policy:rds-offhours-stop due to execution conditions
tomarv2
@tomarv2
@kapilt any suggestion for above
8 replies
mpradeep23
@mpradeep23
Hey @kapilt I have a question regarding the thread we have been discussing on github-cloud-custodian/cloud-custodian#5921, So I see that the lambda function created with custodian prefix isn't attached to any vpc or security group. So is it possible to add vpc and sg in our policy yaml file? Could I refer any example policy yaml file?
2 replies
Engineer Francis
@Onamemba
Hi,Please help! How can I parameterize the role in the config.json, I want to execute the lambda with different rolearn parameters?
2 replies
Engineer Francis
@Onamemba
@kapilt
Kristina Trump
@KristinaTrump_twitter
@jtroberts83 , Using CC is there a way to identify whether the AWS Config is enabled or not
2 replies
David Xiao
@davxiao_twitter
What did I miss? Lambda function is not triggered at all. Policy is deployed. New deployment on a new test account. Cloudtrail is enabled on the same region.
3 replies
Ajay Misra
@ajmsra
I have this policy for azure but its giving me an error does anyone know what am I doing wrong here
policies:
  - name: storage-container-public
    description: |
      Find all containers with public access enabled
    resource: azure.storage-container
    filters:
      - type: value
        key: properties.publicAccess
        op: not-equal
        value: None   
    actions:
      - type: set-public-access
        value: None
4 replies
@kapilt ^^
tomarv2
@tomarv2
@ajmsra this is what I am using and it works:
  - name: custom-daily-storage-container-public
    description: |
      Find all containers with public access enabled
    resource: azure.storage-container
    mode:
      type: container-periodic
      schedule: '0 5 * * 1'
    filters:
      - type: value
        key: properties.publicAccess
        op: not-equal
        value: None   # Possible values: Blob, Container, None
    actions:
      - <<: *notify
        subject: "[Azure Policy] [Subscription: {{ account_id }}] Container(s) with public access enabled"
        violation_desc: "Container(s) with public access enabled"
the only difference I see is the action part
htalkad
@htalkad
Using the cloud custodian version 0.8.45.3 I had a tool to pull details about resources I had in Azure. Had not used it in few days and now I see the following exception when running through c7n-org. Trying to find out what is causing this error. I have not changed anything in my tool.
11 replies
Traceback (most recent call last):
File "/usr/local/bin/c7n-org", line 33, in <module>
sys.exit(load_entry_point('c7n-org==0.5.6', 'console_scripts', 'c7n-org')())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 829, in call
return self.main(args, kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback,
ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(
args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/c7n_org/cli.py", line 660, in run
account_region_pcounts, account_region_success = f.result()
File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
return self.get_result()
File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in
get_result
raise self._exception
KeyError: 'Invalid resource: webapp for provider: azure'
Ajay Misra
@ajmsra
@tomarv2 yea I was hoping to set public storage containers to private but it just doesn't seem to work correctly not sure why
3 replies
htalkad
@htalkad
This message was deleted
3 replies
Kapil Thangavelu
@kapilt
custodian 0.9.4.0 released (along with all new versions of other packages) release notes - https://github.com/cloud-custodian/cloud-custodian/releases/tag/0.9.4.0
pendyalal
@pendyalal
@kapilt , so when we specify delay in our custodian policy, the lambda is going to wait until the delay time specified or lambda is going to trigger after the delay period
4 replies
Ajay Misra
@ajmsra
I just opened a bug related to azure cloud-custodian/cloud-custodian#5932
2 replies
@kapilt ^
Dwayne Lee
@dwayne-lee

Noob here. Can someone help me understand why I am receiving an email when testing with the custodian command...but no email ever comes when I use the c7n-org command:

c7n-org run -c accounts.yml -u testing/mail-test-policy.yml -s output
2020-07-07 16:52:31,622: c7n_org:INFO Ran account:AWS_PROD-CorporateItHosting-10022 region:us-east-1 policy:c7n-mailer-test matched:1 time:4.28
2020-07-07 16:52:31,640: c7n_org:INFO Policy resource counts Counter({'c7n-mailer-test': 1})

custodian run -c testing/mail-test-policy.yml -s output
2020-07-07 16:54:02,262: custodian.policy:INFO policy:c7n-mailer-test resource:sqs region:us-east-1 count:1 time:1.63
2020-07-07 16:54:03,372: custodian.actions:INFO sent message:5e1fa293-b34a-441a-b45b-83bc95f4eafa policy:c7n-mailer-test template:default count:1
2020-07-07 16:54:03,373: custodian.policy:INFO policy:c7n-mailer-test action:notify resources:1 execution_time:1.11

3 replies
malkochoglu
@malkochoglu
Hi. Are there any CLI options to collect only resources that matches filter? Instead of reading output file or console?Thx
7 replies
pendyalal
@pendyalal
@kapilt, For SageMaker resource, we have a different resource types, it would be nice if all sagemaker-job,sagemaker-endpoint,sagemaker-notebook are under one resourcetype sagemaker. For tag compliance checks we have to deploy 3 eventrules and 3 lambda's.
7 replies
tomarv2
@tomarv2
@kapilt there is no MSK support is 8.46.1 did I get it right?
2 replies
htalkad
@htalkad
This message was deleted
2 replies
sl805
@sl805
@kapilt Hi do you know if there's a refernces on custodian policy variables ? I need something similar to {now} or $date
5 replies
@kapilt Please disregard that, I faound it. Sorry for baothering you
sl805
@sl805
@kapilt Can I somehow extract an arbitrary field value from cloudtrail-event in cloudtrail mode ? I mean something like:
- type: tag
   tag:  Foo
   value: "{event.detail.object.foo}"
2 replies
KVInventoR
@KVInventoR

Hi all,

I am trying to find a solution for next issue:
We have a couple of CC policies in cloudtrail mode and most of this policies included: put tags and terminate EMR clusters.
My with:

[ERROR] InvalidRequestException: An error occurred (InvalidRequestException) when calling the AddTags operation: Tags cannot be modified on terminated clusters.
Traceback (most recent call last):
  File "/var/task/custodian_policy.py", line 4, in run
    return handler.dispatch_event(event, context)
  File "/var/task/c7n/handler.py", line 176, in dispatch_event
    p.push(event, context)
  File "/var/task/c7n/policy.py", line 1135, in push
    return mode.run(event, lambda_ctx)
  File "/var/task/c7n/policy.py", line 460, in run
    return self.run_resource_set(event, resources)
  File "/var/task/c7n/policy.py", line 490, in run_resource_set
    results = action.process(resources)
  File "/var/task/c7n/tags.py", line 427, in process
    _common_tag_processer(
  File "/var/task/c7n/tags.py", line 135, in _common_tag_processer
    raise error
  File "/var/lang/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/var/task/c7n/resources/emr.py", line 175, in process_resource_set
    self.retry(client.add_tags, ResourceId=r['Id'], Tags=tags)
  File "/var/task/c7n/utils.py", line 373, in _retry
    return func(*args, **kw)
  File "/var/runtime/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/var/runtime/botocore/client.py", line 626, in _make_api_call
    raise error_class(parsed_response, operation_name)

Do you have any solutions where terminate action can have a delay 30seconds? just to wait when all another checks will be finished
mostly it's related to run parallel different checks...

1 reply
bhuvanesh_kj
@bhuvaneshkj
image.png
Hi all,
I am trying to deploy policies as Azure Function, I created SP and set all varibles, but still I am getting this error
raise NotImplementedError(
NotImplementedError: Service Principal credentials are the only supported auth mechanism for deploying functions.
2020-07-08 13:30:58,844: custodian.commands:ERROR The following policies had errors while executing
  • find-all-sql-databases
@kapilt Kindly please provide any inputs on this
3 replies
tomarv2
@tomarv2
@bhuvaneshkj did you set all 4 service principals output as env variables
David Xiao
@davxiao_twitter
This might be a bug but I'm not sure. custodian run --assume arn:aws:iam::845011799417:role/CloudCustodianAdminRole --output-dir output/logs policies/* --region us-east-1 --region ca-central-1 -m aws -l /cloud-custodian/policies throw out the following error when updating an existing lambda function but deploying the same policy in a new region works fine.
32 replies
htalkad
@htalkad
Getting the following error when trying to get details about gcp.vpc using c7n-org. Other resources like gcp.disk , gcp.firewall are working fine with same account, project. "Invalid value for field 'project': 'projects/custodian-test/global/networks'. Must be a match of regex '(?:(?:[-a-z0-9]{1,63}\.)*(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?):)?(?:[0-9]{1,19}|(?:[a-z0-9](?:[-a-z0-9]{0,61}[a-z0-9])?))'"> Any pointers/suggestions?
4 replies
Lauren
@llavin
Hi fellow custodians, was just wondering if anyone started seeing any (ValidationException) errors out of the blue during deploying policies? We started seeing some errors yesterday with c7n-org in us-east-1, eu-west-1 and sa-east-1 without making changes on our end: us-east-1 error:An error occurred (ValidationException) when calling the UpdateFunctionConfiguration operation: 1 validation error detected: Value 'All EC2 instances launched with attached unencrypted EBS storage shall be terminated immediately upon creation' at 'description' failed to satisfy constraint: Member must satisfy regular expression pattern: .*
3 replies
deepthimm
@deepthimm

Do I have to add all the users to SES in order to send emails if custodian detects any non- compliant resource?

actions:

    - type: notify
      template: redefault.html
      priority_header: 1
      subject: "Open Security Group Rule Created-[custodian {{ account }} - {{ region }}]"
      violation_desc: "Security Group(s) Which Had Rules Open To The World:"
      action_desc: |
          "Actions Taken:  The Violating Security Group Rule Has Been Removed As It Typically
          Allows Direct Incoming Public Internet Traffic Access To Your Resource Which Violates Our
          Company's Cloud Security Policy.  If This Ingress Rule Is Required You May Contact The Security
          Team To Request An Exception."
      to:
          - event-owner
      transport:
          type: sqs
          queue: https://sqs.eu-central-1.amazonaws.com/xxxxxxxx/custodian-sqs
          region: eu-central-1