Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Aristenio Monteiro
    @aristenio
    This option allows you to aggregate multiple matches together into one alert. Every time a match is found, ElastAlert will wait for the aggregation period, and send all of the matches that have occurred in that time for a particular rule together.
    Vamshi Krishna Santhapuri
    @rrskris
    So aggregation works on the future time, is there any way that I can apply aggregation till current time
    1 reply
    Vamshi Krishna Santhapuri
    @rrskris
    I was also working on a custom Ruletype trying to figure it out with add_data(self, data) accumulating the matched events.. I will try some more things and let you know... Thank you @aristenio for your quick help
    Rehan Ali
    @rehannali
    Hi, i have problem which need to extend as event continue. Example.
    I use aggregation for alert if count of query is less than 50 for last 2 mins. I want to implement if after 1 min if still count less than 50 then i need alert it is less than for last 3 mins and so on.
    How can i achieve this?
    4 replies
    nithishthirmul
    @nithishthirmul
    How do I send the Average Threshold value computed in email....
    1 reply
    Vasek
    @vennca
    Hello,
    can anyone help me with finding solution for these questions please?
    Denis Granha
    @denisgranha

    Hi!

    I tried to figure this out by reading the documentation and past issues but couldn't find the answer, it seems to not be possible at the moment.

    We have a rule with the following definition:

    # (Required)
    # Type of alert.
    # the frequency rule type alerts when num_events events occur with timeframe time
    type: any
    
    # (Required)
    # Index to search, wildcard supported
    index: logstash-*
    include: ["kubernetes.labels.app", "log"]
    
    # (Required)
    # A list of Elasticsearch filters used for find events
    # These filters are joined with AND and nested in a filtered query
    # For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html
    filter:
    - query:
        query_string:
          query: 'kubernetes.labels.app:"dev-safe-transaction-service" AND log:"ERROR"'

    It's a general rule for all messages with ERROR level. Would it be possible to don't repeat errors with the same error content during a certain timeframe but show errors with the same query but different error content?

    for example. "ERROR issue 1" and "ERROR issue2"

    By looking at realert, having the same query if error 1 it's triggered, error 2 would be shadowed and that's not desirable

    Thank you very much!

    1 reply
    kasramtm
    @kasramtm

    Hi all
    After some days(totally random) some rules(totally random) get disabled by themselves and there are no signs of Error in logs regarding those. ( after restarting the service of elastalert (rerunning the elastalert) those disabled rules come back to the running rules and again this will happen on other rules!)
    has anyone faced this issue?

    elastalert: 0.2.2

    1 reply
    Caesar
    @928234269
    WARNING:apscheduler.scheduler:Execution of job "ElastAlerter.handle_rule_execution (trigger: interval[0:01:00], next run at: 2020-06-03 10:39:07 CST)" skipped: maximum number of running instances reached (1)
    who can help me
    William Muzyka
    @WillMuzyka
    Hi, everyone,
    There's any way to use multiple Aggregation keys on an Metric_aggregation alert? I'm trying to create an alert for disk usage for multiple hosts, so I need to agg each host and each of its partition (agent.hostname and system.filesystem.mount_point). Eg (Host1 with partition / and /opt, host2 with partition /, /opt and /etc... and so on
    1 reply
    ShotgunSpider
    @ShotgunSpider
    Hi guys!
    anyone ever come across the problem with percentage match alerts where in kibana the query shows 4.1% and in elastalert shows 41%?
    Sourabh Agrawal
    @Sourabh181097_twitter
    I wanted to monitor my kafka cluster and get customized slack alerts for cpus and disk usages. Finally was able to get it done.
    Using metric_aggregation and any.
    If someone want to take reference then do checkout
    https://gist.github.com/sourabh-agrawal/0efc4f6b7968afce66741a221faa4e7d
    https://gist.github.com/sourabh-agrawal/89f842b5f61f48a1985b4d23be7e4084
    Aristenio Monteiro
    @aristenio
    Nice @Sourabh181097_twitter ! Very usefull!
    raghav9329
    @raghav9329
    Hello Everyone,
    Did syntax error in one rule impact the whole elastalert system ?
    I have missed one bracket in query rule because of it elastalert failed to send other rules alert
    Vlad Vetshtein
    @vvvlad
    Is elastalert still maintained?
    3 replies
    slzzz
    @slzzz
    Hi guys!
    How to discover rule disable, how to automatically turn on?
    ERROR:root:Uncaught exception running rule myrule: ConnectionTimeout caused by - ReadTimeout
    INFO:elastalert:Rule myrule disabled
    Vamshi Krishna Santhapuri
    @rrskris
    Hi All,
    I have added a timebased logrotate handler to the elastalert config log which captures all the logs to a given file.
    If anybody is interested please do check it out here: https://gist.github.com/rrskris/ea0c53b3c5808e65815821cee24c9b28
    mcoklica-div
    @mcoklica-div
    Hello everyone,
    One question: Does current Elastalert version support dots in fields when addressing them in {match[]} section?

    Because, my error is the following:

    "Traceback (most recent call last):",
    " File \"/opt/elastalert/elastalert/elastalert.py\", line 1341, in alert",
    " return self.send_alert(matches, rule, alert_time=alert_time, retried=retried)",
    " File \"/opt/elastalert/elastalert/elastalert.py\", line 1430, in send_alert",
    " alert.alert(matches)",
    " File \"elastalert/alerts.py\", line 2151, in alert",
    " alert_config[alert_config_field] = alert_config_value.format(**context)",
    "KeyError: 'beat.hostname'"

    mcoklica-div
    @mcoklica-div
    Just to clarify - beat.hostname exist as a field, as you can see, rule is written for auditbeat docs...
    mcoklica-div
    @mcoklica-div
    Found a solution:
    Don't write as {match[beat.hostname]}
    Instead write as {match[beat][hostname]}
    Vamshi Krishna Santhapuri
    @rrskris
    @aristenio , I have faced a similar situation, so I added the date TZ to UTC like this: $(TZ=UTC date +'%F''T''%T' ) for --start and --end flags
    Hope this helps
    Aristenio Monteiro
    @aristenio

    @rrskris At really my devops team made a mistake, didn't restart the container after mapped timezone and localtime on docker-compose.yml. A mapping like this:

      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro

    was enough to waranty the container assume the timezone of the host instead UTC that is the timezone default of the ElastAlert

    Vamshi Krishna Santhapuri
    @rrskris
    ok got it.. I had my local time in PST and faced a similar issue which didn't result in current events.. thanks for letting me know. Its infact a better way in containers.
    Negrocris
    @negrocris2012_twitter
    Hello, does anyone know why this error appears in praeco
    6 replies
    image.png
    João Vitor Ramos
    @joaovitor3

    Hello everyone, I'm trying to send an alert every time a document from the error.exception is saved in elasticsearch using the new_term type. The alert is sent, but when I try to access some fields in the alert text and in the alert subject in Slack the fields return MISSING VALUE.

    Here is my rule:

    type: new_term
    
    fields:
      - error.id
      - error.grouping_key
      - error.exception.type
      - error.exception.message
    
    alert:
      - "slack"
    
    # bold subject with clickable link to kibana
    alert_subject: "Exception {0}: <http://localhost:5601/app/apm#/services/brelo-logs/errors/{1}|Access>"
    alert_subject_args:
      - error.exception.type
      - error.grouping_key
    
    # alert name and attachment body
    alert_text_type: alert_text_only
    alert_text: "Exception {0}: {1}"
    alert_text_args:
      - error.exception.type
      - error.exception.message
    
    
    slack_emoji_override: ':lock:'
    slack_msg_color: 'danger'
    slack_parse_override: full

    Anyone could help me please?

    Thanks in advance!

    2 replies
    cvdjv
    @cvdjv
    can someone help with this issue Yelp/elastalert#3038
    i tried everything and can't seem to get hive and elastalert to work with 7.9.1
    Aristenio Monteiro
    @aristenio

    Found a solution:
    Don't write as {match[beat.hostname]}
    Instead write as {match[beat][hostname]}

    @cvdjv @mcoklica-div had the same problem

    ozgursuder
    @ozgursuder
    How can I print the src_ip address here into the send.sh file?
    Aristenio Monteiro
    @aristenio
    @ozgursuder Take a look at https://elastalert.readthedocs.io/en/latest/ruletypes.html#command and try to use the new style format command: ["/opt/send.sh", "{match[src_ip]}"]
    ozgursuder
    @ozgursuder
    ERROR:root:Error while running alert command: Error while running command /opt/send.sh {match[src_ip]}: [Errno 8] Exec format error: '/opt/send.sh'
    send.sh IP="src_ip"
    Dennis
    @Dennis18374415_twitter

    Hello, I am trying to use the 'spike' rule, but the timeframe does not appear to be working for me. Here is a description of my issue: Yelp/elastalert#3041

    Can anyone please offer help on this? Is this a bug?

    saiprathapdp
    @saiprathapdp

    Hi Team,
    we have configured url monitoring in heartbeat YAML like below example.
    we are trying to configure certificate alert from kibana as like the following "Document to index" and we would like to expect the id or the name "MMT CMC" in to the index.

    Example:
    type:http
    hosts: ["https://mmt-cmc.com"]
    id: MMT CMC
    Name: MMT CMC
    Schedule:: "@every 10s"

    we have created alert for that in kibana "Document to index" with below configuration.

    {
    "context_message": "{{context.message}}",
    "monitor_id": "{{monitor.id}}",
    "alert_id": "{{alertId}}",
    "space_id": "{{spaceId}}",
    "alert_name": "{{alertName}}",
    "alert_instance_id": "{{alertInstanceId}}",
    "monitor_name": "{{monitor.name}}"
    }

    In index we can see only few details not the monitor_id and monitor_name

    _id Cv2BtnUBoLrrRG-s_Ipn

    _index tlsexpiry

    _score 0

    _type _doc

    alert_id 5251c140-70d8-42c0-94c6-001fd90f75fd

    alert_instance_id xpack.uptime.alerts.actionGroups.tls

    alert_name TLS Expiry alert

    context_message

    monitor_id

    monitor_name

    space_id default

    please let us know how to achieve this.

    Joe Bruns
    @JoeBrunsTR
    anyone have experience using the http post alert type?
    venmaniselvan
    @venmaniselvan
    Hi all,
    I am new to elastalert.
    Here is my config and rules file.
    While sending an email subject's total hit and matches are a mismatch. can you help me to find out the configuration issue?

    config.yaml

    rules_folder: example_rules
    run_every:
      minutes: 1
    buffer_time:
      minutes: 15
    es_host: x.x.x.x
    es_port: 9200
    es_username: elastic
    es_password: elastic
    writeback_index: elastalert_status
    writeback_alias: elastalert_alerts
    alert_time_limit:
      days: 2

    rules.yaml

    name: Example frequency rule
    type: frequency
    index: esb-audit-log-*
    num_events: 1
    timeframe:
      minutes: 15
    filter:
    - match:
        applicationname.keyword: "SRReceiver"
    alert:
    - "email"
    email:
    - "abcd@gmail.com"
    email_format: html
    from_addr: "no-reply@elastalert.com"
    email_reply_to: "no-reply@elastalert.com"
    aggregation:
      minutes: 2
    alert_text_type: alert_text_only
    alert_subject_args:
      - num_hits
      - num_matches
    alert_subject : "Number of hits: {0}, Number of matches:{1}"
    aggregation_key: 'applicationname'

    console output

    INFO:elastalert:Background configuration change check run at 2020-11-29 11:19 +03
    INFO:elastalert:Background alerts thread 0 pending alerts sent at 2020-11-29 11:19 +03
    INFO:elastalert:Queried rule Example frequency rule from 2020-11-29 11:04 +03 to 2020-11-29 11:19 +03: 1 / 1 hits
    INFO:elastalert:Ran Example frequency rule from 2020-11-29 11:04 +03 to 2020-11-29 11:19 +03: 1 query hits (1 already seen), 0 matches, 0 alerts sent
    INFO:elastalert:Background configuration change check run at 2020-11-29 11:20 +03
    INFO:elastalert:Disabled rules are: []
    INFO:elastalert:Sleeping for 59.99979 seconds
    INFO:elastalert:Background alerts thread 0 pending alerts sent at 2020-11-29 11:20 +03
    INFO:elastalert:Queried rule Example frequency rule from 2020-11-29 11:05 +03 to 2020-11-29 11:20 +03: 3 / 3 hits
    INFO:elastalert:Adding alert for Example frequency rule to aggregation(id: gfwVE3YBJ3tobqa0SSQQ, aggregation_key: SRReceiver), next alert at 2020-11-29 08:20:50.232804+00:00
    INFO:elastalert:Adding alert for Example frequency rule to aggregation(id: gfwVE3YBJ3tobqa0SSQQ, aggregation_key: SRReceiver), next alert at 2020-11-29 08:20:50.232804+00:00
    INFO:elastalert:Ran Example frequency rule from 2020-11-29 11:05 +03 to 2020-11-29 11:20 +03: 3 query hits (1 already seen), 2 matches, 0 alerts sent
    INFO:elastalert:Background configuration change check run at 2020-11-29 11:21 +03
    INFO:elastalert:Disabled rules are: []
    INFO:elastalert:Sleeping for 59.99975 seconds
    INFO:elastalert:Sent email to ['abcd@gmail.com']
    INFO:elastalert:Background alerts thread 1 pending alerts sent at 2020-11-29 11:21 +03
    INFO:elastalert:Queried rule Example frequency rule from 2020-11-29 11:06 +03 to 2020-11-29 11:21 +03: 3 / 3 hits
    INFO:elastalert:Ran Example frequency rule from 2020-11-29 11:06 +03 to 2020-11-29 11:21 +03: 3 query hits (3 already seen), 0 matches, 0 alerts sent
    INFO:elastalert:Background configuration change check run at 2020-11-29 11:22 +03
    INFO:elastalert:Background alerts thread 0 pending alerts sent at 2020-11-29 11:22 +03

    My Email Subject Show as

    Number of hits: 1, Number of matches:1

    But expected Output is

    Number of hits: 3, Number of matches:3
    PH03NIX
    @tarek_chalan_au_twitter
    Hello all, I am wondering is there a way to specify what to include in the alert message body to expand it and include stored fields in elastalert document ingested to es ?
    mechanic-315
    @mechanic-315
    image.png
    Hi all, Just want to ask about the filter in elastalert. So can I multiple/divide the value of the field. As the picture, I want to set a rule that alerts me in case of "value(system.cpu.idle.pct) / value(system.cpu.cores) < 0.1"
    ozgursuder
    @ozgursuder
    ERROR:root:Error while running alert email: Error connecting to SMTP host: [SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1123)
    smeesheady
    @smeesheady
    Does anyone know how to mimic this watcher alert feature in elastalert ? {
    "trigger": {
    "schedule": {
    "interval": "2m"
    }
    }, Thanks a lot.
    PH03NIX
    @tarek_chalan_au_twitter
    @JoeBrunsTR I am using it what are u after ?
    Joe Bruns
    @JoeBrunsTR
    @tarek_chalan_au_twitter I was able to get it to work, but thanks for followup