Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    RashmiChoudhary
    @rashmichoudhary07
    I have an issue in production environment(at work) where elastalert is showing a high time_taken values in elastalert_status_status index. This happens intermittenly for a continuous timeframe during certain times in a day which is troubling my client since eventually real time alerts are delayed. I have been trying to find Root cause for the delay introduced at elastalert but no luck since it is intermittent. Please find a sample document from prod for your help.@timestamp: 2021-02-18T07:45:16.048284Z
    _id: SZkZtHcBUW6Vb-ZGp3YW
    _index: elastalert_status_status
    _type: _doc
    endtime: 2021-02-18T06:03:42.463717Z
    hits: 407
    matches: 34
    num_hits: 3
    num_matches: 1
    rule_name: Shop - Severity - Minor [400 TO 499] And Not [401,403,424]
    starttime: 2021-02-18T05:38:43.224775Z
    time_taken: 6093.584537982941Is time_taken the exact time for elastalert to only query elasticsearch?
    We have millions of documents in our ES and Kibana gives faster result for the same query for time_taken is a high value. Also, if i manually run the cURL query that elastalert generates the result is fetched only with in few 100 milliseconds. We are seeing 1 to 4+ hours to time_taken value in producton and unable to get to the root cause of this.Could please help us to get to the root of high time_taken value. I am stuck in a real prod issue here with elastalert delaying alert triggers due to huge time_taken value(in seconds as per doc). Looking forward to hearing from you.
    I have trying to research on what is causing the above but no luck yet
    i have also opened ticket at github issues site
    RashmiChoudhary
    @rashmichoudhary07
    i am new to this platform Any help from experts is highly appreciated.. TIA!
    Ashish Singh
    @ashishkaransingh

    Hello,
    Anybody running Elastalert using python 3.8
    I am getting error

    python3 -m elastalert.elastalert --verbose --rule test_count_alert.yaml --config config.yaml

    Throwing error
    
    Traceback (most recent call last):
      File "/usr/lib/python3.8/runpy.py", line 185, in _run_module_as_main
        mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
      File "/usr/lib/python3.8/runpy.py", line 111, in _get_module_details
        __import__(pkg_name)
     File "/opt/elastalert/elastalert/elastalert.py", line 29, in <module>
        from . import kibana
    ImportError: attempted relative import with no known parent package

    Ref
    https://github.com/Yelp/elastalert/issues/2439#issuecomment-534058583
    Yelp/elastalert#2776

    Ashish Singh
    @ashishkaransingh
    I figured it out...
    Kumar ankit
    @Ankitxroot

    pkg_resources.DistributionNotFound: The 'elastalert==0.1.28' distribution was not found and is required by the application

    How to fix this.
    Thanks in advance.

    4 replies
    danny zak
    @dannyzak_gitlab
    is there a way to specify a timewindow when alerting should (not) occure for a certrain rule, eg. not after friday 5pm till monday 9am
    Kyle Parrish
    @arnydo

    Hiya! Anyone have any feedback on building filters? I am trying to create an alert for Office 365 sign ins outside of the US. I am trying thie:

    filter:
    - query:
            query_string: {query: "NOT source.geo.country_iso_code:US"}
            query_string: {query: 'o365.audit.Operation: UserLoggedIn'}

    But...it doesn't seem to work as expected. I can't find documentation on building boolean filters. "this AND NOT that..."

    7 replies
    Kyle Parrish
    @arnydo
    Also, why are there hits when running elastalert-test-rule but not elastalert? Both are using the same rule.
    ankita1596
    @ankita1596
    My elastalert and elasticsearch is running in docker environment. Can you please help me to send output from elastalert to zabbix running in docker.
    dunter-d
    @dunter-d
    hi! I'm having issues with elastalert. Running ubuntu 20.04/elasticsearch 7.10/python3.8 & running systemctl start elastalert outputs this:
    '''
    rule['alert'] = self.load_alerts(rule, alert_field=rule['alert'])
    File "/usr/local/lib/python3.8/dist-packages/elastalert-0.2.4-py3.8.egg/elastalert/loaders.py", line 469, in load_alerts
    alert_field = [normalize_config(x) for x in alert_field]
    File "/usr/local/lib/python3.8/dist-packages/elastalert-0.2.4-py3.8.egg/elastalert/loaders.py", line 469, in <listcomp>
    alert_field = [normalize_config(x) for x in alert_field]
    File "/usr/local/lib/python3.8/dist-packages/elastalert-0.2.4-py3.8.egg/elastalert/loaders.py", line 450, in normalize_config
    config_copy.update(config) # warning, this (intentionally) mutates the rule dict
    ValueError: dictionary update sequence element #0 has length 1; 2 is required
    elastalert.service: Main process exited, code=exited, status=1/FAILURE
    '''
    dunter-d
    @dunter-d
    @ashishkaransingh how did you solved that issue with the "from . import kibana" error?
    dplgrail
    @dplgrail
    image.png
    Hi, i've this issue... TestController: Failed to test rule with error: INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
    To send them but remain verbose, use --verbose instead.
    Im using SIEMonster and Praeco for ElastAlert
    1 reply
    raks25
    @raks25:matrix.org
    [m]
    Hi can anyone help me out with this "1|INFO|0|1|ibs| |2021/04/22 18:54:30.999234|http: TLS handshake error from **: remote error: tls: bad certificate" log displayed on my kibana and i am using elastalert for notifying the string " tls: bad certificate" but it sends alerts when we have word like only certificate in it , but i want to be notified only when the log has keyword "bad certificate"
    Naoyuki Sano
    @nsano-rururu
    Yelp/elastalert is no longer maintained. Please use jertel/elastalert2. Questions to the discussion below
    https://github.com/jertel/elastalert2/discussions
    itruivitorino
    @itruivitorino
    Hi everyone. I've been trying to make an alert go when the status of a heartbeat monitor goes up AFTER being down. Anyone made any progress with similar thing?
    ramprasadavirineni
    @ramprasadavirineni:matrix.org
    [m]

    Hi @nsano-rururu
    Hello all,
    I've been installed elastalert on elk7.11, I configured config and rules files, but alert are not getting on mails..
    please anyone have idea on help me out....

    1 rules loaded
    INFO:elastalert:Starting up
    INFO:elastalert:Disabled rules are: []
    INFO:elastalert:Sleeping for 59.999878 seconds
    INFO:elastalert:Queried rule Hello Test mail from ELK Stack please ignore from 2021-05-07 07:41 EDT to 2021-05-07 07:56 EDT: 0 / 0 hits
    INFO:elastalert:Ran Hello Test mail from ELK Stack please ignore from 2021-05-07 07:41 EDT to 2021-05-07 07:56 EDT: 0 query hits (0 already seen), 0 matches, 0 alerts sent
    INFO:elastalert:Background configuration change check run at 2021-05-07 07:57 EDT

    INFO:elastalert:Background alerts thread 0 pending alerts sent at 2021-05-07 07:57 EDT

    My config.yml:
    cat config.yaml|grep -v "^#"
    rules_folder: example_rules

    run_every:
    minutes: 1

    buffer_time:
    minutes: 15

    es_host: 192.168.0.1

    es_port: 9200

    writeback_index: elastalert_status
    writeback_alias: elastalert_alerts

    alert_time_limit:
    days: 0

    smtp_port: 25

    smtp_host: 'xxx.com'

    name: Hello Test mail from ELK Stack please ignore

    type: frequency

    index: filebeat-*

    num_events: 3

    timeframe:
    hours: 1

    filter:

    • query:
      query_string:
      query: "message: authentication failure OR failure password"
      timestamp_field: "@timestamp"

      - term:

      process_name: "DefaultQuartzScheduler6"

    alert:

    • "email"

    email:

    Naoyuki Sano
    @nsano-rururu
    @ramprasadavirineni:matrix.org
    I don't use elastalert.
    ramprasadavirineni
    @ramprasadavirineni:matrix.org
    [m]

    Hello,

    could you please suggest which one good for alert notification for ELK 7.x

    Mohd Rashid
    @MohdRashid01

    Hi All,

    I have install ELK in my laptop and after that i configure elastialert for triggering email like issue coming into docker container so it will trigger an email to my gmail account for that i setup rules.yml file in kibana plugin of elastialert after that it send an email which u can see above one content in that email i want to remove which i have mention above word. Only Message should show in my email whenever it trigger How to do it, Please do let me ASAP

    Below is the content which im getting in my gmail inbox. I want to

    remove _id, _index, _type, num_matches, num_hits and @timestamp

    all this only i want message to be include in my email triggering everytime.

    Below is getting into gmail inbox

    @timestamp: 2021-05-24T11:16:06Z
    _id: 2S0WnnkBz7SOxaiw1TZk
    _index: logstash-2021.05.24
    _type: _doc
    message: <30>May 24 11:16:06 fx-prod-1 prod_fx-control-plane.1.knel5yam 2021-05-24 11:16:06.926 INFO 1 --- [nio-8080-exec-9] com.fxlabs.fxt.rest.run.RunController : Find Latest by job id [8a8089ba777311370177734530902ec8] org [8a8081066e02d6a2016e04eacd2005c7] principal [8a808155647d283a01647d7c5e0d07ba]
    num_hits: 8
    num_matches: 1

    and

    Here is the rules.yaml file which i used to trigger email for issue

    # Alert when the rate of events exceeds a threshold
    
    # (Optional)
    # Elasticsearch host
    # es_host: elasticsearch.example.com
    
    es_host: elasticsearch.test.com
    # (Optional)
    # Elasticsearch port
    #es_port: 14900
    es_port: 9200
    
    # (OptionaL) Connect with SSL to Elasticsearch
    #use_ssl: True
    
    # (Optional) basic-auth username and password for Elasticsearch
    #es_username: someusername
    #es_password: somepassword
    es_username: testelastic
    es_password: xxxx
    
    
    # (Required)
    # Rule name, must be unique
    name: Exception Alert
    
    # (Required)
    # Type of alert.
    # the frequency rule type alerts when num_events events occur with timeframe time
    #type: blacklist
    type: any
    include: ["message"]
    
    # (Required)
    # Index to search, wildcard supported
    #index: logstash*
    index: filebeat*
    
    # (Required, frequency specific)
    # Alert when this many documents matching the query occur within a timeframe
    #num_events: 1
    
    # (Required, frequency specific)
    # num_events must occur within this amount of time to trigger an alert
    timeframe:
    #  hours: 1
    #  minutes: 1
      seconds: 1
    
    #compare_key: "message"
    
    #blacklist:
    
    #- "error"
    
    realert:
      minutes: 5
    
     # This is send out all matches in one email
    # aggregation:
    #  minutes: 1
    
    #  seconds: 0
    #filter:
    #- term:
    #    message: "[error]"
    
    #filter:
    # - match:
     #   message: "job"
    
    filter:
     - query:
          query_string:
     #       query: "message: exception AND  message: control"
            query: "message: job"
    #filter:
    #- query:
    #   query_string:
    #    query: "message: error"
    
    alert:
    
    - "slack"
    - "email"
    
    
    #- slack
    slack_webhook_url: "https://hooks.slack.com/services/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"
    #slack_username_override: "ElastAlert"
    slack_username_override: "Mohd Rashid"
    
    #- email
    email: ["rashidmd777@gmail.com"]
    smtp_host: "smtp.gmail.com"
    smtp_port: "587"
    from_addr: "rashidmd777@gmail.com"

    How to do it, Please do let me ASAP

    Mohd Rashid
    @MohdRashid01
    ?
    Naoyuki Sano
    @nsano-rururu
    Vlad Vetshtein
    @vvvlad
    Hi everyone,
    Where can I find some information about why Yelp stopped using and updating elastalert?
    What is the future of elastalert? Does elastalert 2 is just a bug fix and support version, or does it have some future roadmap?
    3 replies
    tl-Bruno-Braga
    @tl-Bruno-Braga
    Hi. Is it possible to store the credentials for http post basic auth on env variables? We are using elastalert on CI/CD and hardcoding creds on each rule seems like a bad wau to go about it
    Aniket
    @ani2fun

    My use case is something similar to following. I have nested array of objects warehouses and trying to filter based on the last element of the array.

    I am getting some results but not correct one. Like to know how exactly it is working though as well.

    Let's say ,

    I want to search for a product based on stocks last element of warehouse array. This is the product document looks like:

    {
      "productId": 5,
      "productName": "Shoes",
      "warehouses": [
        {
          "location": "Location A",
          "quantity": 100
        },
        {
          "location": "Location B",
          "quantity": 10
        },
        {
          "location": "Location C",
          "quantity": 50
        }
      ]
    }

    And it's mapping is :

    PUT /products
    {
      "mappings": {
        "properties": {
          "productId": {
            "type": "integer"
          },
          "productName": {
            "type": "text",
            "fields": {
                "raw": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "warehouses": {
            "properties": {
              "location": {
                "type": "text"
              },
              "quantity": {
                "type": "integer"  
              }
            }
          }
        }
      }
    }

    Let's say, I index following 7 documents:

    POST products/_bulk
    {"index":{"_id":1}}
    {"productId":1,"productName":"Bags","warehouses":[{"location":"Location A","quantity":20},{"location":"Location B","quantity":30},{"location":"Location C","quantity":50}]}
    {"index":{"_id":2}}
    {"productId":2,"productName":"Shirts","warehouses":[{"location":"Location A","quantity":100},{"location":"Location B","quantity":150},{"location":"Location C","quantity":150}]}
    {"index":{"_id":3}}
    {"productId":3,"productName":"Shoes","warehouses":[{"location":"Location A","quantity":100},{"location":"Location B","quantity":10},{"location":"Location C","quantity":50}]}
    {"index":{"_id":4}}
    {"productId":4,"productName":"Shirt","warehouses":[{"location":"Location A","quantity":100},{"location":"Location B","quantity":10},{"location":"Location C","quantity":60}, {"location":"Location F","quantity":70}]}
    {"index":{"_id":5}}
    {"productId":5,"productName":"Socks","warehouses":[{"location":"Location A","quantity":800},{"location":"Location B","quantity":1500},{"location":"Location Z","quantity":1000}]}
    {"index":{"_id":6}}
    {"productId":6,"productName":"TV","warehouses":[{"location":"Location A","quantity":20},{"location":"Location B","quantity":150},{"location":"Location C","quantity":123}]}
    {"index":{"_id":7}}
    {"productId":7,"productName":"Table","warehouses":[{"location":"Location A","quantity":20},{"location":"Location B","quantity":200},{"location":"Location C","quantity":140}, {"location":"Location D","quantity":123}]}
    Now I would like to search and filter products with "quantity": 123. So as per above indexed documents, I suppose to filter and get the products with id:6 and id:7, because it has the quantity: 123 as its last element.
    
    Here is my Painless(ful) script :
    
    GET /products/_search
    {
      "query": {
        "bool": {
          "must": {
            "match_all": {}
          },
          "filter": {
            "bool": {
              "must": {
                "script": {
                  "script": {
                    "lang": "painless",
                    "source": """
                      def x = doc['warehouses.quantity'];
    
                      def flag = false;
                        if(x[x.length - 2 ] == params.limit) {
                          flag = true;
                        }
    
                      return flag;
                    """,
                    "params": {
                      "limit": 123
                    }
                  }
                }
              }
            }
          }
        }
      }
    }

    so in above script I get result for id:6 which is TV product. And when I replace x[x.length - 2 ] with x[x.length - 3 ] I can get result for id:7.

    I am not sure how to get result which contains both documents , [id:6 (TV) and id:7 (Table)].

    I am using Elasticsearch version: 7.8.1.

    1 reply
    sudores
    @sudores
    Hi guys. Can somebody say, is it possible to send alerts with elastalert to telegram when some tomcat app failing without writing modules?
    gittanner
    @gittanner
    I've got an Elastalert set up to alert me when a cert is going to expire. Right now, the alert is set to go off when the certificate expires in 365d or less.
    I am monitoring 13 webpages with 13 different certificates. When the alert triggers, I the email body only contains information about 1 cert, and it is usually the same cert. What changes do I need to make so that all certs get picked up and alerted at the same time?
    name: Certificate Expiration (365 days)
    description: TLS certificate not valid in 365 days
    type: any
    index: heartbeat-*
    filter:
     - range:
         tls.server.x509.not_after:
           gte: now
           lt: now+365d
    realert:
      minutes: 1
    maxbruso
    @maxbruso
    Hi everyone, I have a problem with elastalert. I have an index file in elastisearch with various fields.
    I would like to create a new module of elastalert which calculates the ratio.
    let me explain. I would like to calculate the ratio between all the records with (field1 = A and field2 = B and field3 = C) and the records with (field1 = A and field2 = B and field3 = Z) for a certain timeframe.
    Someone has tried to create a new module to calculate the ratio?
    vvvlad
    @vvvlad_twitter

    Hi,
    Can someone explain to me what is the reason for elastalert_error index to have the following mapping, which means there is no option to filter by rule name or any other parameter by the rule?

    "data" : {
              "type" : "object",
              "enabled" : false
            }

    Any way to change this in the elastalert configuration?
    Thanks!

    vvvlad
    @vvvlad_twitter
    anyone can help me with this question?
    Jabes Pauya
    @yabetsu93

    hello good day, I am querying in elastalert i am using term i.e
    filter

    • term
      container.name: "sample"
    • term
      namespace.name: "testing"
    • term
      message: "level:ERROR"

    in parsed result from kibana i want to extract the level: error on the message result in kibana
    example
    message{"namespace.name":"testing", "container.name":"level":"ERROR", xxxxx }

    can you help me?

    kaihua-liu-exa
    @kaihua-liu-exa
    Hi, I did a metric aggregation sum on my data, but it's not working to alert:
    `metric_agg_key: "bytes_out"
    doc_type: "logs"
    buffer_time:
    hours: 24
    metric_agg_type: "sum"
    realert:
    minutes: 0
    max_threshold: 1000000000
    is_enabled: true
    query_delay:
    minutes: 2
    query_key: "user_email.keyword"
    allow_buffer_time_overlap: true
    top_count_keys:
    • "bytes_out"
    • "user_email.keyword"`
    The query is to sum bytes_out for a specific email
    When I query and sum the bytes_out, there are emails whose sum of bytes_out is larger than 1000000000
    Is it because the query result is too large?
    There is no error from logs.
    wajika
    @wajika
    How to access a value under an array, it returns as MISSING VALUE

    I want to get the data of the message, but it didn't succeed.

    How to access a value under an array?

    alert_text_args:

    • "@timestamp"
    • service.name
    • error.exception.message

    error.exception.message format

    "error": {
    "exception": [
    {
    "stacktrace": [
    XXXXXX
    ......
    ],
    "handled": false,
    "message": "XXXX",
    "type": "XXXX"
    }
    ]
    }

    ✖ E S L A M ✖
    @eslammalii_twitter
    telegram give me an erroe
    1 reply
    error
    Error initiating alert ['slack', 'Telegram']: Could not import module Telegram: not enough values to unpack