Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
    Barry Salas
    All I have tried to install and reinstall elastalert on one of my elasticsearch servers and everything goes well until I try to create an index. I keep getting elastalert-create-index: command not found.....I have looked at every fix that I could find and ensured that I have all of the needed requirements, however I still cannot get it to create. I am on the latest version of ELK so I am wondering if that could be the issues but I am not sure. Any advice would be greatly appreciated.
    8 replies

    Hello there

    I'm generating http post alert with elastalert.

    but the post https I want to write is a site and its certificate is not secure.

    verify_certs: False

    I wonder how can I use the value of elastalert

    alert: post
    http_post_url: "https: // elastic: 9200 / a_alerts / _doc /"

    Content-Type: 'application / json'

    Hernando Agudelo

    Any idea why my elastalert make twice a curl call just for 1 alert?

    curl -H 'Content-Type: application/json' -XPOST 'http://localhost:9200/elastalert_status_status/_doc?pretty' -d '{ "@timestamp": "2021-01-12T06:41:26.490784Z", "endtime": "2021-01-12T06:41:26.013716Z", "hits": 0, "matches": 0, "rule_name": "error_filelogs_stage", "starttime": "2021-01-12T06:36:26.013716Z", "time_taken": 0.4770529270172119 }'

    curl -H 'Content-Type: application/json' -XPOST 'http://localhost:9200/elastalert_status_status/_doc?pretty' -d '{ "@timestamp": "2021-01-12T06:41:26.490784Z", "endtime": "2021-01-12T06:41:26.013716Z", "hits": 0, "matches": 0, "rule_name": "error_filelogs_stage", "starttime": "2021-01-12T06:36:26.013716Z", "time_taken": 0.4770529270172119 }'

    @abhinath3399 show me the error if you still have issues with it
    Hello ! I struggle to create a content for rabbitMQ. My message was correctly created into ES, but if I just take that value from ES and put it in a rabbitMQ message, the receiver cannot parse it as it miss some escape characters. I dont see any possibility on the templating, any solution ? Or do I need to retreate the message before posting it to ES to contains enough escape characters to build a JSON two fingers in the nose ?
    alert_text: '{{"logLevel": "{0}", "thread": "{1}", "class":"{2}",  "message": "{3}", "emitter": "{4}", "@timestamp": "{5}", "@log_name": "{6}", "stackTrace": "{7}", "logType": "{8}", "workflow": {9}}}'
    alert_text_type: alert_text_only
      - logLevel
      - thread
      - class
      - message
      - emitter
      - '@timestamp'
      - '@log_name'
      - stackTrace
      - logType
      - workflow
    Oh I think the issue is the triple quoted field on ES. It removes escape characters in what is stored inside ES. To sum up, I cannot send a JSON if the ES stored field is a triple quote, as escape characters won't be there ?
    Barry Salas
    Okay so I though I could get elastalert to work if I used docker vice just a side by side install with my standing Elasticsearch 7.10 and I also got it to work in my test environment with a full docker environment but that is not the case with my production. In my production I am running CentOs 7 and elasticsearch with version 7.10. Even if I follow step by step using a docker or dockerless install elastalert will not install at all. What I mean by this is I pull the clone from Git
    ensure all requirements are installed including setuptools, run python3 setup.py install, fix the config.yaml file, fix a sudo rule for testing, and go to create index and that is where it stops and I cannot do anything further
    3 replies
    Vlad Vetshtein
    aws slack

    Hello everyone, I got an error while testing the metric_aggregation rule. Does anyone have an answer?

    2021-01-28 15:40:24,352 INFO elasticsearch GET [status:200 request:0.005s]
    2021-01-28 15:40:24,353 ERROR root Traceback (most recent call last):
    File "/usr/local/python36/lib/python3.6/site-packages/elastalert/elastalert.py", line 1269, in handle_rule_execution
    num_matches = self.run_rule(rule, endtime, rule.get('initial_starttime'))
    File "/usr/local/python36/lib/python3.6/site-packages/elastalert/elastalert.py", line 949, in run_rule
    self.add_aggregated_alert(match, rule)
    File "/usr/local/python36/lib/python3.6/site-packages/elastalert/elastalert.py", line 1807, in add_aggregated_alert
    alert_time = ts_now() + rule['aggregation']
    TypeError: unsupported operand type(s) for +: 'datetime.datetime' and 'dict'

    My rules file

    name: test
    type: metric_aggregation
    index: apm-*
    minutes: 1
    metric_agg_key: transaction.duration.us
    metric_agg_type: avg
    query_key: service.name
    sync_bucket_interval: true
    allow_buffer_time_overlap: true
    use_run_every_query_size: true
    seconds: 10
    doc_type: _doc
    max_threshold: 100000

    • term:
      agent.name: "dotnet"
      • "debug"
    4 replies
    Hi all, is there a way to send email report every 5 minutes using elastalert and the body is formatted like this:
    API Calls: 2,872
    Resp. Time: 449 ms
    Error Rate: 0.38%
    Ramesh Pradhan
    Does latest version of elastalert support kibana_url and use_kibana_dashboard feature for kibana 7.x ?
    7 replies
    Bhanupraveen G
    elastalert-test-rule example_rules/example_frequency.yaml by default using " File "/usr/lib/python2.7/site-packages/pkg_resources/init.py"". failing with syntax error. I need to use python3.6 package which is installed. Please let me know?
    1 reply
    Ashish Singh

    Hello All,
    I was trying to modify
    alerts.py (/elastalert/alerts.py)

    Reason: in order to send events to Servicenow
    To muck around i changed the payload, essentially hardcoded the values except description (and it works it triggers an alert and creates an event in Servicenow).
    payload = {
    "source": "oursource",
    "event_class": "Log_Analytics",
    "node": "comp1",
    "metric_name": "Interface",
    "type": "Details hardcoded",
    "resource": "resource",
    "severity": "3",
    "description": "This is dummy description",
    "additional_info": description

    My question is how can i fetch values dynamically from Elasticsearch?
    Example node is a Host.keyword in elasticsearch how can i fetch it?
    I notice description has most of the data already how can make use of it?

    Any help would be great!!

    Thank you

    Ashish Singh
    I guess i should ask how can i access the dict object from matches
    def alert(self, matches):
    for match in matches:
            # Parse everything into description.
            description = str(BasicMatchString(self.rule, match))
    Ashish Singh

    Notice i tried to add "node": match.Host, but got an error

    File \"/opt/elastalert/elastalert/alerts.py\", line 1815, in alert", " \"node\": match.Host,", "AttributeError: 'dict' object has no attribute 'Host'"],

    payload = {
    "source": "SupportOne",
    "event_class": "Log_Analytics",
    "node": match.Host,
    "metric_name": "Interface",
    "type": "Details hardcoded",
    "resource": "resource",
    "severity": "3",
    "description": "This is dummy description_222",
    "additional_info": description

    Ashish Singh

    Looks like this is it

    node = lookup_es_key(matches[0], 'Host')
    message = lookup_es_key(matches [0], 'Message')

    then use it in the payload


    Hi everyone!
    I am working with a few frequency rules but I am facing the following problem.
    Setting the number of events to 3, as the documentation says "This rule matches when there are at least a certain number of events in a given time frame", if I have more than 3 hits, all of the hits that are in the timeframe should be included in the same alert, not triggering new alerts only each 3 hits, that is what I am getting: matches for only the exact num_events, not more. My rule:

    name: r1_redmov
    description: "3 events (any) redes moviles  in 2 minutes"
    type: frequency
    # (Required)
    # Index to search, wildcard supported
    index: redes_moviles
    # If different than default
    timestamp_field: time_stamp
    num_events: 3
      minutes: 2
    # to see all the hits related to the match
    attach_related: true  
    - "debug"

    Moreover, looking into the ruletypes.py, in the frequency rule, the condition is if self.occurrences[key].count() >= self.rules['num_events’]:


    2 replies
    Parul Kushwaha
    Hi everyone! I am creating this below elastalert rule :
    rule1.yaml: |-
    # elasticsearch host
    es_host: my_host
    es_port: my_port
    name: Error Log Alerts
    type: frequency
    num_events: 1
      hours: 1
    index: testindex-applicationlogs-*
    - query:
          query: "appname:xxx AND host_stability:prod AND (function:web.RemotingServlet.handleThrowable* OR loglevel:ERROR)"
    - "slack"
    alert_subject: "Error Log Alerts"
    slack_webhook_url: "slack_webhook_url"
    I am getting this exception while running the rule : ERROR:root:Uncaught exception running rule Error Log Alerts: TransportError(429, 'circuit_breaking_exception', '[parent] Data too large, data for [<http_request>] would be [31155871676/29gb], which is larger than the limit of [31045425561/28.9gb], real usage: [31155870848/29gb], new bytes reserved: [828/828b], usages [request=0/0b, fielddata=60671194/57.8mb, in_flight_requests=9705526/9.2mb, accounting=223410932/213mb]')
    Wanted to understand how to introduce the timestamp in filter ?
    Something like to query for last 1 hour.
    if i want to get a 40% drop in events using spike_alert should the spike_height be 1.6 for 40% and spike_type be down?
    Ashish Singh


    Need help in accessing alert or rule types used in .yaml
    I am able to access and print values filter and name.


    How can i access alert type example whether it is frequency or Average etc
    I tried as get_rules = self.rule.get('type', None) but it print the following line including the class etc
    "Print rules: <elastalert.ruletypes.FrequencyRule object at 0x7fd3e204c310>"

    A little background:
    I am creating a custom payload for class ServiceNowAlerter(Alerter): in alerts.py

    Hello @Qmando Hope you are doing well...
    I have an issue in production environment(at work) where elastalert is showing a high time_taken values in elastalert_status_status index. This happens intermittenly for a continuous timeframe during certain times in a day which is troubling my client since eventually real time alerts are delayed. I have been trying to find Root cause for the delay introduced at elastalert but no luck since it is intermittent. Please find a sample document from prod for your help.@timestamp: 2021-02-18T07:45:16.048284Z
    _id: SZkZtHcBUW6Vb-ZGp3YW
    _index: elastalert_status_status
    _type: _doc
    endtime: 2021-02-18T06:03:42.463717Z
    hits: 407
    matches: 34
    num_hits: 3
    num_matches: 1
    rule_name: Shop - Severity - Minor [400 TO 499] And Not [401,403,424]
    starttime: 2021-02-18T05:38:43.224775Z
    time_taken: 6093.584537982941Is time_taken the exact time for elastalert to only query elasticsearch?
    We have millions of documents in our ES and Kibana gives faster result for the same query for time_taken is a high value. Also, if i manually run the cURL query that elastalert generates the result is fetched only with in few 100 milliseconds. We are seeing 1 to 4+ hours to time_taken value in producton and unable to get to the root cause of this.Could please help us to get to the root of high time_taken value. I am stuck in a real prod issue here with elastalert delaying alert triggers due to huge time_taken value(in seconds as per doc). Looking forward to hearing from you.
    I have trying to research on what is causing the above but no luck yet
    i have also opened ticket at github issues site
    i am new to this platform Any help from experts is highly appreciated.. TIA!
    Ashish Singh

    Anybody running Elastalert using python 3.8
    I am getting error

    python3 -m elastalert.elastalert --verbose --rule test_count_alert.yaml --config config.yaml

    Throwing error
    Traceback (most recent call last):
      File "/usr/lib/python3.8/runpy.py", line 185, in _run_module_as_main
        mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
      File "/usr/lib/python3.8/runpy.py", line 111, in _get_module_details
     File "/opt/elastalert/elastalert/elastalert.py", line 29, in <module>
        from . import kibana
    ImportError: attempted relative import with no known parent package


    Ashish Singh
    I figured it out...
    Kumar ankit

    pkg_resources.DistributionNotFound: The 'elastalert==0.1.28' distribution was not found and is required by the application

    How to fix this.
    Thanks in advance.

    4 replies
    danny zak
    is there a way to specify a timewindow when alerting should (not) occure for a certrain rule, eg. not after friday 5pm till monday 9am
    Kyle Parrish

    Hiya! Anyone have any feedback on building filters? I am trying to create an alert for Office 365 sign ins outside of the US. I am trying thie:

    - query:
            query_string: {query: "NOT source.geo.country_iso_code:US"}
            query_string: {query: 'o365.audit.Operation: UserLoggedIn'}

    But...it doesn't seem to work as expected. I can't find documentation on building boolean filters. "this AND NOT that..."

    7 replies
    Kyle Parrish
    Also, why are there hits when running elastalert-test-rule but not elastalert? Both are using the same rule.
    My elastalert and elasticsearch is running in docker environment. Can you please help me to send output from elastalert to zabbix running in docker.
    hi! I'm having issues with elastalert. Running ubuntu 20.04/elasticsearch 7.10/python3.8 & running systemctl start elastalert outputs this:
    rule['alert'] = self.load_alerts(rule, alert_field=rule['alert'])
    File "/usr/local/lib/python3.8/dist-packages/elastalert-0.2.4-py3.8.egg/elastalert/loaders.py", line 469, in load_alerts
    alert_field = [normalize_config(x) for x in alert_field]
    File "/usr/local/lib/python3.8/dist-packages/elastalert-0.2.4-py3.8.egg/elastalert/loaders.py", line 469, in <listcomp>
    alert_field = [normalize_config(x) for x in alert_field]
    File "/usr/local/lib/python3.8/dist-packages/elastalert-0.2.4-py3.8.egg/elastalert/loaders.py", line 450, in normalize_config
    config_copy.update(config) # warning, this (intentionally) mutates the rule dict
    ValueError: dictionary update sequence element #0 has length 1; 2 is required
    elastalert.service: Main process exited, code=exited, status=1/FAILURE
    @ashishkaransingh how did you solved that issue with the "from . import kibana" error?
    Hi, i've this issue... TestController: Failed to test rule with error: INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
    To send them but remain verbose, use --verbose instead.
    Im using SIEMonster and Praeco for ElastAlert
    1 reply
    Hi can anyone help me out with this "1|INFO|0|1|ibs| |2021/04/22 18:54:30.999234|http: TLS handshake error from **: remote error: tls: bad certificate" log displayed on my kibana and i am using elastalert for notifying the string " tls: bad certificate" but it sends alerts when we have word like only certificate in it , but i want to be notified only when the log has keyword "bad certificate"
    Naoyuki Sano
    Yelp/elastalert is no longer maintained. Please use jertel/elastalert2. Questions to the discussion below
    Hi everyone. I've been trying to make an alert go when the status of a heartbeat monitor goes up AFTER being down. Anyone made any progress with similar thing?

    Hi @nsano-rururu
    Hello all,
    I've been installed elastalert on elk7.11, I configured config and rules files, but alert are not getting on mails..
    please anyone have idea on help me out....

    1 rules loaded
    INFO:elastalert:Starting up
    INFO:elastalert:Disabled rules are: []
    INFO:elastalert:Sleeping for 59.999878 seconds
    INFO:elastalert:Queried rule Hello Test mail from ELK Stack please ignore from 2021-05-07 07:41 EDT to 2021-05-07 07:56 EDT: 0 / 0 hits
    INFO:elastalert:Ran Hello Test mail from ELK Stack please ignore from 2021-05-07 07:41 EDT to 2021-05-07 07:56 EDT: 0 query hits (0 already seen), 0 matches, 0 alerts sent
    INFO:elastalert:Background configuration change check run at 2021-05-07 07:57 EDT

    INFO:elastalert:Background alerts thread 0 pending alerts sent at 2021-05-07 07:57 EDT

    My config.yml:
    cat config.yaml|grep -v "^#"
    rules_folder: example_rules

    minutes: 1

    minutes: 15


    es_port: 9200

    writeback_index: elastalert_status
    writeback_alias: elastalert_alerts

    days: 0

    smtp_port: 25

    smtp_host: 'xxx.com'

    name: Hello Test mail from ELK Stack please ignore

    type: frequency

    index: filebeat-*

    num_events: 3

    hours: 1


    • query:
      query: "message: authentication failure OR failure password"
      timestamp_field: "@timestamp"

      - term:

      process_name: "DefaultQuartzScheduler6"


    • "email"


    Naoyuki Sano
    I don't use elastalert.