Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Hernando Agudelo
    @heagma_gitlab

    Any idea why my elastalert make twice a curl call just for 1 alert?

    curl -H 'Content-Type: application/json' -XPOST 'http://localhost:9200/elastalert_status_status/_doc?pretty' -d '{ "@timestamp": "2021-01-12T06:41:26.490784Z", "endtime": "2021-01-12T06:41:26.013716Z", "hits": 0, "matches": 0, "rule_name": "error_filelogs_stage", "starttime": "2021-01-12T06:36:26.013716Z", "time_taken": 0.4770529270172119 }'

    curl -H 'Content-Type: application/json' -XPOST 'http://localhost:9200/elastalert_status_status/_doc?pretty' -d '{ "@timestamp": "2021-01-12T06:41:26.490784Z", "endtime": "2021-01-12T06:41:26.013716Z", "hits": 0, "matches": 0, "rule_name": "error_filelogs_stage", "starttime": "2021-01-12T06:36:26.013716Z", "time_taken": 0.4770529270172119 }'

    PH03NIX
    @tarek_chalan_au_twitter
    @abhinath3399 show me the error if you still have issues with it
    serut
    @serut
    Hello ! I struggle to create a content for rabbitMQ. My message was correctly created into ES, but if I just take that value from ES and put it in a rabbitMQ message, the receiver cannot parse it as it miss some escape characters. I dont see any possibility on the templating, any solution ? Or do I need to retreate the message before posting it to ES to contains enough escape characters to build a JSON two fingers in the nose ?
    alert_text: '{{"logLevel": "{0}", "thread": "{1}", "class":"{2}",  "message": "{3}", "emitter": "{4}", "@timestamp": "{5}", "@log_name": "{6}", "stackTrace": "{7}", "logType": "{8}", "workflow": {9}}}'
    alert_text_type: alert_text_only
    alert_text_args:
      - logLevel
      - thread
      - class
      - message
      - emitter
      - '@timestamp'
      - '@log_name'
      - stackTrace
      - logType
      - workflow
    serut
    @serut
    Oh I think the issue is the triple quoted field on ES. It removes escape characters in what is stored inside ES. To sum up, I cannot send a JSON if the ES stored field is a triple quote, as escape characters won't be there ?
    Barry Salas
    @Sixxrett_twitter
    Okay so I though I could get elastalert to work if I used docker vice just a side by side install with my standing Elasticsearch 7.10 and I also got it to work in my test environment with a full docker environment but that is not the case with my production. In my production I am running CentOs 7 and elasticsearch with version 7.10. Even if I follow step by step using a docker or dockerless install elastalert will not install at all. What I mean by this is I pull the clone from Git
    ensure all requirements are installed including setuptools, run python3 setup.py install, fix the config.yaml file, fix a sudo rule for testing, and go to create index and that is where it stops and I cannot do anything further
    3 replies
    Vlad Vetshtein
    @vvvlad
    aws slack
    wajika
    @wajika

    Hello everyone, I got an error while testing the metric_aggregation rule. Does anyone have an answer?

    2021-01-28 15:40:24,352 INFO elasticsearch GET http://192.168.10.145:9200/elastalert_status_test/_search?size=1 [status:200 request:0.005s]
    2021-01-28 15:40:24,353 ERROR root Traceback (most recent call last):
    File "/usr/local/python36/lib/python3.6/site-packages/elastalert/elastalert.py", line 1269, in handle_rule_execution
    num_matches = self.run_rule(rule, endtime, rule.get('initial_starttime'))
    File "/usr/local/python36/lib/python3.6/site-packages/elastalert/elastalert.py", line 949, in run_rule
    self.add_aggregated_alert(match, rule)
    File "/usr/local/python36/lib/python3.6/site-packages/elastalert/elastalert.py", line 1807, in add_aggregated_alert
    alert_time = ts_now() + rule['aggregation']
    TypeError: unsupported operand type(s) for +: 'datetime.datetime' and 'dict'

    My rules file

    name: test
    type: metric_aggregation
    index: apm-*
    timeframe:
    minutes: 1
    metric_agg_key: transaction.duration.us
    metric_agg_type: avg
    query_key: service.name
    sync_bucket_interval: true
    allow_buffer_time_overlap: true
    use_run_every_query_size: true
    bucket_interval:
    seconds: 10
    doc_type: _doc
    max_threshold: 100000
    filter:

    • term:
      agent.name: "dotnet"
      alert:
      • "debug"
    4 replies
    ocelino
    @ocelino
    Hi all, is there a way to send email report every 5 minutes using elastalert and the body is formatted like this:
    API Calls: 2,872
    Resp. Time: 449 ms
    Error Rate: 0.38%
    Ramesh Pradhan
    @rms-sth
    Does latest version of elastalert support kibana_url and use_kibana_dashboard feature for kibana 7.x ?
    7 replies
    Bhanupraveen G
    @bhanupraveeng
    elastalert-test-rule example_rules/example_frequency.yaml by default using " File "/usr/lib/python2.7/site-packages/pkg_resources/init.py"". failing with syntax error. I need to use python3.6 package which is installed. Please let me know?
    1 reply
    Ashish Singh
    @ashishkaransingh

    Hello All,
    I was trying to modify
    alerts.py (/elastalert/alerts.py)
    https://github.com/Yelp/elastalert/blob/1dc4f30f30d39a689f419ce19c7e2e4d67a50be3/elastalert/alerts.py#L1798

    Reason: in order to send events to Servicenow
    To muck around i changed the payload, essentially hardcoded the values except description (and it works it triggers an alert and creates an event in Servicenow).
    https://github.com/Yelp/elastalert/blob/1dc4f30f30d39a689f419ce19c7e2e4d67a50be3/elastalert/alerts.py#L1809
    payload = {
    "records":[{
    "source": "oursource",
    "event_class": "Log_Analytics",
    "node": "comp1",
    "metric_name": "Interface",
    "type": "Details hardcoded",
    "resource": "resource",
    "severity": "3",
    "description": "This is dummy description",
    "additional_info": description
    }]}

    My question is how can i fetch values dynamically from Elasticsearch?
    Example node is a Host.keyword in elasticsearch how can i fetch it?
    I notice description has most of the data already how can make use of it?

    Any help would be great!!

    Thank you

    Ashish Singh
    @ashishkaransingh
    I guess i should ask how can i access the dict object from matches
    https://github.com/Yelp/elastalert/blob/1dc4f30f30d39a689f419ce19c7e2e4d67a50be3/elastalert/alerts.py#L1798
    def alert(self, matches):
    for match in matches:
            # Parse everything into description.
            description = str(BasicMatchString(self.rule, match))
    Ashish Singh
    @ashishkaransingh

    Notice i tried to add "node": match.Host, but got an error

    File \"/opt/elastalert/elastalert/alerts.py\", line 1815, in alert", " \"node\": match.Host,", "AttributeError: 'dict' object has no attribute 'Host'"],

    payload = {
    "records":[{
    "source": "SupportOne",
    "event_class": "Log_Analytics",
    "node": match.Host,
    "metric_name": "Interface",
    "type": "Details hardcoded",
    "resource": "resource",
    "severity": "3",
    "description": "This is dummy description_222",
    "additional_info": description
    }]}

    Ashish Singh
    @ashishkaransingh

    Looks like this is it
    lookup_es_key

    node = lookup_es_key(matches[0], 'Host')
    message = lookup_es_key(matches [0], 'Message')

    then use it in the payload

    beaesteban01
    @beaesteban01

    Hi everyone!
    I am working with a few frequency rules but I am facing the following problem.
    Setting the number of events to 3, as the documentation says "This rule matches when there are at least a certain number of events in a given time frame", if I have more than 3 hits, all of the hits that are in the timeframe should be included in the same alert, not triggering new alerts only each 3 hits, that is what I am getting: matches for only the exact num_events, not more. My rule:

    name: r1_redmov
    description: "3 events (any) redes moviles  in 2 minutes"
    
    type: frequency
    
    # (Required)
    # Index to search, wildcard supported
    index: redes_moviles
    
    # If different than default
    timestamp_field: time_stamp
    num_events: 3
    timeframe:
      minutes: 2
    
    # to see all the hits related to the match
    attach_related: true  
    
    alert:
    - "debug"

    Moreover, looking into the ruletypes.py, in the frequency rule, the condition is if self.occurrences[key].count() >= self.rules['num_events’]:

    Thanks!!

    2 replies
    Parul Kushwaha
    @Parulk
    Hi everyone! I am creating this below elastalert rule :
    rule1.yaml: |-
    # elasticsearch host
    es_host: my_host
    es_port: my_port
    name: Error Log Alerts
    type: frequency
    num_events: 1
    timeframe:
      hours: 1
    
    index: testindex-applicationlogs-*
    
    filter:
    - query:
        query_string:
          query: "appname:xxx AND host_stability:prod AND (function:web.RemotingServlet.handleThrowable* OR loglevel:ERROR)"
    
    
    alert:
    - "slack"
    alert_subject: "Error Log Alerts"
    slack:
    slack_webhook_url: "slack_webhook_url"
    I am getting this exception while running the rule : ERROR:root:Uncaught exception running rule Error Log Alerts: TransportError(429, 'circuit_breaking_exception', '[parent] Data too large, data for [<http_request>] would be [31155871676/29gb], which is larger than the limit of [31045425561/28.9gb], real usage: [31155870848/29gb], new bytes reserved: [828/828b], usages [request=0/0b, fielddata=60671194/57.8mb, in_flight_requests=9705526/9.2mb, accounting=223410932/213mb]')
    Wanted to understand how to introduce the timestamp in filter ?
    Something like to query for last 1 hour.
    ShotgunSpider
    @ShotgunSpider
    if i want to get a 40% drop in events using spike_alert should the spike_height be 1.6 for 40% and spike_type be down?
    Ashish Singh
    @ashishkaransingh

    Hi,

    Need help in accessing alert or rule types used in .yaml
    /elastralert/elastalert/alerts.py
    I am able to access and print values filter and name.

    image.png

    How can i access alert type example whether it is frequency or Average etc
    I tried as get_rules = self.rule.get('type', None) but it print the following line including the class etc
    "Print rules: <elastalert.ruletypes.FrequencyRule object at 0x7fd3e204c310>"

    A little background:
    I am creating a custom payload for class ServiceNowAlerter(Alerter): in alerts.py

    RashmiChoudhary
    @rashmichoudhary07
    Hello @Qmando Hope you are doing well...
    I have an issue in production environment(at work) where elastalert is showing a high time_taken values in elastalert_status_status index. This happens intermittenly for a continuous timeframe during certain times in a day which is troubling my client since eventually real time alerts are delayed. I have been trying to find Root cause for the delay introduced at elastalert but no luck since it is intermittent. Please find a sample document from prod for your help.@timestamp: 2021-02-18T07:45:16.048284Z
    _id: SZkZtHcBUW6Vb-ZGp3YW
    _index: elastalert_status_status
    _type: _doc
    endtime: 2021-02-18T06:03:42.463717Z
    hits: 407
    matches: 34
    num_hits: 3
    num_matches: 1
    rule_name: Shop - Severity - Minor [400 TO 499] And Not [401,403,424]
    starttime: 2021-02-18T05:38:43.224775Z
    time_taken: 6093.584537982941Is time_taken the exact time for elastalert to only query elasticsearch?
    We have millions of documents in our ES and Kibana gives faster result for the same query for time_taken is a high value. Also, if i manually run the cURL query that elastalert generates the result is fetched only with in few 100 milliseconds. We are seeing 1 to 4+ hours to time_taken value in producton and unable to get to the root cause of this.Could please help us to get to the root of high time_taken value. I am stuck in a real prod issue here with elastalert delaying alert triggers due to huge time_taken value(in seconds as per doc). Looking forward to hearing from you.
    I have trying to research on what is causing the above but no luck yet
    i have also opened ticket at github issues site
    RashmiChoudhary
    @rashmichoudhary07
    i am new to this platform Any help from experts is highly appreciated.. TIA!
    Ashish Singh
    @ashishkaransingh

    Hello,
    Anybody running Elastalert using python 3.8
    I am getting error

    python3 -m elastalert.elastalert --verbose --rule test_count_alert.yaml --config config.yaml

    Throwing error
    
    Traceback (most recent call last):
      File "/usr/lib/python3.8/runpy.py", line 185, in _run_module_as_main
        mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
      File "/usr/lib/python3.8/runpy.py", line 111, in _get_module_details
        __import__(pkg_name)
     File "/opt/elastalert/elastalert/elastalert.py", line 29, in <module>
        from . import kibana
    ImportError: attempted relative import with no known parent package

    Ref
    https://github.com/Yelp/elastalert/issues/2439#issuecomment-534058583
    Yelp/elastalert#2776

    Ashish Singh
    @ashishkaransingh
    I figured it out...
    Kumar ankit
    @Ankitxroot

    pkg_resources.DistributionNotFound: The 'elastalert==0.1.28' distribution was not found and is required by the application

    How to fix this.
    Thanks in advance.

    4 replies
    danny zak
    @dannyzak_gitlab
    is there a way to specify a timewindow when alerting should (not) occure for a certrain rule, eg. not after friday 5pm till monday 9am
    Kyle Parrish
    @arnydo

    Hiya! Anyone have any feedback on building filters? I am trying to create an alert for Office 365 sign ins outside of the US. I am trying thie:

    filter:
    - query:
            query_string: {query: "NOT source.geo.country_iso_code:US"}
            query_string: {query: 'o365.audit.Operation: UserLoggedIn'}

    But...it doesn't seem to work as expected. I can't find documentation on building boolean filters. "this AND NOT that..."

    7 replies
    Kyle Parrish
    @arnydo
    Also, why are there hits when running elastalert-test-rule but not elastalert? Both are using the same rule.
    ankita1596
    @ankita1596
    My elastalert and elasticsearch is running in docker environment. Can you please help me to send output from elastalert to zabbix running in docker.
    dunter-d
    @dunter-d
    hi! I'm having issues with elastalert. Running ubuntu 20.04/elasticsearch 7.10/python3.8 & running systemctl start elastalert outputs this:
    '''
    rule['alert'] = self.load_alerts(rule, alert_field=rule['alert'])
    File "/usr/local/lib/python3.8/dist-packages/elastalert-0.2.4-py3.8.egg/elastalert/loaders.py", line 469, in load_alerts
    alert_field = [normalize_config(x) for x in alert_field]
    File "/usr/local/lib/python3.8/dist-packages/elastalert-0.2.4-py3.8.egg/elastalert/loaders.py", line 469, in <listcomp>
    alert_field = [normalize_config(x) for x in alert_field]
    File "/usr/local/lib/python3.8/dist-packages/elastalert-0.2.4-py3.8.egg/elastalert/loaders.py", line 450, in normalize_config
    config_copy.update(config) # warning, this (intentionally) mutates the rule dict
    ValueError: dictionary update sequence element #0 has length 1; 2 is required
    elastalert.service: Main process exited, code=exited, status=1/FAILURE
    '''
    dunter-d
    @dunter-d
    @ashishkaransingh how did you solved that issue with the "from . import kibana" error?
    dplgrail
    @dplgrail
    image.png
    Hi, i've this issue... TestController: Failed to test rule with error: INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
    To send them but remain verbose, use --verbose instead.
    Im using SIEMonster and Praeco for ElastAlert
    1 reply
    raks25
    @raks25:matrix.org
    [m]
    Hi can anyone help me out with this "1|INFO|0|1|ibs| |2021/04/22 18:54:30.999234|http: TLS handshake error from **: remote error: tls: bad certificate" log displayed on my kibana and i am using elastalert for notifying the string " tls: bad certificate" but it sends alerts when we have word like only certificate in it , but i want to be notified only when the log has keyword "bad certificate"
    Naoyuki Sano
    @nsano-rururu
    Yelp/elastalert is no longer maintained. Please use jertel/elastalert2. Questions to the discussion below
    https://github.com/jertel/elastalert2/discussions
    itruivitorino
    @itruivitorino
    Hi everyone. I've been trying to make an alert go when the status of a heartbeat monitor goes up AFTER being down. Anyone made any progress with similar thing?
    ramprasadavirineni
    @ramprasadavirineni:matrix.org
    [m]

    Hi @nsano-rururu
    Hello all,
    I've been installed elastalert on elk7.11, I configured config and rules files, but alert are not getting on mails..
    please anyone have idea on help me out....

    1 rules loaded
    INFO:elastalert:Starting up
    INFO:elastalert:Disabled rules are: []
    INFO:elastalert:Sleeping for 59.999878 seconds
    INFO:elastalert:Queried rule Hello Test mail from ELK Stack please ignore from 2021-05-07 07:41 EDT to 2021-05-07 07:56 EDT: 0 / 0 hits
    INFO:elastalert:Ran Hello Test mail from ELK Stack please ignore from 2021-05-07 07:41 EDT to 2021-05-07 07:56 EDT: 0 query hits (0 already seen), 0 matches, 0 alerts sent
    INFO:elastalert:Background configuration change check run at 2021-05-07 07:57 EDT

    INFO:elastalert:Background alerts thread 0 pending alerts sent at 2021-05-07 07:57 EDT

    My config.yml:
    cat config.yaml|grep -v "^#"
    rules_folder: example_rules

    run_every:
    minutes: 1

    buffer_time:
    minutes: 15

    es_host: 192.168.0.1

    es_port: 9200

    writeback_index: elastalert_status
    writeback_alias: elastalert_alerts

    alert_time_limit:
    days: 0

    smtp_port: 25

    smtp_host: 'xxx.com'

    name: Hello Test mail from ELK Stack please ignore

    type: frequency

    index: filebeat-*

    num_events: 3

    timeframe:
    hours: 1

    filter:

    • query:
      query_string:
      query: "message: authentication failure OR failure password"
      timestamp_field: "@timestamp"

      - term:

      process_name: "DefaultQuartzScheduler6"

    alert:

    • "email"

    email:

    Naoyuki Sano
    @nsano-rururu
    @ramprasadavirineni:matrix.org
    I don't use elastalert.
    ramprasadavirineni
    @ramprasadavirineni:matrix.org
    [m]

    Hello,

    could you please suggest which one good for alert notification for ELK 7.x

    Mohd Rashid
    @MohdRashid01

    Hi All,

    I have install ELK in my laptop and after that i configure elastialert for triggering email like issue coming into docker container so it will trigger an email to my gmail account for that i setup rules.yml file in kibana plugin of elastialert after that it send an email which u can see above one content in that email i want to remove which i have mention above word. Only Message should show in my email whenever it trigger How to do it, Please do let me ASAP

    Below is the content which im getting in my gmail inbox. I want to

    remove _id, _index, _type, num_matches, num_hits and @timestamp

    all this only i want message to be include in my email triggering everytime.

    Below is getting into gmail inbox

    @timestamp: 2021-05-24T11:16:06Z
    _id: 2S0WnnkBz7SOxaiw1TZk
    _index: logstash-2021.05.24
    _type: _doc
    message: <30>May 24 11:16:06 fx-prod-1 prod_fx-control-plane.1.knel5yam 2021-05-24 11:16:06.926 INFO 1 --- [nio-8080-exec-9] com.fxlabs.fxt.rest.run.RunController : Find Latest by job id [8a8089ba777311370177734530902ec8] org [8a8081066e02d6a2016e04eacd2005c7] principal [8a808155647d283a01647d7c5e0d07ba]
    num_hits: 8
    num_matches: 1

    and

    Here is the rules.yaml file which i used to trigger email for issue

    # Alert when the rate of events exceeds a threshold
    
    # (Optional)
    # Elasticsearch host
    # es_host: elasticsearch.example.com
    
    es_host: elasticsearch.test.com
    # (Optional)
    # Elasticsearch port
    #es_port: 14900
    es_port: 9200
    
    # (OptionaL) Connect with SSL to Elasticsearch
    #use_ssl: True
    
    # (Optional) basic-auth username and password for Elasticsearch
    #es_username: someusername
    #es_password: somepassword
    es_username: testelastic
    es_password: xxxx
    
    
    # (Required)
    # Rule name, must be unique
    name: Exception Alert
    
    # (Required)
    # Type of alert.
    # the frequency rule type alerts when num_events events occur with timeframe time
    #type: blacklist
    type: any
    include: ["message"]
    
    # (Required)
    # Index to search, wildcard supported
    #index: logstash*
    index: filebeat*
    
    # (Required, frequency specific)
    # Alert when this many documents matching the query occur within a timeframe
    #num_events: 1
    
    # (Required, frequency specific)
    # num_events must occur within this amount of time to trigger an alert
    timeframe:
    #  hours: 1
    #  minutes: 1
      seconds: 1
    
    #compare_key: "message"
    
    #blacklist:
    
    #- "error"
    
    realert:
      minutes: 5
    
     # This is send out all matches in one email
    # aggregation:
    #  minutes: 1
    
    #  seconds: 0
    #filter:
    #- term:
    #    message: "[error]"
    
    #filter:
    # - match:
     #   message: "job"
    
    filter:
     - query:
          query_string:
     #       query: "message: exception AND  message: control"
            query: "message: job"
    #filter:
    #- query:
    #   query_string:
    #    query: "message: error"
    
    alert:
    
    - "slack"
    - "email"
    
    
    #- slack
    slack_webhook_url: "https://hooks.slack.com/services/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"
    #slack_username_override: "ElastAlert"
    slack_username_override: "Mohd Rashid"
    
    #- email
    email: ["rashidmd777@gmail.com"]
    smtp_host: "smtp.gmail.com"
    smtp_port: "587"
    from_addr: "rashidmd777@gmail.com"

    How to do it, Please do let me ASAP

    Mohd Rashid
    @MohdRashid01
    ?
    Naoyuki Sano
    @nsano-rururu