Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    matrixbot
    @matrixbot
    @ptman:kapsi.fi kibitzr was called an alternative to IFTTT
    @ptman:kapsi.fi any good examples?
    @ptman:kapsi.fi and also, notification support for matrix could be nice, and quite easy, a single HTTP(S) call
    @ptman:kapsi.fi (with JSON)
    Peter Demin
    @peterdemin
    Hi @ptman would you consider these as examples? https://kibitzr.readthedocs.io/en/latest/recipes.html It's not really comparing to IFTT, but gives an idea of what's possible.
    Botter
    @BotterAir_gitlab
    hey,
    anyone having issues with telegram? i am pretty sure my token and chatid (negative chat id) are correct by using curl
    however in kibitzr:
    2020-08-09 19:56:30,241 [DEBUG] telegram.vendor.ptb_urllib3.urllib3.connectionpool: https://api.telegram.org:443 "POST /botXXXXX:YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY/sendMessage HTTP/1.1" 401 58 2020-08-09 19:56:30,241 [ERROR] kibitzr.notifier.factory: Exception occurred during sending notification Traceback (most recent call last): File "c:\python38\lib\site-packages\kibitzr\notifier\factory.py", line 69, in notify notifier(report=report) File "c:\python38\lib\site-packages\kibitzr\notifier\telegram.py", line 44, in post self.send_message(m) File "c:\python38\lib\site-packages\kibitzr\notifier\telegram.py", line 47, in send_message message = self.bot.send_message( File "<decorator-gen-2>", line 2, in send_message File "c:\python38\lib\site-packages\telegram\bot.py", line 67, in decorator result = func(*args, **kwargs) File "c:\python38\lib\site-packages\telegram\bot.py", line 346, in send_message return self._message(url, data, disable_notification=disable_notification, File "c:\python38\lib\site-packages\telegram\bot.py", line 175, in _message result = self._request.post(url, data, timeout=timeout) File "c:\python38\lib\site-packages\telegram\utils\request.py", line 330, in post result = self._request_wrapper('POST', url, File "c:\python38\lib\site-packages\telegram\utils\request.py", line 242, in _request_wrapper raise Unauthorized(message) telegram.error.Unauthorized: Unauthorized
    Peter Demin
    @peterdemin
    Hmm. I’ve been using telegram integration for quite some time, and it works fine for me. I’m likely to have outdated library versions though. Can’t tell the numbers, as I’m on vacation atm
    Botter
    @BotterAir_gitlab
    @peterdemin no worries, i ended up nuking everything and with the same kibitzr.yml it works fine. cant complain :D
    On to the next issue : how to cut up the result of transform (ersult being a list of urls) into seperate notify calls
    lesha-dev
    @lesha-dev
    Hi. Can someone help me with gmail notifications? I'm trying to notify via smtp: I set up just like in docs (https://kibitzr.readthedocs.io/en/latest/smtp.html), set port to the one from gmail settings page and authorized blocked attempt to login. And still I'm getting the same error: smtplib.SMTPServerDisconnected: Connection unexpectedly closed. How can I fix this?
    Peter Demin
    @peterdemin
    @BotterAir_gitlab for more than one notification, you’ll need to use python or shell notifier and put this logic explicitly. Kibitzr doesn’t support this out of the box.
    @lesha-dev I’ll need to check it next week. Could you please provide kibitzr.yml if it’s anyhow different from the docs?
    lesha-dev
    @lesha-dev
    kibitzr.yml:
      - name: Current Time
        url: https://www.worldtimeserver.com/current_time_in_US-NY.aspx
        transform:
          - css: "span#theTime"
          - text
        notify:
          - python: print(content)
          - smtp:
              subject: Kibitzr current time
              recepients:
                - bla-bla-bla@gmail.com
        period: 3
    kibitzr-creds.yml:
        host: smtp.gmail.com
        port: 465
        user: bla-bla-bla@gmail.com
        password: something
    Botter
    @BotterAir_gitlab
    @lesha-dev i believe this might be due to google blocking thirdparty. you have to enable "allow unsafe applications" or something like that
    did you get a warning in the gmail account about unauthorized application
    ?
    lesha-dev
    @lesha-dev
    @BotterAir_gitlab , I got a warning from gmail and after that allowed unsafe applications in the account. Didn't help.
    Peter Demin
    @peterdemin
    @lesha-dev I just checked my kibitzr-creds.yml and noticed, that SMTP port is 587. Could you please check it solves the issue?
    Peter Demin
    @peterdemin
    Also, I fixed typo in s/recepients/recipients/ in the check. I received the email with current time then.
    Peter Demin
    @peterdemin
    lesha-dev
    @lesha-dev
    @peterdemin , port 687 worked. It's strange, why 465 isn't working, I copied it from gmail help page with settings for mail clients. Also new error appears - SMTPRecipientsRefused, is it caused by typo in recipients? When will this fix be released?
    Peter Demin
    @peterdemin
    It is released. The typo was in docs
    lesha-dev
    @lesha-dev
    @peterdemin , how to solve following error?
    2020-08-19 16:23:13,176 [ERROR] kibitzr.notifier.smtp: Failed to send the e-mail Traceback (most recent call last): File "M:\bin\kibitzr\.venv\lib\site-packages\kibitzr\notifier\smtp.py", line 77, in send_email server.sendmail(user, recipients, message.encode("utf-8")) File "c:\Python37\lib\smtplib.py", line 881, in sendmail raise SMTPRecipientsRefused(senderrs) smtplib.SMTPRecipientsRefused: {'subject': (553, b'5.1.3 The recipient address <subject> is not a valid RFC-5321 address. s4sm7297552lfc.56 - gsmtp'), 'recepients': (553, b'5.1.3 The recipient address <recepients> is not a valid RFC-5321 address. s4sm7297552lfc.56 - gsmtp')}
    Why does it load smtplib.py from the system? Local .venv is activated (M:\bin\kibitzr.venv)
    Peter Demin
    @peterdemin
    Builtin libraries are shared with virtualenvs. You need to change "recepients" to "recipients" in the config.
    lcq92
    @lcq92
    Hi! I wanted to thank you for making this project. (I've always wanted a way to create scrapers/notifiers combos quickly without always re-writings the same python scripts)
    I've got a question: is it possible to use the changes transform tool, but only output the new content variable (not the Old & New way using changes: verbose) ?
    Peter Demin
    @peterdemin
    Yes, changes: new
    Rafael Perez
    @rafaelperez
    I've used the kibitzr firefox to log in the websites I want to check. How can I make sure the recipe uses the selenium driver?
    Peter Demin
    @peterdemin
    @rafaelperez Selenium driver kicks in if one of these fields is defined for the check: delay, scenario, form. If you don't need any of them, I'd suggest adding delay: 0.1
    https://github.com/kibitzr/kibitzr/blob/ebcbd76e4618fa3adcdbcedf033e36049f64049e/kibitzr/fetcher/loader.py#L40-L44
    lcq92
    @lcq92
    :point_up: August 20, 2020 3:00 PM Thanks! I indeed found where it is in the source code
    lcq92
    @lcq92

    I've also got a question regarding the selenium driver:

    If I understood it correctly: using kibitzr run, on second, third etc. executions of jobs that use selenium, the driver is never closed, it uses the previous open one. I see how ideally this should be the preferred way, as far as resource usage is concerned, but when "planning" detailed scenarios using python selenium code, I have to add if-else clauses I didn't except, that check if some buttons have "already" been clicked... Is there an option to specify kibitzr to really close drivers and re-create new ones at each job exectution when using using kibitzr run?

    Rafael Perez
    @rafaelperez
    Thanks, @peterdemin . Using delay: 0.1 solved perfectly my problem!
    Peter Demin
    @peterdemin
    @lcq92 sounds like you want to start from the freshly loaded profile each time
    Currently, Kibitzr doesn't have that option in config. There are 2 ways: you can use kibitzr once and schedule the job externally, say using crontab. Or you can contribute the config option to Kibitzr source.
    Paul Massendari
    @paulmassen
    Hi ! Just discovered kibitzr, great tool, bravo 👍I was wondering whether there is somewhere a "cookbook" with more examples of kibitzr.yml files ?
    Peter Demin
    @peterdemin
    Thanks, I think https://kibitzr.readthedocs.io/en/latest/recipes.html is intended for this purpose. The problem is that in my experience kibitzr rules are so customized and personal, that it's hard to find shareable ones.
    Paul Massendari
    @paulmassen
    Many thanks @peterdemin , these recipes are helpful indeed, although I would be glad to read some other use cases. I would gladly share my file once everything is set up. At the moment I made a check that notify me when someone upload a video of my favorite artist on youtube, and am currently working on a check that would send me the status of the air quality in my city (Paris) via telegram with some nice icons etc.
    Peter Demin
    @peterdemin
    Some more involved examples are linked at https://kibitzr.github.io/ in Explore section. I like your use-case, would love to see how you've done it!
    Paul Massendari
    @paulmassen
    My youtube checker is very simple:
    checks:
      - name: Bob Wilber
        url: https://www.youtube.com/results?search_query=bob+wilber&sp=CAI%253D
        delay: 5
        transform:
          - css: "#video-title.ytd-video-renderer"
          - text
          - changes
        notify:
          - telegram
        period: 12 hours
    Now I'm a bit stuck with string concatenation and Telegram notification
    Paul Massendari
    @paulmassen
    In this example: https://gist.github.com/peterdemin/49610071aed04ddc5bf40a2b6d2c4d9c
    Where does the lines variable comes from?
    ah, I reread the docs and now I understand
    Paul Massendari
    @paulmassen
    Got it working, here is my yml, it was very simple:
      - name: Air Quality Today in Paris
        url: https://www.airparif.asso.fr/accueil-airparif
        delay: 3
        transform:
          - css-all: ".indice-color.text-light"
          - text
          - jinja: |
              🇫 🇷  Pollution in Paris 🇫 🇷
              📅 **Today**📅
              Ozone:  {{ lines.0.lower() }}
              Dioxyde d'Azote: {{ lines.1.lower() }}
              Particules PM10:{{ lines.2.lower() }}
              Particules PM2: {{ lines.3.lower() }}
              📅**Tomorrow**📅
              Ozone:  {{ lines.4.lower() }}
              Dioxyde d'Azote: {{ lines.5.lower() }}
              Particules PM10:{{ lines.6.lower() }}
              Particules PM2: {{ lines.7.lower() }}
        notify:
          - telegram
    Peter Demin
    @peterdemin
    Nice! Thanks for sharing! Would you mind adding it to the docs in a PR?
    Paul Massendari
    @paulmassen
    Sure, will do
    Paul Massendari
    @paulmassen
    And I have tons of other ideas for the doc: I'm thinking of a check that will send me the most popular tv show this week every week with their rating, a check to see if an artist has announced new concert dates etc. I will PR this too if you want.
    Peter Demin
    @peterdemin
    Sounds cool!
    Paul Massendari
    @paulmassen
    Hi @peterdemin , do you have any idea why I'm not able to display the value from Stash ?
    checks:
      - name: Fetch Rating TV Shows
        url: https://www.imdb.com/chart/tvmeter
        transform:
          - css-all: .imdbRating>strong
          - text
        notify:
          - stash:
              rating: '{{ lines }}'
    
      - name: Popular TV Shows
        url: https://www.imdb.com/chart/tvmeter
        transform:
          - css-all: .titleColumn>a
          - text
          - jinja: |
               Popular TV shows this week
               {{ lines.0.lower()|title}} - {{ stash.rating|first }}
               {{ lines.1.lower()|title }} - {{ stash["rating"][0] }}
               {{ lines.2.lower()|title }}  - {{ stash["rating"].0 }}
               {{ lines.3.lower()|title }}
               {{ lines.4.lower()|title }}
               {{ lines.5.lower()|title }}
               {{ lines.6.lower()|title }}
               {{ lines.7.lower()|title }}
               {{ lines.8.lower()|title }}
               {{ lines.9.lower()|title }}
               {{ lines.10.lower()|title }}
        notify:
          - telegram
    I'm only able to display the whole list with {{ stash["rating"] }} but I can't access any individual value