Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Peter Demin
    @peterdemin
    Hmm. I’ve been using telegram integration for quite some time, and it works fine for me. I’m likely to have outdated library versions though. Can’t tell the numbers, as I’m on vacation atm
    Botter
    @BotterAir_gitlab
    @peterdemin no worries, i ended up nuking everything and with the same kibitzr.yml it works fine. cant complain :D
    On to the next issue : how to cut up the result of transform (ersult being a list of urls) into seperate notify calls
    lesha-dev
    @lesha-dev
    Hi. Can someone help me with gmail notifications? I'm trying to notify via smtp: I set up just like in docs (https://kibitzr.readthedocs.io/en/latest/smtp.html), set port to the one from gmail settings page and authorized blocked attempt to login. And still I'm getting the same error: smtplib.SMTPServerDisconnected: Connection unexpectedly closed. How can I fix this?
    Peter Demin
    @peterdemin
    @BotterAir_gitlab for more than one notification, you’ll need to use python or shell notifier and put this logic explicitly. Kibitzr doesn’t support this out of the box.
    @lesha-dev I’ll need to check it next week. Could you please provide kibitzr.yml if it’s anyhow different from the docs?
    lesha-dev
    @lesha-dev
    kibitzr.yml:
      - name: Current Time
        url: https://www.worldtimeserver.com/current_time_in_US-NY.aspx
        transform:
          - css: "span#theTime"
          - text
        notify:
          - python: print(content)
          - smtp:
              subject: Kibitzr current time
              recepients:
                - bla-bla-bla@gmail.com
        period: 3
    kibitzr-creds.yml:
        host: smtp.gmail.com
        port: 465
        user: bla-bla-bla@gmail.com
        password: something
    Botter
    @BotterAir_gitlab
    @lesha-dev i believe this might be due to google blocking thirdparty. you have to enable "allow unsafe applications" or something like that
    did you get a warning in the gmail account about unauthorized application
    ?
    lesha-dev
    @lesha-dev
    @BotterAir_gitlab , I got a warning from gmail and after that allowed unsafe applications in the account. Didn't help.
    Peter Demin
    @peterdemin
    @lesha-dev I just checked my kibitzr-creds.yml and noticed, that SMTP port is 587. Could you please check it solves the issue?
    Peter Demin
    @peterdemin
    Also, I fixed typo in s/recepients/recipients/ in the check. I received the email with current time then.
    Peter Demin
    @peterdemin
    lesha-dev
    @lesha-dev
    @peterdemin , port 687 worked. It's strange, why 465 isn't working, I copied it from gmail help page with settings for mail clients. Also new error appears - SMTPRecipientsRefused, is it caused by typo in recipients? When will this fix be released?
    Peter Demin
    @peterdemin
    It is released. The typo was in docs
    lesha-dev
    @lesha-dev
    @peterdemin , how to solve following error?
    2020-08-19 16:23:13,176 [ERROR] kibitzr.notifier.smtp: Failed to send the e-mail Traceback (most recent call last): File "M:\bin\kibitzr\.venv\lib\site-packages\kibitzr\notifier\smtp.py", line 77, in send_email server.sendmail(user, recipients, message.encode("utf-8")) File "c:\Python37\lib\smtplib.py", line 881, in sendmail raise SMTPRecipientsRefused(senderrs) smtplib.SMTPRecipientsRefused: {'subject': (553, b'5.1.3 The recipient address <subject> is not a valid RFC-5321 address. s4sm7297552lfc.56 - gsmtp'), 'recepients': (553, b'5.1.3 The recipient address <recepients> is not a valid RFC-5321 address. s4sm7297552lfc.56 - gsmtp')}
    Why does it load smtplib.py from the system? Local .venv is activated (M:\bin\kibitzr.venv)
    Peter Demin
    @peterdemin
    Builtin libraries are shared with virtualenvs. You need to change "recepients" to "recipients" in the config.
    lcq92
    @lcq92
    Hi! I wanted to thank you for making this project. (I've always wanted a way to create scrapers/notifiers combos quickly without always re-writings the same python scripts)
    I've got a question: is it possible to use the changes transform tool, but only output the new content variable (not the Old & New way using changes: verbose) ?
    Peter Demin
    @peterdemin
    Yes, changes: new
    Rafael Perez
    @rafaelperez
    I've used the kibitzr firefox to log in the websites I want to check. How can I make sure the recipe uses the selenium driver?
    Peter Demin
    @peterdemin
    @rafaelperez Selenium driver kicks in if one of these fields is defined for the check: delay, scenario, form. If you don't need any of them, I'd suggest adding delay: 0.1
    https://github.com/kibitzr/kibitzr/blob/ebcbd76e4618fa3adcdbcedf033e36049f64049e/kibitzr/fetcher/loader.py#L40-L44
    lcq92
    @lcq92
    :point_up: August 20, 2020 3:00 PM Thanks! I indeed found where it is in the source code
    lcq92
    @lcq92

    I've also got a question regarding the selenium driver:

    If I understood it correctly: using kibitzr run, on second, third etc. executions of jobs that use selenium, the driver is never closed, it uses the previous open one. I see how ideally this should be the preferred way, as far as resource usage is concerned, but when "planning" detailed scenarios using python selenium code, I have to add if-else clauses I didn't except, that check if some buttons have "already" been clicked... Is there an option to specify kibitzr to really close drivers and re-create new ones at each job exectution when using using kibitzr run?

    Rafael Perez
    @rafaelperez
    Thanks, @peterdemin . Using delay: 0.1 solved perfectly my problem!
    Peter Demin
    @peterdemin
    @lcq92 sounds like you want to start from the freshly loaded profile each time
    Currently, Kibitzr doesn't have that option in config. There are 2 ways: you can use kibitzr once and schedule the job externally, say using crontab. Or you can contribute the config option to Kibitzr source.
    Paul Massendari
    @paulmassen
    Hi ! Just discovered kibitzr, great tool, bravo 👍I was wondering whether there is somewhere a "cookbook" with more examples of kibitzr.yml files ?
    Peter Demin
    @peterdemin
    Thanks, I think https://kibitzr.readthedocs.io/en/latest/recipes.html is intended for this purpose. The problem is that in my experience kibitzr rules are so customized and personal, that it's hard to find shareable ones.
    Paul Massendari
    @paulmassen
    Many thanks @peterdemin , these recipes are helpful indeed, although I would be glad to read some other use cases. I would gladly share my file once everything is set up. At the moment I made a check that notify me when someone upload a video of my favorite artist on youtube, and am currently working on a check that would send me the status of the air quality in my city (Paris) via telegram with some nice icons etc.
    Peter Demin
    @peterdemin
    Some more involved examples are linked at https://kibitzr.github.io/ in Explore section. I like your use-case, would love to see how you've done it!
    Paul Massendari
    @paulmassen
    My youtube checker is very simple:
    checks:
      - name: Bob Wilber
        url: https://www.youtube.com/results?search_query=bob+wilber&sp=CAI%253D
        delay: 5
        transform:
          - css: "#video-title.ytd-video-renderer"
          - text
          - changes
        notify:
          - telegram
        period: 12 hours
    Now I'm a bit stuck with string concatenation and Telegram notification
    Paul Massendari
    @paulmassen
    In this example: https://gist.github.com/peterdemin/49610071aed04ddc5bf40a2b6d2c4d9c
    Where does the lines variable comes from?
    ah, I reread the docs and now I understand
    Paul Massendari
    @paulmassen
    Got it working, here is my yml, it was very simple:
      - name: Air Quality Today in Paris
        url: https://www.airparif.asso.fr/accueil-airparif
        delay: 3
        transform:
          - css-all: ".indice-color.text-light"
          - text
          - jinja: |
              🇫 🇷  Pollution in Paris 🇫 🇷
              📅 **Today**📅
              Ozone:  {{ lines.0.lower() }}
              Dioxyde d'Azote: {{ lines.1.lower() }}
              Particules PM10:{{ lines.2.lower() }}
              Particules PM2: {{ lines.3.lower() }}
              📅**Tomorrow**📅
              Ozone:  {{ lines.4.lower() }}
              Dioxyde d'Azote: {{ lines.5.lower() }}
              Particules PM10:{{ lines.6.lower() }}
              Particules PM2: {{ lines.7.lower() }}
        notify:
          - telegram
    Peter Demin
    @peterdemin
    Nice! Thanks for sharing! Would you mind adding it to the docs in a PR?
    Paul Massendari
    @paulmassen
    Sure, will do
    Paul Massendari
    @paulmassen
    And I have tons of other ideas for the doc: I'm thinking of a check that will send me the most popular tv show this week every week with their rating, a check to see if an artist has announced new concert dates etc. I will PR this too if you want.
    Peter Demin
    @peterdemin
    Sounds cool!
    Paul Massendari
    @paulmassen
    Hi @peterdemin , do you have any idea why I'm not able to display the value from Stash ?
    checks:
      - name: Fetch Rating TV Shows
        url: https://www.imdb.com/chart/tvmeter
        transform:
          - css-all: .imdbRating>strong
          - text
        notify:
          - stash:
              rating: '{{ lines }}'
    
      - name: Popular TV Shows
        url: https://www.imdb.com/chart/tvmeter
        transform:
          - css-all: .titleColumn>a
          - text
          - jinja: |
               Popular TV shows this week
               {{ lines.0.lower()|title}} - {{ stash.rating|first }}
               {{ lines.1.lower()|title }} - {{ stash["rating"][0] }}
               {{ lines.2.lower()|title }}  - {{ stash["rating"].0 }}
               {{ lines.3.lower()|title }}
               {{ lines.4.lower()|title }}
               {{ lines.5.lower()|title }}
               {{ lines.6.lower()|title }}
               {{ lines.7.lower()|title }}
               {{ lines.8.lower()|title }}
               {{ lines.9.lower()|title }}
               {{ lines.10.lower()|title }}
        notify:
          - telegram
    I'm only able to display the whole list with {{ stash["rating"] }} but I can't access any individual value
    Paul Massendari
    @paulmassen
    I thought maybe it is stored as text, and not list, so I tried to use {% set foo = stash["rating"].split(',') %} {{ foo[0] }}with no success
    Paul Massendari
    @paulmassen
    ah, got it working by using substring, but it does not feel very clean !
    - name: Popular TV Shows
        url: https://www.imdb.com/chart/tvmeter
        transform:
          - css-all: .titleColumn>a
          - text
          - jinja: |
               📼 Popular TV shows this week 📼
               {{ lines.0.lower()|title}} - ⭐ {{ stash["rating"][2:5] }}
               {{ lines.1.lower()|title }} - ⭐ {{ stash["rating"][9:12] }}
               {{ lines.2.lower()|title }} - ⭐ {{ stash["rating"][16:19] }}
               {{ lines.3.lower()|title }} - ⭐ {{ stash["rating"][23:26] }}
               {{ lines.4.lower()|title }} - ⭐ {{ stash["rating"][30:33] }}
               {{ lines.5.lower()|title }} - ⭐ {{ stash["rating"][37:40] }}
               {{ lines.6.lower()|title }} - ⭐ {{ stash["rating"][44:47] }}
               {{ lines.7.lower()|title }} - ⭐ {{ stash["rating"][51:54] }}
               {{ lines.8.lower()|title }} - ⭐ {{ stash["rating"][58:61] }}
               {{ lines.9.lower()|title }} - ⭐ {{ stash["rating"][65:68] }}
               {{ lines.10.lower()|title }} - ⭐ {{ stash["rating"][72:75] }}
        notify:
          - telegram
    Paul Massendari
    @paulmassen
    I noticed that, on this line: https://github.com/kibitzr/kibitzr/blob/d1005a1cb5df27c566c9ce6913cebc26ddf92dc3/kibitzr/notifier/mailgun.py#L31 replacing text:reportwith html:reportwill send instead an html email
    It could be nice to be able to select this somewhere in the conf, for some data-heavy check, it is nice to send the data nicely formatted
    Peter Demin
    @peterdemin
    Makes sense to me. Let me get back to you next week.
    RYeah Sh
    @pantchox
    hi, its this project active?