Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    lesha-dev
    @lesha-dev
    kibitzr.yml:
      - name: Current Time
        url: https://www.worldtimeserver.com/current_time_in_US-NY.aspx
        transform:
          - css: "span#theTime"
          - text
        notify:
          - python: print(content)
          - smtp:
              subject: Kibitzr current time
              recepients:
                - bla-bla-bla@gmail.com
        period: 3
    kibitzr-creds.yml:
        host: smtp.gmail.com
        port: 465
        user: bla-bla-bla@gmail.com
        password: something
    Botter
    @BotterAir_gitlab
    @lesha-dev i believe this might be due to google blocking thirdparty. you have to enable "allow unsafe applications" or something like that
    did you get a warning in the gmail account about unauthorized application
    ?
    lesha-dev
    @lesha-dev
    @BotterAir_gitlab , I got a warning from gmail and after that allowed unsafe applications in the account. Didn't help.
    Peter Demin
    @peterdemin
    @lesha-dev I just checked my kibitzr-creds.yml and noticed, that SMTP port is 587. Could you please check it solves the issue?
    Peter Demin
    @peterdemin
    Also, I fixed typo in s/recepients/recipients/ in the check. I received the email with current time then.
    Peter Demin
    @peterdemin
    lesha-dev
    @lesha-dev
    @peterdemin , port 687 worked. It's strange, why 465 isn't working, I copied it from gmail help page with settings for mail clients. Also new error appears - SMTPRecipientsRefused, is it caused by typo in recipients? When will this fix be released?
    Peter Demin
    @peterdemin
    It is released. The typo was in docs
    lesha-dev
    @lesha-dev
    @peterdemin , how to solve following error?
    2020-08-19 16:23:13,176 [ERROR] kibitzr.notifier.smtp: Failed to send the e-mail Traceback (most recent call last): File "M:\bin\kibitzr\.venv\lib\site-packages\kibitzr\notifier\smtp.py", line 77, in send_email server.sendmail(user, recipients, message.encode("utf-8")) File "c:\Python37\lib\smtplib.py", line 881, in sendmail raise SMTPRecipientsRefused(senderrs) smtplib.SMTPRecipientsRefused: {'subject': (553, b'5.1.3 The recipient address <subject> is not a valid RFC-5321 address. s4sm7297552lfc.56 - gsmtp'), 'recepients': (553, b'5.1.3 The recipient address <recepients> is not a valid RFC-5321 address. s4sm7297552lfc.56 - gsmtp')}
    Why does it load smtplib.py from the system? Local .venv is activated (M:\bin\kibitzr.venv)
    Peter Demin
    @peterdemin
    Builtin libraries are shared with virtualenvs. You need to change "recepients" to "recipients" in the config.
    lcq92
    @lcq92
    Hi! I wanted to thank you for making this project. (I've always wanted a way to create scrapers/notifiers combos quickly without always re-writings the same python scripts)
    I've got a question: is it possible to use the changes transform tool, but only output the new content variable (not the Old & New way using changes: verbose) ?
    Peter Demin
    @peterdemin
    Yes, changes: new
    Rafael Perez
    @rafaelperez
    I've used the kibitzr firefox to log in the websites I want to check. How can I make sure the recipe uses the selenium driver?
    Peter Demin
    @peterdemin
    @rafaelperez Selenium driver kicks in if one of these fields is defined for the check: delay, scenario, form. If you don't need any of them, I'd suggest adding delay: 0.1
    https://github.com/kibitzr/kibitzr/blob/ebcbd76e4618fa3adcdbcedf033e36049f64049e/kibitzr/fetcher/loader.py#L40-L44
    lcq92
    @lcq92
    :point_up: August 20, 2020 3:00 PM Thanks! I indeed found where it is in the source code
    lcq92
    @lcq92

    I've also got a question regarding the selenium driver:

    If I understood it correctly: using kibitzr run, on second, third etc. executions of jobs that use selenium, the driver is never closed, it uses the previous open one. I see how ideally this should be the preferred way, as far as resource usage is concerned, but when "planning" detailed scenarios using python selenium code, I have to add if-else clauses I didn't except, that check if some buttons have "already" been clicked... Is there an option to specify kibitzr to really close drivers and re-create new ones at each job exectution when using using kibitzr run?

    Rafael Perez
    @rafaelperez
    Thanks, @peterdemin . Using delay: 0.1 solved perfectly my problem!
    Peter Demin
    @peterdemin
    @lcq92 sounds like you want to start from the freshly loaded profile each time
    Currently, Kibitzr doesn't have that option in config. There are 2 ways: you can use kibitzr once and schedule the job externally, say using crontab. Or you can contribute the config option to Kibitzr source.
    Paul Massendari
    @paulmassen
    Hi ! Just discovered kibitzr, great tool, bravo 👍I was wondering whether there is somewhere a "cookbook" with more examples of kibitzr.yml files ?
    Peter Demin
    @peterdemin
    Thanks, I think https://kibitzr.readthedocs.io/en/latest/recipes.html is intended for this purpose. The problem is that in my experience kibitzr rules are so customized and personal, that it's hard to find shareable ones.
    Paul Massendari
    @paulmassen
    Many thanks @peterdemin , these recipes are helpful indeed, although I would be glad to read some other use cases. I would gladly share my file once everything is set up. At the moment I made a check that notify me when someone upload a video of my favorite artist on youtube, and am currently working on a check that would send me the status of the air quality in my city (Paris) via telegram with some nice icons etc.
    Peter Demin
    @peterdemin
    Some more involved examples are linked at https://kibitzr.github.io/ in Explore section. I like your use-case, would love to see how you've done it!
    Paul Massendari
    @paulmassen
    My youtube checker is very simple:
    checks:
      - name: Bob Wilber
        url: https://www.youtube.com/results?search_query=bob+wilber&sp=CAI%253D
        delay: 5
        transform:
          - css: "#video-title.ytd-video-renderer"
          - text
          - changes
        notify:
          - telegram
        period: 12 hours
    Now I'm a bit stuck with string concatenation and Telegram notification
    Paul Massendari
    @paulmassen
    In this example: https://gist.github.com/peterdemin/49610071aed04ddc5bf40a2b6d2c4d9c
    Where does the lines variable comes from?
    ah, I reread the docs and now I understand
    Paul Massendari
    @paulmassen
    Got it working, here is my yml, it was very simple:
      - name: Air Quality Today in Paris
        url: https://www.airparif.asso.fr/accueil-airparif
        delay: 3
        transform:
          - css-all: ".indice-color.text-light"
          - text
          - jinja: |
              🇫 🇷  Pollution in Paris 🇫 🇷
              📅 **Today**📅
              Ozone:  {{ lines.0.lower() }}
              Dioxyde d'Azote: {{ lines.1.lower() }}
              Particules PM10:{{ lines.2.lower() }}
              Particules PM2: {{ lines.3.lower() }}
              📅**Tomorrow**📅
              Ozone:  {{ lines.4.lower() }}
              Dioxyde d'Azote: {{ lines.5.lower() }}
              Particules PM10:{{ lines.6.lower() }}
              Particules PM2: {{ lines.7.lower() }}
        notify:
          - telegram
    Peter Demin
    @peterdemin
    Nice! Thanks for sharing! Would you mind adding it to the docs in a PR?
    Paul Massendari
    @paulmassen
    Sure, will do
    Paul Massendari
    @paulmassen
    And I have tons of other ideas for the doc: I'm thinking of a check that will send me the most popular tv show this week every week with their rating, a check to see if an artist has announced new concert dates etc. I will PR this too if you want.
    Peter Demin
    @peterdemin
    Sounds cool!
    Paul Massendari
    @paulmassen
    Hi @peterdemin , do you have any idea why I'm not able to display the value from Stash ?
    checks:
      - name: Fetch Rating TV Shows
        url: https://www.imdb.com/chart/tvmeter
        transform:
          - css-all: .imdbRating>strong
          - text
        notify:
          - stash:
              rating: '{{ lines }}'
    
      - name: Popular TV Shows
        url: https://www.imdb.com/chart/tvmeter
        transform:
          - css-all: .titleColumn>a
          - text
          - jinja: |
               Popular TV shows this week
               {{ lines.0.lower()|title}} - {{ stash.rating|first }}
               {{ lines.1.lower()|title }} - {{ stash["rating"][0] }}
               {{ lines.2.lower()|title }}  - {{ stash["rating"].0 }}
               {{ lines.3.lower()|title }}
               {{ lines.4.lower()|title }}
               {{ lines.5.lower()|title }}
               {{ lines.6.lower()|title }}
               {{ lines.7.lower()|title }}
               {{ lines.8.lower()|title }}
               {{ lines.9.lower()|title }}
               {{ lines.10.lower()|title }}
        notify:
          - telegram
    I'm only able to display the whole list with {{ stash["rating"] }} but I can't access any individual value
    Paul Massendari
    @paulmassen
    I thought maybe it is stored as text, and not list, so I tried to use {% set foo = stash["rating"].split(',') %} {{ foo[0] }}with no success
    Paul Massendari
    @paulmassen
    ah, got it working by using substring, but it does not feel very clean !
    - name: Popular TV Shows
        url: https://www.imdb.com/chart/tvmeter
        transform:
          - css-all: .titleColumn>a
          - text
          - jinja: |
               📼 Popular TV shows this week 📼
               {{ lines.0.lower()|title}} - ⭐ {{ stash["rating"][2:5] }}
               {{ lines.1.lower()|title }} - ⭐ {{ stash["rating"][9:12] }}
               {{ lines.2.lower()|title }} - ⭐ {{ stash["rating"][16:19] }}
               {{ lines.3.lower()|title }} - ⭐ {{ stash["rating"][23:26] }}
               {{ lines.4.lower()|title }} - ⭐ {{ stash["rating"][30:33] }}
               {{ lines.5.lower()|title }} - ⭐ {{ stash["rating"][37:40] }}
               {{ lines.6.lower()|title }} - ⭐ {{ stash["rating"][44:47] }}
               {{ lines.7.lower()|title }} - ⭐ {{ stash["rating"][51:54] }}
               {{ lines.8.lower()|title }} - ⭐ {{ stash["rating"][58:61] }}
               {{ lines.9.lower()|title }} - ⭐ {{ stash["rating"][65:68] }}
               {{ lines.10.lower()|title }} - ⭐ {{ stash["rating"][72:75] }}
        notify:
          - telegram
    Paul Massendari
    @paulmassen
    I noticed that, on this line: https://github.com/kibitzr/kibitzr/blob/d1005a1cb5df27c566c9ce6913cebc26ddf92dc3/kibitzr/notifier/mailgun.py#L31 replacing text:reportwith html:reportwill send instead an html email
    It could be nice to be able to select this somewhere in the conf, for some data-heavy check, it is nice to send the data nicely formatted
    Peter Demin
    @peterdemin
    Makes sense to me. Let me get back to you next week.
    RYeah Sh
    @pantchox
    hi, its this project active?
    Peter Demin
    @peterdemin
    Hi @pantchox , yes, it is. But I'm somewhat slow to answer these days.
    @paulmassen, regarding HTML reports for mailgun. I don't have a mailgun account configured atm. Would you mind contributing this option to the mailgun notifier conf and verifying it works as expected?
    Paul Massendari
    @paulmassen
    @peterdemin Sure, let me try and I will propose a PR
    Peter Demin
    @peterdemin
    Thanks! :thumbsup:
    RYeah Sh
    @pantchox
    @peterdemin I am working on a web app that one of its features are real time push notifications for desktop and mobile. do you think there is enough demend for Kibitzr users for that feature?