kibitzr/Lobby

Personal Network Assistant

hillerliao

@hillerliao

xpath cannot get the cdata content

Peter Demin

@peterdemin

@hillerliao could you please provide more details? This SO answer suggest, that lxml supports CDATA access through XPATH: https://stackoverflow.com/a/14432066/135079

James Nobles

@VacantFuture

Trying to run the docker version with the command from the documentation and getting this error '''

2020-04-01 22:52:25,130 [INFO] kibitzr.conf: No credentials file found at /root/.config/kibitzr/kibitzr-creds.yml"

Peter Demin

@peterdemin

Strange. I am out of keyboard right now. Try running kibitzr init

Please open an issue on Github if the problem persists

James Nobles

@VacantFuture

This is what I got replacing run with init
2020-04-01 23:22:36,135 [INFO] kibitzr.bootstrap: Saving sample check in kibitzr.yml
2020-04-01 23:22:36,136 [INFO] kibitzr.bootstrap: Creating kibitzr-creds.yml

But when i run the docker command with run, I get the same error.

I will open an issue on github

it is looking for the cred file inside the docker or in the $PWD that is is using with -v?

Peter Demin

@peterdemin

It’s meant to use mounted dir

Because both checks and creds belong out of container

matrixbot

@matrixbot

@ptman:kapsi.fi kibitzr was called an alternative to IFTTT

@ptman:kapsi.fi any good examples?

@ptman:kapsi.fi and also, notification support for matrix could be nice, and quite easy, a single HTTP(S) call

@ptman:kapsi.fi (with JSON)

Peter Demin

@peterdemin

Hi @ptman would you consider these as examples? https://kibitzr.readthedocs.io/en/latest/recipes.html It's not really comparing to IFTT, but gives an idea of what's possible.

Botter

@BotterAir_gitlab

hey,
anyone having issues with telegram? i am pretty sure my token and chatid (negative chat id) are correct by using curl
however in kibitzr:

2020-08-09 19:56:30,241 [DEBUG] telegram.vendor.ptb_urllib3.urllib3.connectionpool: https://api.telegram.org:443 "POST /botXXXXX:YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY/sendMessage HTTP/1.1" 401 58
2020-08-09 19:56:30,241 [ERROR] kibitzr.notifier.factory: Exception occurred during sending notification
Traceback (most recent call last):
  File "c:\python38\lib\site-packages\kibitzr\notifier\factory.py", line 69, in notify
    notifier(report=report)
  File "c:\python38\lib\site-packages\kibitzr\notifier\telegram.py", line 44, in post
    self.send_message(m)
  File "c:\python38\lib\site-packages\kibitzr\notifier\telegram.py", line 47, in send_message
    message = self.bot.send_message(
  File "<decorator-gen-2>", line 2, in send_message
  File "c:\python38\lib\site-packages\telegram\bot.py", line 67, in decorator
    result = func(*args, **kwargs)
  File "c:\python38\lib\site-packages\telegram\bot.py", line 346, in send_message
    return self._message(url, data, disable_notification=disable_notification,
  File "c:\python38\lib\site-packages\telegram\bot.py", line 175, in _message
    result = self._request.post(url, data, timeout=timeout)
  File "c:\python38\lib\site-packages\telegram\utils\request.py", line 330, in post
    result = self._request_wrapper('POST', url,
  File "c:\python38\lib\site-packages\telegram\utils\request.py", line 242, in _request_wrapper
    raise Unauthorized(message)
telegram.error.Unauthorized: Unauthorized

Peter Demin

@peterdemin

Hmm. I’ve been using telegram integration for quite some time, and it works fine for me. I’m likely to have outdated library versions though. Can’t tell the numbers, as I’m on vacation atm

Botter

@BotterAir_gitlab

@peterdemin no worries, i ended up nuking everything and with the same kibitzr.yml it works fine. cant complain :D
On to the next issue : how to cut up the result of transform (ersult being a list of urls) into seperate notify calls

lesha-dev

@lesha-dev

Hi. Can someone help me with gmail notifications? I'm trying to notify via smtp: I set up just like in docs (https://kibitzr.readthedocs.io/en/latest/smtp.html), set port to the one from gmail settings page and authorized blocked attempt to login. And still I'm getting the same error: smtplib.SMTPServerDisconnected: Connection unexpectedly closed. How can I fix this?

Peter Demin

@peterdemin

@BotterAir_gitlab for more than one notification, you’ll need to use python or shell notifier and put this logic explicitly. Kibitzr doesn’t support this out of the box.

@lesha-dev I’ll need to check it next week. Could you please provide kibitzr.yml if it’s anyhow different from the docs?

lesha-dev

@lesha-dev

kibitzr.yml:

  - name: Current Time
    url: https://www.worldtimeserver.com/current_time_in_US-NY.aspx
    transform:
      - css: "span#theTime"
      - text
    notify:
      - python: print(content)
      - smtp:
          subject: Kibitzr current time
          recepients:
            - bla-bla-bla@gmail.com
    period: 3

kibitzr-creds.yml:

    host: smtp.gmail.com
    port: 465
    user: bla-bla-bla@gmail.com
    password: something

Botter

@BotterAir_gitlab

@lesha-dev i believe this might be due to google blocking thirdparty. you have to enable "allow unsafe applications" or something like that

did you get a warning in the gmail account about unauthorized application

lesha-dev

@lesha-dev

@BotterAir_gitlab , I got a warning from gmail and after that allowed unsafe applications in the account. Didn't help.

Peter Demin

@peterdemin

@lesha-dev I just checked my kibitzr-creds.yml and noticed, that SMTP port is 587. Could you please check it solves the issue?

Peter Demin

@peterdemin

Also, I fixed typo in s/recepients/recipients/ in the check. I received the email with current time then.

Peter Demin

@peterdemin

I updated the docs as well: https://kibitzr.readthedocs.io/en/latest/smtp.html#smtp

lesha-dev

@lesha-dev

@peterdemin , port 687 worked. It's strange, why 465 isn't working, I copied it from gmail help page with settings for mail clients. Also new error appears - SMTPRecipientsRefused, is it caused by typo in recipients? When will this fix be released?

Peter Demin

@peterdemin

It is released. The typo was in docs

lesha-dev

@lesha-dev

@peterdemin , how to solve following error?

2020-08-19 16:23:13,176 [ERROR] kibitzr.notifier.smtp: Failed to send the e-mail
Traceback (most recent call last):
  File "M:\bin\kibitzr\.venv\lib\site-packages\kibitzr\notifier\smtp.py", line 77, in send_email
    server.sendmail(user, recipients, message.encode("utf-8"))
  File "c:\Python37\lib\smtplib.py", line 881, in sendmail
    raise SMTPRecipientsRefused(senderrs)
smtplib.SMTPRecipientsRefused: {'subject': (553, b'5.1.3 The recipient address <subject> is not a valid RFC-5321 address. s4sm7297552lfc.56 - gsmtp'), 'recepients': (553, b'5.1.3 The recipient address <recepients> is not a valid RFC-5321 address. s4sm7297552lfc.56 - gsmtp')}

Why does it load smtplib.py from the system? Local .venv is activated (M:\bin\kibitzr.venv)

Peter Demin

@peterdemin

Builtin libraries are shared with virtualenvs. You need to change "recepients" to "recipients" in the config.

lcq92

@lcq92

Hi! I wanted to thank you for making this project. (I've always wanted a way to create scrapers/notifiers combos quickly without always re-writings the same python scripts)

I've got a question: is it possible to use the changes transform tool, but only output the new content variable (not the Old & New way using changes: verbose) ?

Peter Demin

@peterdemin

Yes, changes: new

Rafael Perez

@rafaelperez

I've used the kibitzr firefox to log in the websites I want to check. How can I make sure the recipe uses the selenium driver?

Peter Demin

@peterdemin

@rafaelperez Selenium driver kicks in if one of these fields is defined for the check: delay, scenario, form. If you don't need any of them, I'd suggest adding delay: 0.1
https://github.com/kibitzr/kibitzr/blob/ebcbd76e4618fa3adcdbcedf033e36049f64049e/kibitzr/fetcher/loader.py#L40-L44

lcq92

@lcq92

:point_up: August 20, 2020 3:00 PM Thanks! I indeed found where it is in the source code

lcq92

@lcq92

I've also got a question regarding the selenium driver:

If I understood it correctly: using kibitzr run, on second, third etc. executions of jobs that use selenium, the driver is never closed, it uses the previous open one. I see how ideally this should be the preferred way, as far as resource usage is concerned, but when "planning" detailed scenarios using python selenium code, I have to add if-else clauses I didn't except, that check if some buttons have "already" been clicked... Is there an option to specify kibitzr to really close drivers and re-create new ones at each job exectution when using using kibitzr run?

Rafael Perez

@rafaelperez

Thanks, @peterdemin . Using delay: 0.1 solved perfectly my problem!

Peter Demin

@peterdemin

@lcq92 sounds like you want to start from the freshly loaded profile each time

Currently, Kibitzr doesn't have that option in config. There are 2 ways: you can use kibitzr once and schedule the job externally, say using crontab. Or you can contribute the config option to Kibitzr source.

Paul Massendari

@paulmassen

Hi ! Just discovered kibitzr, great tool, bravo 👍I was wondering whether there is somewhere a "cookbook" with more examples of kibitzr.yml files ?

Peter Demin

@peterdemin

Thanks, I think https://kibitzr.readthedocs.io/en/latest/recipes.html is intended for this purpose. The problem is that in my experience kibitzr rules are so customized and personal, that it's hard to find shareable ones.

Where communities thrive

People

Repo info

Activity