by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Lewys Martin
@CountParadox
good morning all
I am struggling a lot in understanding how selectors and such work for my use scenario
I am trying to monitor a local auction website for new items that match a keyword
but I am either getting everything on the page or nothing at all and I dont know what to do..
Anton Myrberg
@AntonMyr
Hey everyone, I'm planning on creating a new agent for Huginn. Is creating an agent gem from this repository https://github.com/huginn/huginn_agent, the best way to get started?
The Doctor
@virtadpt
I think so, yeah. That's the template agent, isn't it?
I think I know the answer to this question already, so this is more of a confirmation than anything else: Would it be possible to migrate from MySQL to Postgres, and if so how difficult would it be on a scale of "using duct tape" to "performing brain surgery on yourself"?
Anton Myrberg
@AntonMyr
image.png
Another question, I am running my Huginn setup locally without docker and when I press "Create new agent" I get this error message ^
And when I'm running it with docker, I don't get that message
I am totally new to Ruby so it's hard to interpret the error
Anyone knows what this could be?
The Doctor
@virtadpt
That's really weird, I've not seen that one before.
When you set up, did you do a rake assets:precompile (I think)?
GChester
@GChester
Is there a better place to get help. There only seem to be questions here and no answers.
The Doctor
@virtadpt
Usually the bug tracker. Gitter's kind of hit or miss due to folks being spread across multiple timezones.
Anton Myrberg
@AntonMyr
@virtadpt Thanks for the tip, I tried it but it didn't work, unfortunately. I followed the getting started section https://github.com/huginn/huginn#getting-started. My goal is to set it up so I can do development on it, not just to run it as production.
Anton Myrberg
@AntonMyr
I managed to solve this through updating the foreman gem and also the kramdown gem. I don't know if this is recommended but maybe I should open an issue on this with my solution?
Alexandre Nizoux
@deurk
Is anyone using a PostAgent to fill out a website form and get the result back ?
I am running into trouble doing something as simple as this page: http://www.herault.gouv.fr/booking/create/34226
it has one checkbox and a button, plus a Cookie value and that's it
It works using this curl:
curl --location --request POST 'http://www.herault.gouv.fr/booking/create/34226/0' \
--header 'content-type: application/x-www-form-urlencoded' \
--header 'Cookie: eZSESSID=b5l9673clukjuecurfb7mp3757' \
--data-raw 'condition=on&nextButton=Effectuer+une+demande+de+rendez-vous'
but won't configured like this:
{
  "post_url": "http://www.herault.gouv.fr/booking/create/34226/0",
  "expected_receive_period_in_days": "1",
  "method": "post",
  "content_type": "application/x-www-form-urlencoded",
  "payload": "condition=on&nextButton=Effectuer+une+demande+de+rendez-vous",
  "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36",
  "emit_events": "true",
  "no_merge": "true",
  "output_mode": "clean",
  "headers": {
    "Cookie": "eZSESSID=b5l9673clukjuecurfb7mp3757"
  }
}
Anyone able to help ?
Anton Myrberg
@AntonMyr
@deurk is there any errors or it just doesn't work?
Are you sure that the emit_events and no_merge should be true?
Alexandre Nizoux
@deurk
It doesn't work as in it doesn't validate the post action as the same as the curl
yes, emit_events has to be true since I want to parse the returned html, and no_merge has to be true as payload is a string
it feels to me as if the payload somehow goes through some transformation to be sent
is there any way for me to see the actual complete payload sent?
and headers and such?
Anton Myrberg
@AntonMyr
The only way I can think of (I'm new to Huginn tho) is to log what gets sent inside the code
So in the post agent code you can log the payload
I don't know how familiar you are with coding but that's a viable option.
Alexandre Nizoux
@deurk
hmm, I'll try that, and I'll parse the code of the PostAgent to see if something happens to the payload on the way
I'm pretty sure it's very close to working, just a matter of formatting that payload
I tried with payload as an array with condition and nextButton in it, but same result (403 from destination)
Anton Myrberg
@AntonMyr
Hmm so you got 403 from the website you linked earlier?
Alexandre Nizoux
@deurk
yes only with postagent
curl or manual owrks
Anton Myrberg
@AntonMyr
I tried your configuration on Post Agent and I think I got status 200
Alexandre Nizoux
@deurk
oh really? Oo
interesting
Alexandre Nizoux
@deurk
well, I think that my host is on blocked subnet then :(
that would explain why it won't work anyway
Anton Myrberg
@AntonMyr
Hope you figure it out!
The Doctor
@virtadpt
Try proxying through something, like a search engine direct request? Or change the user-agent string for that Website Agent?
malcmail
@malcmail
Hey guys. Not sure if this is the right place to ask what I suspect is an easy question for those in the know. JUst installed the repo with a view to using it to watch for changes on webpages. But having a struggle working out what to put where in the website agent. Managed the URL but looking to monitor the bulk of the page for changes. This is way out of my usual comfort zone so be gentle ...please :)
willwrong
@willwrong
image.png
Hi guys. Having a good time with Huginn lately but struggling with the IMAP agent. It scans alright and sees how many emails are in the folder, but doesn't seem to pick anything up based on the Conditions (per this one, looking for any containing the word "call" in the subject). I've tried with case sensitive/insensitive tags. Any ideas? Thanks in advance!