Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 05:27
    domenicogioia starred MechanicalSoup/MechanicalSoup
  • Apr 20 14:48
    johnhawkinson commented #360
  • Apr 20 14:46
    johnhawkinson commented #360
  • Apr 20 14:35
    ism commented #360
  • Apr 20 13:21
    johnhawkinson commented #360
  • Apr 20 13:21
    johnhawkinson commented #360
  • Apr 20 10:30
    ism commented #360
  • Apr 19 08:19
    LVerneyPEReN commented #358
  • Apr 18 12:36
    johnhawkinson commented #366
  • Apr 18 11:59
    johnhawkinson edited #366
  • Apr 18 05:07
    johnhawkinson commented #360
  • Apr 18 04:44
    hemberger commented #355
  • Apr 18 04:42
    hemberger closed #354
  • Apr 18 04:40
    hemberger labeled #358
  • Apr 18 04:40
    hemberger commented #358
  • Apr 18 04:32
    hemberger commented #360
  • Apr 18 04:23
    hemberger labeled #366
  • Apr 18 04:22
    hemberger commented #366
  • Apr 18 03:46
    johnhawkinson synchronize #368
Dan Hemberger
@hemberger
@OliverBui MechanicalSoup does not support JavaScript. You will need to use a tool like Selenium, which emulates a fully-fledged browser.
@moy We've always said "does not support JavaScript". But I wonder: could it support JavaScript? Almost certainly not within the scope of the tool, but I wonder how difficult it would be.
phillmac
@phillmac
Calmjs for e.g. makes it substantially easier to implement simple stuff. For e.g. I needed to find a var assignment and grab the value out of a nested object
parsing for a redirect might be tricky esp. If there's two of them
Just find all the redirects and yeild them as an array of addresses?
OliverBui
@OliverBui
@hemberger Thanks, Selenium is too heavy.
Matthieu Moy
@moy
Could MechanicalSoup support JS? I suspect that if we go this way, we would essentially re-implement Selenium.
Dan Hemberger
@hemberger
Aha, okay. Makes sense.
phillmac
@phillmac
The problem with Selenium is it's damned high barrier to entry. If that can be improved by re-implementing, I'm all for it.
OliverBui
@OliverBui
Hello all, how to add new <option></option> to <select></select>?
Matthieu Moy
@moy
I don't think we have anything built-in to add <option>, but you can always do that via BeautifulSoup. Stg like https://www.crummy.com/software/BeautifulSoup/bs4/doc/#insert.
OliverBui
@OliverBui
@moy OK, i will try
Dan Hemberger
@hemberger
@moy Would you like to discuss our next release? I'm happy releasing at any time, and I can do the release if you'd like me to. And if we want to make this the 1.0 release, I'm also fine with that.
Matthieu Moy
@moy
@hemberger: indeed, we should definitely make a new release, we haven't done so for a while. I'm hesitant between two scenarios: in an ideal world, we would finish MechanicalSoup/MechanicalSoup#185 (getter names) before the 1.0, so we could release a non-1.0 now. In this case, I'd make the release as-is, and merge MechanicalSoup/MechanicalSoup#290 (enctype stuff) afterwards, as it's not urgent and we can give it more testing time before including it in a release. But the getter names stuff hasn't really moved for almost 2 years, perhaps it's time to admit that we may not ever do it and release 1.0 without it, in which case I'd merge #290 before to get as much as possible in 1.0.
I'm OK with you doing the release, although I may get a bit of time to do it myself if needed.
Dan Hemberger
@hemberger
@moy Sounds good. I've approved #185, so feel free to merge that in at any time (I'm happy with it in a 0.12 or 1.0 release). I think more discussion about how to implement the getters is warranted if we're going to add them, but to be honest I'm unlikely to proactively work on anything because I'm discouraged that we can't clean up the interface. I think there is a lot of room for improvement for the sake of all future users, but that we're hamstrung by backwards compatibility due to interface decisions that made sense in the past, but don't make sense today.
I'll let you make the decision about the release, but I'm happy to prepare it on your command! :)
Matthieu Moy
@moy
OK, I've merged the PR. Let's release 1.0 :-). It's perfect if you can take care of it. Thanks.
Dan Hemberger
@hemberger
:thumbsup:
Matthieu Moy
@moy
I have boring stuff to procrastinate, I may actually give a try to #185 (no promise) ;-).
Matthieu Moy
@moy
Matthieu Moy
@moy
Dan Hemberger
@hemberger
@moy Do you want me to release 1.0 without #294, release 0.12 now and then a not-too-distant 1.0 release with #294, or wait to release until we have resolved #294? My preference is for the second option.
Matthieu Moy
@moy
I think #294 is "ready" (as in "you can review it and I hope you'll like it", not necessarily as "I don't expect any comment in review"), so we can wait for it to be merged before releasing 1.0. Perhaps this deserves a 1.0-pre1 or so before the actual 1.0 to draw a bit of attention from our users/testers before calling it 1.0, but we can also consider we're not such a huge and active and release 1.0 directly.
Dan Hemberger
@hemberger
Sorry for the delay, this is a very busy couple weeks for me. Thinking about it more, I'd prefer to release 0.12 and then review the PR afterwards. This would give it more soak time and, I think, be preferable to a 1.0-pre1 release. Does that sound okay?
Matthieu Moy
@moy
OK for me.
Dan Hemberger
@hemberger
I've also migrated Travis-CI to the GitHub Apps. It's also set up to run on master weekly if there is no other activity so that if some of our dependencies break our tests, we'll know even if there are no active PR's.
Guardian-Plus Team
@Mohammadwh
hi how i can complete form by mechanicalsoup?
my form not have name
image.png
Matthieu Moy
@moy
you don't need a name. You can use any CSS selector to select the from, and if you have only one form, just a select_form() without argument will select it.
freeway58
@freeway58
How to add proxy support to mechanicalsoup ?
Dan Hemberger
@hemberger
@freeway58 I've added a comment to your GitHub issue. I hope it helps solve your problem. Cheers!
freeway58
@freeway58
Hi @hemberger , thank you very much, your comment way didn't show the proxy ip, still show my real ip, It seem mechanicalsoup didn't support proxy right way, am i right ?
Matthieu Moy
@moy
Hi
Like I did last year, I'll offer my students to contribute to MechanicalSoup as part of a school project, starting end of January. To help them, I need a handful of easy issues (to get started with the code, GitHub, ...), and a few medium-sized issues for the project itself.
Matthieu Moy
@moy
We already have some issues tagged as "easy?", probably enough for this year.
Matthieu Moy
@moy
For medium-sized ones, I'm thinking of #299 (a fun way to dive into continuous integration, dependency management and automated tests) and the #309 I've just added.
#127 could be fun too.
Dan Hemberger
@hemberger
Sounds great! If I can help in any way, organizationally or otherwise, please let me know.
Matthieu Moy
@moy
Any idea what's going on with the last PRs? The first was adding grammar errors, then you closed one adding an irrelevant comment. Now we have one removing one space, another with useless rewording, all by different users.
I don't want to be rude towards beginners, but at this point this starts looking a lot like some kind of spam (maybe people trying to get contributions accepted to get links to their profile where they would later put links to commercial sites?)
Dan Hemberger
@hemberger
Yeah, I think it must be some kind of spam.
I think it's fine if we politely close them, and maybe encourage them to contribute if they want to take a look at the "Help Wanted" tag.
Matthieu Moy
@moy
Hmm, indeed. So users are probably not bots, but just users trying to get a free tee-shirt by getting PRs accepted.
I've added a spam tag to the PRs.
oprypin
@oprypin:matrix.org
[m]
hi.
I have a link that I want to follow:
<a href="foo"><span>SomeText</span></a>
I tried browser.follow_link(text=re.compile("SomeText")) but that doesn't find the link, presumably because the text is nested.
do i have to resort to something as ugly as browser.follow_link(browser.get_current_page().find(lambda el: el.name == "a" and "SomeText" in el.text)) or is there any decent alternative?
1 reply
oprypin
@oprypin:matrix.org
[m]
also so strange, i'm using MechanicalSoup 0.12.0 but getting AttributeError: 'StatefulBrowser' object has no attribute 'page' even though https://github.com/MechanicalSoup/MechanicalSoup/pull/294/files
oh i see MechanicalSoup/MechanicalSoup@a2c0c3c
so it's not released after all
oprypin
@oprypin:matrix.org
[m]
this part could've been useful
https://github.com/MechanicalSoup/MechanicalSoup/blob/9c2a5b7e9/mechanicalsoup/stateful_browser.py#L277
but unfortunately I can't predict what whitespace there will be around the link's text