Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 19 2020 17:20
    paps commented #73
  • Jun 19 2020 17:07

    paps on master

    0.3.8 (compare)

  • Jun 19 2020 15:21
    paps commented #80
  • Jun 19 2020 14:52
    msaus opened #80
  • Jun 19 2020 14:30
    dependabot[bot] labeled #79
  • Jun 19 2020 14:30
    dependabot[bot] opened #79
  • Jun 19 2020 14:30
    dependabot[bot] labeled #78
  • Jun 19 2020 14:30

    dependabot[bot] on npm_and_yarn

    Bump extend from 3.0.1 to 3.0.2… (compare)

  • Jun 19 2020 14:30
    dependabot[bot] opened #78
  • Jun 19 2020 14:30

    dependabot[bot] on npm_and_yarn

    Bump stringstream from 0.0.5 to… (compare)

  • Jun 19 2020 14:30

    paps on master

    Add deprecation notice in README fix puppeteer link fix deprecation notice in README and 2 more (compare)

  • Jun 19 2020 14:30
    paps closed #77
  • Jun 17 2020 12:13
    stressGC review_requested #77
  • Jun 17 2020 12:13
    stressGC assigned #77
  • Jun 17 2020 12:13
    stressGC opened #77
  • Jun 17 2020 12:09

    stressGC on deprecation

    remove doc broken link (compare)

  • Jun 17 2020 12:06

    stressGC on deprecation

    fix deprecation notice in README (compare)

  • Jun 17 2020 07:51

    stressGC on deprecation

    fix puppeteer link (compare)

  • Jun 17 2020 07:50

    stressGC on deprecation

    Add deprecation notice in README (compare)

  • Apr 03 2020 13:56
    spearmootz opened #76
Agung Darmanto
@adarmanto
Hello, for debugging purpose, is there any option to run nickjs script and launch chrome browser without in headless mode? is it possible? Similar to puppeteer option {headless: false}
Antoine Gunzburger
@SaShimy
@adarmanto for now it isn’t possible but you can connect to the debugger and use it for debugging, when you launch your script just connect to 127.0.0.1:9222 and click on the link
mattjhusted
@mattjhusted
Any way to navigate tabs or step back a page?
Martin Tapia
@paps
@mattjhusted you can try an evaluate() call with window.history.back()
mattjhusted
@mattjhusted
Thanks @paps
Solr007
@Solr007
anyone here?
Hello, is it easy to occurrence Man-machine verification when I use in google crawl
?
Solr007
@Solr007
how can I do this step export CHROME_PATH=/path/to/chrome ?
Agung Darmanto
@adarmanto
Thanks, @SaShimy!
Martin Tapia
@paps
@Solr007 it depends. What OS are you using?
Solr007
@Solr007
@paps windows 7
Martin Tapia
@paps
@Solr007 then go into the windows settings for environment variables like explained here https://www.computerhope.com/issues/ch000549.htm
Solr007
@Solr007
@paps Thanks!
Ronnie
@webstractions
@paps @Solr007 There's a little quirk when setting the env variable in Windows. You need to open a new command window for it to be recognized -- weird that. Found that answer at https://superuser.com/a/832962
Ronnie
@webstractions
Is there any way to access iframes with nickjs?
Antoine Gunzburger
@SaShimy
Hi @webstractions what do you mean by access iframes ?
Ronnie
@webstractions
Well, not just iframes. Frames in general. Puppeteer has a page.frames() method that returns a list of frames. You get access any of the frames from the list and do evaluations on them (like get input values, click buttons, etc.)
Alex Adusei
@alexadusei
Hello there! Is this chat still active?
Sulthon Zainul Habib
@sulthonzh
hi i have tried nickjs but example always give error timeout, Something went wrong: Error: injection of script "http://code.jquery.com/jquery-3.2.1.min.js" timed out after 11036ms
Martin Barreau
@TASnomad
Hi @sulthonzh, which example give you this kind of error ?
mrhodd
@mrhodd
So using NickJs but having trouble managing cookies with multiple instances of NickJs running at the same time on the same box. Is there a way I can define what user chromium is running under to avoid issues here?
Sulthon Zainul Habib
@sulthonzh
hi @TASnomad from github https://github.com/phantombuster/nickjs README
Martin Barreau
@TASnomad
@sulthonzh, could you try to run the example with https instead of http ? If it stills a failure, could try to download the file and inject it from your filesystem path
Martin Tapia
@paps
@mrhodd sorry for the delay. I can understand why that's causing problems. I guess the multiple instances of NickJS are using the same chrome process? How do you do that? I thought it wasn't possible
mrhodd
@mrhodd
@paps The multiple instance didn't work. It looks like managing cookies one instance at a time isn't possible. I had to setup the project so each instance only kicked off after the previous completed.
A more general question, I'm trying to call tab.open() while passing in a header. I'm examining the request in Fiddler but I don't see my header being added to it. Here's the call I'm making:
var options = {headers: {"Accept-Language": "en-US,en;q=0.9"}}
await tab.open("https://www.google.com/", options)
Martin Tapia
@paps
@mrhodd I've seen your issue, thanks :) I will respond in a few minutes
mrhodd
@mrhodd
@paps thanks!!
Bouget Maxime
@bougetmaxime

Hi everyone,
I would like to know from which version of Nickjs started to use ES6?

The following error appear on my console --> "Note: Using ES5 version of NickJS"
So i am looking for the last version of Nickjs with ES5.
Thanks

Martin Tapia
@paps
@bougetmaxime what version of node are you running? Or are you using PhantomJS?
Argenis Leon
@argenisleon
Hi guys, is there any tutorial where I can learn how to write a spider? I am looking to navigate through a list of pages, click the next and continue to the next page?
Martin Tapia
@paps
Hey @argenisleon not a tutorial per se but you can try our Scraping Challenges. It starts easy and ramps up in difficulty, you have the answers available to you if you don't manage to solve a challenge too. Check it out here: http://scraping-challenges.phantombuster.com/
crimsonsoccer55
@crimsonsoccer55
Does tab.inject occur before the tab loads?
James Russo
@jamesrusso
Good Morning!!
What is the best way to wait for certain text on a page?
Seems like not possible with a Selector?
James Russo
@jamesrusso
I wish I could do a :contains selector..
Martin Tapia
@paps
@jamesrusso you can do a tab.evaluate() containing a setTimeout() recursive loop, waiting for something to appear
@crimsonsoccer55 no, to do that you can use await tab.driver.client.Page.addScriptToEvaluateOnNewDocument, see here https://chromedevtools.github.io/devtools-protocol/tot/Page#method-addScriptToEvaluateOnNewDocument
James Russo
@jamesrusso
Thanks @paps.. I was just able to find some other elements..
James Russo
@jamesrusso
Any want to “wait for next page to load”? I’m running into an issue where I’m clicking on something which is causing a location.href change, and then I’m trying to wait for new elements to load in an evaluate, but the problem is that the evaluate runs on the page prior to it navigating away.
Martin Tapia
@paps
@jamesrusso yeah the evaluate() call will be forcefully terminated when the page changes
You have to return from it (with useful data if needed) and make another evaluate() call just after, when the new page has begun loading
DrunkenTaco619
@DrunkenTaco619
can someone explain to me what this does and why it quit working after i ran it once?
Ghost
@ghost~5b1f4bcbd73408ce4f9d0151
The output is next to it. @DrunkenTaco619
DrunkenTaco619
@DrunkenTaco619
is there a way to see the nick() method? i understand it's being used but would like to see the method itself.
Dysproh Sium
@dysproh_gitlab
Headless chrome (running through Nickjs) isn't downloading files (specifically, downloadable .mp3 files). Is there a fix?
jeremiecourlivant
@jeremiecourlivant
Hello. How access to an element in an iframe with Nickjs ?
Ghost
@ghost~5b1f4bcbd73408ce4f9d0151
@jeremiecourlivant I guess you can use evaluate() to get iframe.