These are chat archives for rosshinkley/nightmare

19th
Apr 2017
Amrutha Ramesh
@amrutharamesh
Apr 19 2017 02:17
Hello. So I am running Nightmarejs to do web scraping. On some URLs it just hangs. Meaning it does not proceed after emitting 'did-get-response-details'. It doesnt emit any failed to load events and doesnt timeout. I am not able to find any related issues. Thanks for the help!
Federico Marcos
@marcosfede
Apr 19 2017 12:58
hi guys, I'm trying to pass a set of functions that are inside a js module to .evaluate(), I've tried importing the module and passing the function as an argument to evalute, and then calling the function inside it, and also using inject() but I can't access those functions inside my evaluate code
johnferro
@johnferro
Apr 19 2017 13:59
@marcosfede Maybe try requiring it within the preload script? Can find examples of requiring modules in the default preload script here: https://github.com/segmentio/nightmare/blob/master/lib/preload.js
@amrutharamesh It's hard to say without knowing more specifics about your situation (ie an example url it happens on and what nightmare actions you're calling). But in general I'd double check to make sure that you're calling the end action in all scenarios/regardless of branching logic
Federico Marcos
@marcosfede
Apr 19 2017 14:18
inject() ended up working when I tweaked my module pattern a little. thanks!
Tamir Twina
@TamirTwina
Apr 19 2017 14:36
How can i navigate into an internal link ('/bla.html') after the initial .goto('http://someurl.com") .. without .click , just changing the href
Amrutha Ramesh
@amrutharamesh
Apr 19 2017 14:41
@johnferro So an example URL for this is https://www.santanderbank.com/us/personal/banking/digital-banking/online-banking-enrollment. So in general, my aim is to cycle through sets of 100 urls, extract some info from the page and repeat it. I am doing it with array.reduce and i am creating nightmare instance for each run of the loop.
The thing is it doesnt emit any 'did-finish-load' or 'did-fail-load' or any event after 'did-get-response-details'. So its hard for me to check for an event and move on to the next url
Amrutha Ramesh
@amrutharamesh
Apr 19 2017 14:54
When I test it with this url specifically, it opens an electron instance, and it just stays that way without loading anything inside the page.
johnferro
@johnferro
Apr 19 2017 17:43
@amrutharamesh What version of nightmare and electron are you on? I put together an extremely simplified script that runs and exits correctly. It is also displaying a did-finish-load event. For reference I am on nightmare 2.7.0 and electron 1.3.4. I'd recommend seeing if you could run this script for your nightmare and electron version first and then go from there.
process.env.DEBUG = 'nightmare:log*'

var Nightmare = require('nightmare');
var nightmare = Nightmare({ show: false});


nightmare
  .goto('https://www.santanderbank.com/us/personal/banking/digital-banking/online-banking-enrollment')
  .wait(5000)
  .evaluate(function () {
    return document.querySelector('title').innerText;
  })
  .end()
  .then(function (result) {
    console.log(result);
  })
  .catch(function (error) {
    console.error('Error:', error);
  });
@TamirTwina If you know the link in advance you can simple call the goto action again, or retrieve it with an evaluate action and then call goto with it
The following example shows how to retrieve the link (https://github.com/segmentio/nightmare#examples)
Amrutha Ramesh
@amrutharamesh
Apr 19 2017 21:25
@johnferro The electron version I have is 1.4.4 and nightmare is 2.10.0. And the script you have given as example doesnt work for me :/
It does not move past the did-get-response-details stage as usual
johnferro
@johnferro
Apr 19 2017 21:37
@amrutharamesh I just tried with nightmare 2.10.0 but electron still on 1.3.4 and it worked. Unfortunately I can't test the electron version to right now but my guess is it looks like that's the issue. Could be either an electron issue itself or how that version of electron interacts with nightmare. Do you have any flexibility as to what version of electron you use?
Amrutha Ramesh
@amrutharamesh
Apr 19 2017 21:38
Yeah I can change the version that wont be a problem. But this version came with nightmarejs and i did not install it separately.
So is there a way to use a different electron version with nightmare?
johnferro
@johnferro
Apr 19 2017 21:41
yes you can use the electronPath option to point it to another electron
Amrutha Ramesh
@amrutharamesh
Apr 19 2017 21:43
Okay i will this out and let know.
*try
Thank you!
johnferro
@johnferro
Apr 19 2017 21:43
no problem, I hope that works
Amrutha Ramesh
@amrutharamesh
Apr 19 2017 22:00
@johnferro yep that worked!
Why would it differ on 1.4.4 though? It is very confusing.
Amrutha Ramesh
@amrutharamesh
Apr 19 2017 22:53
@johnferro Okay that worked for one site. But for some other websites it still hangs and doesnt do anything. I dont really understand what is happening. And also is it mandatory to include the nightmare preload script?