These are chat archives for rosshinkley/nightmare

17th
Jun 2016
Aaron Rosenberg
@agrberg
Jun 17 2016 15:22
Hi there. I'm pretty new to node and nightmare but know a decent amount of front end JS. I'm having trouble understanding the example code to get the phantom session's cookies. I think it's because I don't see the context but the yield keyword does not make sense to me there
Aaron Rosenberg
@agrberg
Jun 17 2016 15:29
Specifically I want to log into a site, save the cookie somewhere for later access and continue to perform actions.
basically getting cookies values or stuff is asynchronous so you have to do it inside an async function, if you don't want to use a callback.
Aaron Rosenberg
@agrberg
Jun 17 2016 19:16
Thanks, I'll give that a read over. Is it possible to get these intermediate values while still chaining other actions? I would like to log in, capture a cookie, and then perform actions on the page and return the result back to the node context.
coyotte508
@coyotte508
Jun 17 2016 19:18
well you can do it with evaluate(), since you can get cookies with javascript inside the browser's context
nightmare.doStuff().evaluate( function() { return document.cookie; }).then(function(result) { ... } );
Aaron Rosenberg
@agrberg
Jun 17 2016 19:21
Would it be incorrect to say that you can use each instance of nightmare to perform any number of actions in a headless browser but only return the last value evaluated?
coyotte508
@coyotte508
Jun 17 2016 19:22
you can do whatever in the context of the browser, and what data object you put in the return statement will be gotten back by the node context
Aaron Rosenberg
@agrberg
Jun 17 2016 19:52
After reading that it looks like the cookies example is run within a main generator function. However, none of the examples show this setup, how it gets wrapped, or how/where next() is being called. I would assume the following to work based on the article and the documentation but it does not look like it does:
var Nightmare = require('nightmare');

function* main() {
  var nightmare = Nightmare({show: true});
  nightmare.goto('https://secure.splitwise.com').wait();
  yield nightmare.cookies.get();
}
var iter = main();
console.log(iter.next());
but I just get back the nightmare object and not the cookies
for reference I'm on node 6.2.1 npm 3.9.3 and nightmare 2.5.1
without initializing nightmare in the generator, I this error
/node_modules/nightmare/lib/nightmare.js:178
  if (instance.proc.connected) {
                   ^

TypeError: Cannot read property 'connected' of undefined
    at endInstance (/node_modules/nightmare/lib/nightmare.js:178:20)
    at emitOne (events.js:96:13)
    at process.emit (events.js:188:7)
    at process.exit (internal/process.js:146:15)
    at REPLServer.<anonymous> (node.js:177:27)
    at REPLServer.g (events.js:286:16)
    at emitNone (events.js:86:13)
    at REPLServer.emit (events.js:185:7)
    at onwritten (internal/repl.js:214:14)
    at FSReqWrap.wrapper [as oncomplete] (fs.js:741:5)
coyotte508
@coyotte508
Jun 17 2016 19:56
hmm
what does your wait() do?
Aaron Rosenberg
@agrberg
Jun 17 2016 19:56
oh I suppose I may have left that in from something earlier
1 sec and I'll see what happens when I remove it
coyotte508
@coyotte508
Jun 17 2016 19:56
still
Aaron Rosenberg
@agrberg
Jun 17 2016 19:57
no difference
coyotte508
@coyotte508
Jun 17 2016 19:57
var cookies = yield nightmare.goto('....').wait(500).cookies.get();

console.log(cookies);
Aaron Rosenberg
@agrberg
Jun 17 2016 19:58
does that go in the generator?
coyotte508
@coyotte508
Jun 17 2016 19:58
yea
hm
Aaron Rosenberg
@agrberg
Jun 17 2016 19:59
and nightmare was created outside of it?
coyotte508
@coyotte508
Jun 17 2016 19:59
actually the problem in your code is that you didn't chain the cookies.get() call to the rest
so .goto() didn't finish running
I'll test, because I'm not too sure either
Aaron Rosenberg
@agrberg
Jun 17 2016 20:00
Oh! Actions do not mutate the nightmare object?
So it has to all be one chain in total or one chain per over arching goal you want to accomplish in the browser?
coyotte508
@coyotte508
Jun 17 2016 20:01
no
it doesn't have to be chained
the thing is just that the next function has to be called after the first functoin finishes running
and that's what chaining achieved
oh wait, I'm confusing myself
I'm sorry, I'll just test and see.
Aaron Rosenberg
@agrberg
Jun 17 2016 20:02
okay, thank you for your help
for ref this is what I have right now that is not working as intended
var Nightmare = require('nightmare');

function* main() {
  var nightmare = Nightmare({show: true}),
      cookies = yield nightmare.goto('https://secure.splitwise.com').wait(500).cookies.get();

  console.log(cookies);
}

var iter = main();
iter.next();
Aaron Rosenberg
@agrberg
Jun 17 2016 20:13

More interesting info. I've been working both by running node sample.js and copy pasting that into the node REPL. I get the nightmare object as the iterator's valuein the REPL but only get the following error when running it with node (with or without --harmony):

.../node_modules/nightmare/lib/nightmare.js:178
  if (instance.proc.connected) {
                   ^

TypeError: Cannot read property 'connected' of undefined
    at endInstance (.../node_modules/nightmare/lib/nightmare.js:178:20)
    at emitOne (events.js:96:13)
    at process.emit (events.js:188:7)

the ... is omitting my long project path for convenience

chaining .run(function(){}) onto cookies.get() allows it to run with node however I get no output. Electron opens and shows me the site and then it all just hangs until I kill the process.
copy pasting it into the REPL gives me the exact same thing except I can still type in the REPL
coyotte508
@coyotte508
Jun 17 2016 20:15
yea, .goto() doesn't do anything if not a .then() or sthing after it
Aaron Rosenberg
@agrberg
Jun 17 2016 20:16
and that is the cause of the lack of the connection when running it with the node command?
coyotte508
@coyotte508
Jun 17 2016 20:16
yea
but I added it and got a nightmare object as the result of the cookies.get() call
Aaron Rosenberg
@agrberg
Jun 17 2016 20:17
oh yes, I also get the nightmare object output when using the REPL that I don't get running it through node
coyotte508
@coyotte508
Jun 17 2016 20:17
    it('.set(name, value) & .get(name)', function*() {
      var cookies = nightmare.cookies

      yield cookies.set('hi', 'hello')
      var cookie = yield cookies.get('hi')

      cookie.name.should.equal('hi')
      cookie.value.should.equal('hello')
      cookie.path.should.equal('/')
      cookie.secure.should.equal(false)
    })
there's code like that in the test file though, so it should work
Aaron Rosenberg
@agrberg
Jun 17 2016 20:18
I saw that test! I don't know enough about the testing framework to know how the itmethod works with a generator function
coyotte508
@coyotte508
Jun 17 2016 20:18
This message was deleted
Aaron Rosenberg
@agrberg
Jun 17 2016 20:18
but it does pass locally so it has to be valid somewhere
I'm primarily a Ruby guy so my instinct is to put binding.pry after the first var statement and poke around but it is my understanding that node does not work this way
if you're not familiar with pry-byebug, binding.pry kinda is like what I would expect debugger; to do
coyotte508
@coyotte508
Jun 17 2016 20:20
I see
coyotte508
@coyotte508
Jun 17 2016 20:49
@agrberg
var Nightmare = require('nightmare');
var Q = require("q");

var nightmare = Nightmare({show: true});

Q.spawn(function *() {
  var cookies = yield nightmare.goto('https://secure.splitwise.com').cookies.get({});
  console.log("done", cookies);
});
Aaron Rosenberg
@agrberg
Jun 17 2016 20:55
well that does work but I now have more reading to do :P
coyotte508
@coyotte508
Jun 17 2016 20:58
Without Q:
var Nightmare = require('nightmare');

var nightmare = Nightmare({show: true});

nightmare.goto('https://secure.splitwise.com').cookies.get({}).then(function(cookies){
    console.log(cookies);
});
chaining is still possible by returning another promise object inside the callback (i.e. return nightmare.type("abc").then()) but it's a lot less smoother
Aaron Rosenberg
@agrberg
Jun 17 2016 21:00
So I can continue using the nightmare object to do other things on the page? In my testing it seems to crash / really not like it but as we've seen I don't exactly have all this correct yet.
coyotte508
@coyotte508
Jun 17 2016 21:03
well, as long as you don't .end(), you're supposed to be able to
i run a monitor on a website for days without crashing
but I make sure to call the next goto() after all the previous actions have finished executing
coyotte508
@coyotte508
Jun 17 2016 21:12
This message was deleted
Ross Hinkley
@rosshinkley
Jun 17 2016 21:16
by the by, the crashing problem was fixed, but i haven't merged it in
segmentio/nightmare#684 for what it's worth
also, judging from the above conversation, the readme should be standardized to use .then()
... i feel like that conversation has been had in the past but wasn't ever formalized
it is a result of poor test cases around an instance that hasn't been started yet
and moving the start of the electron process out of the constructor and into the execution chain
makes construction of Nightmare less side-effect-y
when i have more than ten spare minutes, i'll make sure that gets folded in
@Mr0grog re did-finish-load... i'll have to double-check when i get back into it, but memory serving, i don't think it does, otherwise it would still Just Work :tm:
Ross Hinkley
@rosshinkley
Jun 17 2016 21:22
i also need to double check... is there like... a canonical for-serious-load-finished-regardless-of-state event? (i don't think so, but i'll look and see)
otherwise, yeah, we could certainly bake our own
the other thing... pondering out loud... i can't remember if did-fail-load and/or did-fail-provisional-load have isMainFrame offhand... i'll have to check into that too
(i'm almost positive the answer is yes)
coyotte508
@coyotte508
Jun 17 2016 21:26
regarding the readme... A lot of functions say "return xxx" (like .title(), .url()) while actually they return that but with the need to call .then() after to get it, that's what's confusing I think
rosshinkley @rosshinkley nods
Ross Hinkley
@rosshinkley
Jun 17 2016 21:27
there are also (obviously) some that still have yield
probably ought to be cleaned up :)
coyotte508
@coyotte508
Jun 17 2016 21:32
I can start to work on a PR with changes to the readme, but how would you like it to be said? "Returns a promise with XXX" instead of "Returns XXX" (I'm not sure of the proper terminology to use)?
(unless you're planning on doing it)
Ross Hinkley
@rosshinkley
Jun 17 2016 21:34
referring to ... which part?
i can too, but probably won't have time until tomorrow or the day after
coyotte508
@coyotte508
Jun 17 2016 21:35

Referring to this:

.url()

Returns the url of the current page.

Ross Hinkley
@rosshinkley
Jun 17 2016 21:35
... ah, okay. yeah, that's ... kind of misleading
true if you're using q/vo/co
not so much if you're using .then() explicitly
maybe "Resolves the url of the current page."?
that's... hm.
i'd probably just as soon leave those two alone
i'm much more concerned with the misleading examples that use yield
Aaron Rosenberg
@agrberg
Jun 17 2016 21:47
Can you elaborate a bit more on how the yield examples worked and why to move away from them?
Ross Hinkley
@rosshinkley
Jun 17 2016 21:48
sure!
yield is used in tandem with generators, and more to the point, makes thennables very easy to use with vo/co/q
or in the case of the mocha unit tests, mocha-generators
Aaron Rosenberg
@agrberg
Jun 17 2016 21:49
and the it method takes a generator function for the test case?
I need to read up on vo and co as they're unfamiliar to me (q is already on the list)
Ross Hinkley
@rosshinkley
Jun 17 2016 21:50
the move away from using generators + a helper library to reduce the number of dependencies and (hopefully) the learning curve
Aaron Rosenberg
@agrberg
Jun 17 2016 21:50
:thumbsup: no disagreement here
Is thennables is another name for promises that respond to then?
Ross Hinkley
@rosshinkley
Jun 17 2016 21:51
it - when mocha-generators is installed - will take either regular plain ol' functions with a done callback or generators
yes, anything that implements a .then()
Aaron Rosenberg
@agrberg
Jun 17 2016 21:52
so function(doneCallback) { ... doneCallback(); } vs function *() { ... yield() }
Ross Hinkley
@rosshinkley
Jun 17 2016 21:52
you've got it
...hang on
coyotte508
@coyotte508
Jun 17 2016 21:53
sorry, posted midway
Aaron Rosenberg
@agrberg
Jun 17 2016 21:55
looks like the done() example would either be difficult with yield or not possible the same way since it's called in a listener handler
Ross Hinkley
@rosshinkley
Jun 17 2016 21:55
yeah, that was probably not the best example :P
you're totally right
... for what it's worth
coyotte508
@coyotte508
Jun 17 2016 21:56
Q.spawn(function*() {
    var cookies = yield nightmare.cookies.get();
    var url = yield nightmare.url();
    var title = yield nightmare.title();

    //...
Instead of
nightmare.cookies.get().then((cookies) => {
    //do stuff

    return nightmare.url();
}).then((url) => {
    //do stuff

    return nightmare.title();
}).then((title) => {
    //...
});
Ross Hinkley
@rosshinkley
Jun 17 2016 21:56
Aaron Rosenberg
@agrberg
Jun 17 2016 21:58
oh very interesting! I did not know you could continue using nightmare within the then callback, however I do see the benefit of the yield in that example
Ross Hinkley
@rosshinkley
Jun 17 2016 21:58
yield also makes iteration a snap
Aaron Rosenberg
@agrberg
Jun 17 2016 21:59
@rosshinkley bookmarked those for later. Iteration as in .next() on an iterator or iteration as in rapid development?
Ross Hinkley
@rosshinkley
Jun 17 2016 21:59
i should have been more careful with wordchoice, iteration as in loops
Aaron Rosenberg
@agrberg
Jun 17 2016 22:00
haha no worries, I do the same thing with all my overloaded vocab O_o
Ross Hinkley
@rosshinkley
Jun 17 2016 22:00
doing it with promises is a bit of a mess
Aaron Rosenberg
@agrberg
Jun 17 2016 22:01
coyotte508 linked me to an article earlier on how yield can be useful with for loops
Ross Hinkley
@rosshinkley
Jun 17 2016 22:01
:100:
Aaron Rosenberg
@agrberg
Jun 17 2016 22:01
all new stuff but very interesting
coyotte508
@coyotte508
Jun 17 2016 22:01
...Is there a link to those in the main readme? I didn't see it and it would have helped me
Aaron Rosenberg
@agrberg
Jun 17 2016 22:01
I also have a good place to start the next time I pick up
Ross Hinkley
@rosshinkley
Jun 17 2016 22:01
there's a PR for it @coyotte508
:)
another thing to get folded in when i can carve out a minute
Aaron Rosenberg
@agrberg
Jun 17 2016 22:02
not urgent but I also made a small PR the other day adding info about using nightmare with proxies as I had to dig thru issues before I could figure it out
Ross Hinkley
@rosshinkley
Jun 17 2016 22:02
the PR is... segmentio/nightmare#685
i saw
er
69?
wow
edited :P
Aaron Rosenberg
@agrberg
Jun 17 2016 22:04
Awesome, thanks for your and @coyotte508's help
Ross Hinkley
@rosshinkley
Jun 17 2016 22:04
i can type. I promise.
np
coyotte508
@coyotte508
Jun 17 2016 22:05
np... I learned too ._.
coyotte508
@coyotte508
Jun 17 2016 22:22
I think I better not touch the readme. I tried changing the clear() example, it crashes because of what was pointed out earlier, so I tried to put it in a nested call to see if it worked: https://gist.github.com/coyotte508/4b6d1cb99c8c1f84c45ca9092a421106 and the second goto() doesn't do anything.
Ross Hinkley
@rosshinkley
Jun 17 2016 22:57
you've queued it but not executed it with .then()