These are chat archives for rosshinkley/nightmare

27th
May 2016
bersam
@Bersam
May 27 2016 13:06
Hi everybody! Is there anyway to make a post request within nightmare environment? I want to make an ajax call.
Ross Hinkley
@rosshinkley
May 27 2016 15:04
@Bersam do you need the session context from the page?
Ross Hinkley
@rosshinkley
May 27 2016 16:11
fixing to cut 2.5.0, unless there's a protest
i'm purposefully omitting moving the electron start to the queue pr, i'd like feedback on that, or at least let it sit for a spell
also, anyone subscribed to the circle builds: i apologize for the possible horror about to ensue
i think i've got stuff set up right, but we're about to find out the hard way :|
Bill Brock
@bbrock25
May 27 2016 16:38
@rosshinkley I'm looking at needing session context from a page, is there an easy way to grab localstorage from nightmare?
Ross Hinkley
@rosshinkley
May 27 2016 16:40
if you need information out of it, i think you can specify where to put the localstorage bits on disk
if you're asking if you can rebuild a request context from that to issue a request with request or got or something, the answer is probably "no"
it really depends on what you're trying to do
Bill Brock
@bbrock25
May 27 2016 16:42
its kindof a hack, log into service, get bearer creds and then start interacting with api
Timothy Obiso
@TimmyO18
May 27 2016 16:43
Hey! I'm trying to host a web parsing bot on heroku that uses nightmare and it keeps timing out on the server but not when I test it locally, are there any changes I could make so that it would run faster?
Bill Brock
@bbrock25
May 27 2016 16:43
i was trying to launch the nightmare service with the options {show:true, 'web-preferences':{ partition: 'persist:foo' }} and it seemed to have killed my login form actions, so i wasn't sure if i was missing something or not
Ross Hinkley
@rosshinkley
May 27 2016 16:44
@bbrock25 i wouldn't think so... hm. Weird.
@TimmyO18 are you running xvfb?
Bill Brock
@bbrock25
May 27 2016 16:44
i saw that this was an issue in an older version...
Ross Hinkley
@rosshinkley
May 27 2016 16:44
oooh, i'd like to know which one
Timothy Obiso
@TimmyO18
May 27 2016 16:44
@rosshinkley I don't think so, how would I do that through heroku
Ross Hinkley
@rosshinkley
May 27 2016 16:45
@TimmyO18 Electron (which nightmare is built on) requires a framebuffer to work
Timothy Obiso
@TimmyO18
May 27 2016 16:46
wouldn't it get installed as a dependency of nightmare?
Ross Hinkley
@rosshinkley
May 27 2016 16:46
no.
framebuffers are internal to the operating system
Timothy Obiso
@TimmyO18
May 27 2016 16:47
so how would i install this on the heroku machine
Ross Hinkley
@rosshinkley
May 27 2016 16:48
segmentio/nightmare#224 is probably worth a read, specifically this comment
Timothy Obiso
@TimmyO18
May 27 2016 16:49
thank you!
Ross Hinkley
@rosshinkley
May 27 2016 16:49
noooo problem :)
@bbrock25 find that issue?
Bill Brock
@bbrock25
May 27 2016 16:49
i was able to grab localstorage, launching with the paths: { userData: '...' } option
Ross Hinkley
@rosshinkley
May 27 2016 16:49
ah, nice
Bill Brock
@bbrock25
May 27 2016 16:50
i think i was just following other bad examples from the internet
Ross Hinkley
@rosshinkley
May 27 2016 16:50
hahahah
Bill Brock
@bbrock25
May 27 2016 16:50
silly internet
Ross Hinkley
@rosshinkley
May 27 2016 16:50
i'm certainly guilty of that
;)
Bill Brock
@bbrock25
May 27 2016 16:51
no, not me
this was c&p from github issues :trollface:
Ross Hinkley
@rosshinkley
May 27 2016 16:51
hahah
fair enough!
Bill Brock
@bbrock25
May 27 2016 16:51
maybe started on SO though, who knows
Ross Hinkley
@rosshinkley
May 27 2016 16:51
and probably half of it was mine :|
i think i can issue a blanket apology
Bill Brock
@bbrock25
May 27 2016 16:51
yeah, definitely your fault
Ross Hinkley
@rosshinkley
May 27 2016 16:52
right?
Bill Brock
@bbrock25
May 27 2016 16:52
hahaha
Ross Hinkley
@rosshinkley
May 27 2016 16:52
;)
offtopic (or... on... topic?), 2.5.0 is out in the universe for use
Bill Brock
@bbrock25
May 27 2016 16:52
no, i'm just new with this package, and not sure the where the line between nightmare and electron lies
i'm on 2.4, i'll upgrade
Ross Hinkley
@rosshinkley
May 27 2016 16:52
probably won't effect you
Bill Brock
@bbrock25
May 27 2016 16:53
yeah, still appears ok.
thanks for the quick feedback!
Ross Hinkley
@rosshinkley
May 27 2016 16:53
here for you :)
lmk if you have questions, i can try to help
Bill Brock
@bbrock25
May 27 2016 16:54
awesome
do you support this product officially for segment?
Ross Hinkley
@rosshinkley
May 27 2016 16:55
i am a maintainer, but i don't work for segment... does that answer your question?
Bill Brock
@bbrock25
May 27 2016 16:55
yep
Ross Hinkley
@rosshinkley
May 27 2016 16:55
:)
Bill Brock
@bbrock25
May 27 2016 16:55
much appreciate the hard work
Ross Hinkley
@rosshinkley
May 27 2016 16:55
no problem
Timothy Obiso
@TimmyO18
May 27 2016 16:59
hey sorry to ask again but now i get the error:
Push rejected, failed to detect set buildpack https://github.com/ddollar/heroku-buildpack-apt.git
Ross Hinkley
@rosshinkley
May 27 2016 17:00
i have to apologize, my heroku ability is inch-deep at best...
i don't think you'd need the node one first, no
it looks like the heroku buildback you specified is deprecated
Timothy Obiso
@TimmyO18
May 27 2016 17:01
yeah i saw that too, but thats what referenced in the electron build pack comment
Ross Hinkley
@rosshinkley
May 27 2016 17:02
yeah, i see that now
hm
do you have an aptfile?
Timothy Obiso
@TimmyO18
May 27 2016 17:06
I'm making it now it think that was the problem ;--;
Ross Hinkley
@rosshinkley
May 27 2016 17:06
give that a scream
lmk if it works :)
Timothy Obiso
@TimmyO18
May 27 2016 17:07
nope,
still getting
Push rejected, failed to detect set buildpack https://github.com/ddollar/heroku-buildpack-apt.git
Amol
@amolc
May 27 2016 17:10

/ jshint -W097 /
'use strict';

const Nightmare = require('nightmare');
const realMouse = require('nightmare-real-mouse');
realMouse(Nightmare);
var nightmare = Nightmare({ show: true });

nightmare
.goto('http://www.housefly.com.sg/#bytop')
.select('#ddlTOP', '2016')
.realClick('#btnSearchByTOP')
.click('button.btnSearchByTOP')
.wait('1000')
.wait('#sample_1')
.click('.hidden-480')
.screenshot('./camp1.png')
.then(() => { console.log('done'); });

/ jshint -W097 /
'use strict';

const Nightmare = require('nightmare');
const realMouse = require('nightmare-real-mouse');
realMouse(Nightmare);
var nightmare = Nightmare({ show: true });


nightmare
  .goto('http://www.housefly.com.sg/#bytop')
  .select('#ddlTOP', '2016')
  .realClick('#btnSearchByTOP')
  .click('button.btnSearchByTOP')
  .wait('1000')
  .wait('#sample_1')
  .click('.hidden-480')
  .screenshot('./camp1.png')
  .then(() => { console.log('done'); });
@rosshinkley the button tag worked.
Now I am just having an issue with the screenshot/save still
if I don't call the click events, its able to take the screen shot
Ross Hinkley
@rosshinkley
May 27 2016 17:13
oh?
Amol
@amolc
May 27 2016 17:13
but when I call the click events, its not able to take the screenshot or grab the html
any clue would highly be appreciated
Ross Hinkley
@rosshinkley
May 27 2016 17:16
what happens?
Amol
@amolc
May 27 2016 17:16
There is no response
Ross Hinkley
@rosshinkley
May 27 2016 17:17
@TimmyO18 give me a couple minutes, i'm going to see if i can prop up a heroku instance with nightamre
@amolc it hangs, or it finishes without saving the screenshot?
Amol
@amolc
May 27 2016 17:18
checking
it doesn't finish.
The "done" is not invoked.
Ross Hinkley
@rosshinkley
May 27 2016 17:20
add a .catch() and see if there's an exception
Amol
@amolc
May 27 2016 17:20
copy.
Ross Hinkley
@rosshinkley
May 27 2016 17:22
i'll take another look at that after this
Amol
@amolc
May 27 2016 17:23
Search failed: Cannot read property 'dispatchEvent' of null --> o.p of catch block
Err: Cannot read property 'dispatchEvent' of null
sorry.. copied your catch block to try, which got that Search failed :smile:
Ross Hinkley
@rosshinkley
May 27 2016 17:24
tells me two things: one, one of your selectors is filing (likely the .click()); and two, you might want to consider updating to 2.5.x
Amol
@amolc
May 27 2016 17:24
copy
Ross Hinkley
@rosshinkley
May 27 2016 17:24
it has better error messages for that exact problem
might help diagnose what's wrong
Amol
@amolc
May 27 2016 17:24
yep on it
Your npm library is good
and I think it should get stable with our debug's
working on it
Ross Hinkley
@rosshinkley
May 27 2016 17:25
thanks, i can't take hardly any credit though
Amol
@amolc
May 27 2016 17:28
Err: Unable to find element by selector: .btnSearchByTOP
better error management
the strange part is
the button which is clicked first gets me the search data
and then the same button is throwing an error.
Ross Hinkley
@rosshinkley
May 27 2016 17:29
does the button have that class after the search?
Amol
@amolc
May 27 2016 17:29
checking
nightmare
  .goto('http://www.housefly.com.sg/#bytop')
  .select('#ddlTOP', '2016')
  .realClick('#btnSearchByTOP')
  .click('button.btnSearchByTOP')
  .wait('1000')
  .wait('#sample_1')
  .screenshot('./camp1.png')
  .then(function (result) {
    console.log(result)
  })
  .catch(function (error) {
    console.error('Err:', error);
  });
Timothy Obiso
@TimmyO18
May 27 2016 17:34
@rosshinkley ok thank you!
Amol
@amolc
May 27 2016 17:34
the button is still having the class, don't understand why it throws the error.
if I don't use button.btnSearchByTOP then the click doesn't happen
Ross Hinkley
@rosshinkley
May 27 2016 17:36
... why are you clicking it twice?
Amol
@amolc
May 27 2016 17:37
Okay, just two minutes
debugging the 2.5 based code
Ross Hinkley
@rosshinkley
May 27 2016 17:37
also, button.btnSearchByTOP !== #btnSearchByTOP
Amol
@amolc
May 27 2016 17:37
initially the first realClick wasn't showing the data
but now yoru click function is working well.
Let me check for 15 minutes
keep you posted
Ross Hinkley
@rosshinkley
May 27 2016 17:38
:) thanks
Timothy Obiso
@TimmyO18
May 27 2016 17:39
hey @rosshinkley did u get it to work using heroku?
Ross Hinkley
@rosshinkley
May 27 2016 17:40
@TimmyO18 i think i have a dyno set up right, testing stuff now
Timothy Obiso
@TimmyO18
May 27 2016 17:41
@rosshinkley nice!
Amol
@amolc
May 27 2016 17:43
it works
:smile:
Ross Hinkley
@rosshinkley
May 27 2016 17:44
hooray!
Amol
@amolc
May 27 2016 17:44
you are a champion
nightmare
  .goto('http://www.housefly.com.sg/#bytop')
  .select('#ddlTOP', '2016')
  .click('#btnSearchByTOP')
  .wait('#sample_1')
  .html('./camp1.html')
  .then(function (result) {
    console.log("done");
  })
  .catch(function (error) {
    console.error('Err:', error);
  });
Amol
@amolc
May 27 2016 17:50
silly question now
Ross Hinkley
@rosshinkley
May 27 2016 17:50
shoot
Amol
@amolc
May 27 2016 17:51
<a href="javascript:ShowUnitFloorPlan('ODh5LzJJR0NXVW9pY3FJZ05MVlkzQT09',10172)">View</a>
now do we call the js
now*=how
Ross Hinkley
@rosshinkley
May 27 2016 17:51
couple of options
you could click the a
Amol
@amolc
May 27 2016 17:51
ok
Ross Hinkley
@rosshinkley
May 27 2016 17:51
or you should also be able to call it from .evaluate()
Rob Brackett
@Mr0grog
May 27 2016 17:52
@amolc did realClick() not work with the same selector as click()?
nightmare-real-mouse is sort of a testing ground for potentially re-writing the mouse events in core nightmare
Amol
@amolc
May 27 2016 17:52
It was a bit of double work. Earlier version , the click wasn't working.
Rob Brackett
@Mr0grog
May 27 2016 17:53
so it would be good to know if something is not working with it that is working with the built-in click
Amol
@amolc
May 27 2016 17:53
You remember, when I tried around monday. That time the realclick was working.
I am actually testing the mouseevent on other project
Ross Hinkley
@rosshinkley
May 27 2016 17:53
@Mr0grog i think there was also confusion on what selector to use
Rob Brackett
@Mr0grog
May 27 2016 17:53
that's what it looked like, but wanted to make sure
Amol
@amolc
May 27 2016 17:54
so that I can give you proper suggestions, I know its a bit of early version so wanted to do more test before I shoot questions to you.
Rob Brackett
@Mr0grog
May 27 2016 17:54
:thumbsup:
Amol
@amolc
May 27 2016 17:54
I am using the real-mouse event to test on facebook sites
and doing some testing.
Ross Hinkley
@rosshinkley
May 27 2016 17:54
@TimmyO18 when were you getting your error?
Amol
@amolc
May 27 2016 17:55
I am getting back to coding and open source packages after a long time.
was managing my own startup.. but now it's exciting to work with you on this package since I have spent hours making spiders etc in my career.
Timothy Obiso
@TimmyO18
May 27 2016 17:56
i get push rejected
failed to detect set buildpack
Ross Hinkley
@rosshinkley
May 27 2016 17:57
when you're pushing to the heroku repo?
Timothy Obiso
@TimmyO18
May 27 2016 17:57
yeah
Ross Hinkley
@rosshinkley
May 27 2016 17:57
... weird, i don't get any rejections
Timothy Obiso
@TimmyO18
May 27 2016 17:57
what does your build pack look like
Ross Hinkley
@rosshinkley
May 27 2016 17:58
1. heroku/nodejs
2. https://github.com/ddollar/heroku-buildpack-apt.git
3. https://github.com/captain401/heroku-buildpack-xvfb.git
4. https://github.com/benschwarz/heroku-electron-buildpack.git
(i started on the node prebuilt, fwiw)
Timothy Obiso
@TimmyO18
May 27 2016 18:00
i get Push rejected, failed to detect set buildpack https://github.com/ddollar/heroku-buildpack-apt.git
Ross Hinkley
@rosshinkley
May 27 2016 18:04
mmm
Rob Brackett
@Mr0grog
May 27 2016 18:05
So that message (“failed to detect set buildpack”) means it couldn't figure out your process type (which should probably be node.js in this case)
it could be because something is wrong with your buildpacks, but might just as well be that something is wrong in the way your repo is organized
Timothy Obiso
@TimmyO18
May 27 2016 18:05
how would I fix that
Ross Hinkley
@rosshinkley
May 27 2016 18:06
if i had to guess... is your package.json at the root of the project?
Rob Brackett
@Mr0grog
May 27 2016 18:06
^ that would be the first thing
Timothy Obiso
@TimmyO18
May 27 2016 18:06
yes
Rob Brackett
@Mr0grog
May 27 2016 18:06
is your repo public somewhere we can look at it?
Timothy Obiso
@TimmyO18
May 27 2016 18:06
i can make it public hold on (and i know theres a big security flaw it in but i just want it to work rn)
@Mr0grog any luck?
Ross Hinkley
@rosshinkley
May 27 2016 18:21
hm, buildwise, nothing jumps out at me, but i'm admittedly out of my depth
hhhuh
oh, turns out, i can't type
:|
Rob Brackett
@Mr0grog
May 27 2016 18:21
I… can't see it on github?
if you don't want to make it public, just add me as a collaborator
Ross Hinkley
@rosshinkley
May 27 2016 18:22
@TimmyO18 did you strike the repo down?
@Mr0grog (it was up, briefly)
Timothy Obiso
@TimmyO18
May 27 2016 18:23
yeah, i accidentally pushed a sensitive file that I shouldn't of and I'm fixing it now
Rob Brackett
@Mr0grog
May 27 2016 18:23
just add me as a temporary collaborator instead of making it public
Ross Hinkley
@rosshinkley
May 27 2016 18:24
i'll be super-curious to know what the issue is... i can't reproduce
Timothy Obiso
@TimmyO18
May 27 2016 18:25
I'm pushing it rn
Rob Brackett
@Mr0grog
May 27 2016 18:29
minor note (this shouldn't be your problem): you probably want nightmare to be a normal dep, not a dev dep
Timothy Obiso
@TimmyO18
May 27 2016 18:30
ok ill try fixing that
Timothy Obiso
@TimmyO18
May 27 2016 18:39
anything wrong @Mr0grog @rosshinkley
i set the build packs that it needed
Rob Brackett
@Mr0grog
May 27 2016 18:42
well, it looks like nightmare crashed for me, but it did push and run on heroku fine
rbrackett ⌘ ~/Dev/_test/FPS (master) $ heroku buildpacks
=== damp-coast-65032 Buildpack URLs
1. https://github.com/ddollar/heroku-buildpack-apt.git
2. https://github.com/captain401/heroku-buildpack-xvfb.git
3. heroku/nodejs
that's what I have
Timothy Obiso
@TimmyO18
May 27 2016 18:42
now i get a "bash: xvfb-run: command not found" error even though i have the Aptfile
Rob Brackett
@Mr0grog
May 27 2016 18:43
And I had to modify your package.json:
  "main": "server.js",
  "dependencies": {
    "express": "^4.13.4",
    "nightmare": "^2.4.1",
    "vo": "^3.1.1"
  },
  "devDependencies": {
  },
hmmmm, I definitely had no problems with that part
(xvfb-run, that is)
ah
you may need the nodejs pack to be last
Ross Hinkley
@rosshinkley
May 27 2016 18:46
sorry to poke my head in again: the behavior i'm getting is very close (if not the same) to @Mr0grog's, and my buildpack is the same as @TimmyO18
with nodejs first
Timothy Obiso
@TimmyO18
May 27 2016 18:47
i have nodejs first too and whenever i try to run getGrades, it times out
Rob Brackett
@Mr0grog
May 27 2016 18:47
yeah, I wouldn't think that'd be a problem—the procfile commands shouldn't run until all the builpacks are loaded
but
@Timmy018 but you can run getGrades?
that doesn't jive with bash: xvfb-run: command not found
there should be no node app running if the above happens
Timothy Obiso
@TimmyO18
May 27 2016 18:48
i can run getGrades but it times out
2016-05-27T18:47:19.758659+00:00 app[web.1]: { [Error: navigation error]
2016-05-27T18:47:19.758693+00:00 app[web.1]:      details: 'Navigation timed out after 30000 ms',
2016-05-27T18:47:19.758692+00:00 app[web.1]:      code: -7,
2016-05-27T18:47:19.758694+00:00 app[web.1]:      url: 'http://ps01.bergen.org/public/' },
2016-05-27T18:47:19.758695+00:00 app[web.1]:   errors: 
2016-05-27T18:47:19.758691+00:00 app[web.1]:    { message: 'navigation error',
2016-05-27T18:47:19.758696+00:00 app[web.1]:        code: -7,
2016-05-27T18:47:19.758694+00:00 app[web.1]:   length: 1,
2016-05-27T18:47:19.758687+00:00 app[web.1]:   '0': 
2016-05-27T18:47:19.758698+00:00 app[web.1]:        url: 'http://ps01.bergen.org/public/' } ] }
2016-05-27T18:47:19.758697+00:00 app[web.1]:        details: 'Navigation timed out after 30000 ms',
2016-05-27T18:47:19.758696+00:00 app[web.1]:    [ { message: 'navigation error',
2016-05-27T18:47:22.743702+00:00 app[web.1]: Fri, 27 May 2016 18:47:22 GMT nightmare:actions .type() timobi into [id=fieldAccount]
Ross Hinkley
@rosshinkley
May 27 2016 18:50
... so it's running... after a build failure after a push?
o_O
Timothy Obiso
@TimmyO18
May 27 2016 18:50
theres no more build failure it works now
Rob Brackett
@Mr0grog
May 27 2016 18:50
yeah, it's timing out because it's crashing (a couple lines above your timeout)
the browser process is crashing, not electron, to be clear
also seeing electron:stderr Xlib: extension "RANDR" missing on display ":99" which may or may not be related
Timothy Obiso
@TimmyO18
May 27 2016 18:52
i also just pushed the code to the github repo
should my Procfile just be xvfb-run node server.js
Ross Hinkley
@rosshinkley
May 27 2016 18:54
(randr missing is fine and legit, xvfb-run doesn't run with it out of the box)
Rob Brackett
@Mr0grog
May 27 2016 18:54
what you've got seems reasonable
Ross Hinkley
@rosshinkley
May 27 2016 18:55
(you're probably not going to be changing your virtual framebuffer resolution or size in flight)
(if you wanted to get rid of it, you could run it with +extension RANDR, fwiw)
Rob Brackett
@Mr0grog
May 27 2016 18:56
I thought I read something that it was not available on APT
Ross Hinkley
@rosshinkley
May 27 2016 18:57
it's not a standalone
i don't think
Rob Brackett
@Mr0grog
May 27 2016 18:58
right, I mean it's not included in any of the packages on apt
Timothy Obiso
@TimmyO18
May 27 2016 18:58
so what could i do to fix it
Ross Hinkley
@rosshinkley
May 27 2016 18:58
... it's not included in x11?
Rob Brackett
@Mr0grog
May 27 2016 18:58
right
anyway, that's not the problem
the crash is something either in the page you're loading or the commands you're doing; not sure
(if you change scrape to something like loading google and getting the title, it works fine)
Timothy Obiso
@TimmyO18
May 27 2016 19:00
so is there really anything i can do?
Ross Hinkley
@rosshinkley
May 27 2016 19:02
sounds like you need to test the nightmare parts independently, maybe?
also, glancing at that again, you're doing a login... you may be picking up a previous login
Timothy Obiso
@TimmyO18
May 27 2016 19:06
oh that makes sense, ill try that thanks
Ross Hinkley
@rosshinkley
May 27 2016 19:08
might be worth using the in-memory partition or a random partition or something
the navigation tests do something similar (and i think that was one of @Mr0grog's changes)
Rob Brackett
@Mr0grog
May 27 2016 19:10
yeah, you could try that
in theory, the whole file system is trashed when you deploy on heroku, so it's going to be fresh anyway
Ross Hinkley
@rosshinkley
May 27 2016 19:11
ahhh gotcha
that's probably not it then :P
(can't hurt, but it probably won't help)
Timothy Obiso
@TimmyO18
May 27 2016 19:13
yeah it didn't help
Rob Brackett
@Mr0grog
May 27 2016 19:21
So merely loading http://ps01.bergen.org/public/ is crashing
possibly something unique to electron on ubuntu or electron on ubuntu on xvfb
Ross Hinkley
@rosshinkley
May 27 2016 19:24
crashing.. how?
Rob Brackett
@Mr0grog
May 27 2016 19:25
no useful additional info
the renderer process is crashing, as noted above
anyway, doesn't crash for me locally, just on heroku
gotta run
Ross Hinkley
@rosshinkley
May 27 2016 19:29
i can confirm it doesn't crash on ubuntu running a regular x instance, ubuntu running xvfb independently or xvfb-run
... weird, i wonder what's different
Ross Hinkley
@rosshinkley
May 27 2016 19:35
hhhuh
Timothy Obiso
@TimmyO18
May 27 2016 19:57
hm
Rob Brackett
@Mr0grog
May 27 2016 20:33
Coaxed some more info out of it:
electron:stderr [36:0527/195953:ERROR:browser_main_loop.cc(217)] Running without the SUID sandbox! See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the sandbox on.
electron:stderr Xlib:  extension "RANDR" missing on display ":99".
electron:stderr [36:0527/195954:WARNING:host_discardable_shared_memory_manager.cc(129)] Less than 64MB of free space in temporary directory for shared memory files: 5
electron:stderr Xlib:  extension "RANDR" missing on display ":99".
electron:stderr [55:0527/195954:ERROR:sandbox_linux.cc(334)] InitializeSandbox() called with multiple threads in process gpu-process
could be the SUID sandbox, but I have a feeling it's the lack of disk space
which is not something you can adjust on Heroku
(boosting to mega amounts of RAM does not help here)
Timothy Obiso
@TimmyO18
May 27 2016 20:35
Rob Brackett
@Mr0grog
May 27 2016 20:35
that's going to give you Nightmare v1
Ross Hinkley
@rosshinkley
May 27 2016 20:35
@Mr0grog is that space allocated for assets on the local disk? could you filter some of the assets out?
Rob Brackett
@Mr0grog
May 27 2016 20:36
which does not use Electron
Timothy Obiso
@TimmyO18
May 27 2016 20:36
yeah i was gonna adjust my code for nightmare v1 since it apparently does work on v1
Rob Brackett
@Mr0grog
May 27 2016 20:36
Yeah, webkit/Phantom runs a bit lighter than Chromium/Electron in my experience, so that may work for you
otherwise I'd say Heroku is probably not a good environment for this
Timothy Obiso
@TimmyO18
May 27 2016 20:37
i can't really find any other solutions besides heroku that are free and would work besides hosting it myself
Rob Brackett
@Mr0grog
May 27 2016 20:38
well, if free is your sticking point, you're going to be fairly constrained—any browser runs pretty high on resources
Phantom/WebKit will definitely be lighter, so you'll get farther with it
but still be pushing similar boundaries
Timothy Obiso
@TimmyO18
May 27 2016 20:38
Oh, I meant for development I would want to stick with the free dyno, but in production I would probably get a paid dyno
Rob Brackett
@Mr0grog
May 27 2016 20:39
it's much more work to manage, but you could set up a an EC2 instance on Amazon
which will give you much more headroom
their lower tiers are free for the first year, I believe
Timothy Obiso
@TimmyO18
May 27 2016 20:40
id rather just work with heroku and nightmare v1 since it has worked for other
s
Rob Brackett
@Mr0grog
May 27 2016 20:40
@rosshinkley that error is from somewhere down in Chromium; I'm not sure there's much we can do about it
Ross Hinkley
@rosshinkley
May 27 2016 20:40
:|
yeah
Rob Brackett
@Mr0grog
May 27 2016 20:41
No idea whether the ultimate problem is that the assets of the page are too much or if it's something else, e.g. using 3D transforms and needing to store more paint buffers
rosshinkley @rosshinkley nods
Rob Brackett
@Mr0grog
May 27 2016 20:41
anyway, sorry we couldn't solve this for you, @TimmyO18
Timothy Obiso
@TimmyO18
May 27 2016 20:42
its ok you tried your best and hopefully the boilerplate will work
Ross Hinkley
@rosshinkley
May 27 2016 20:45
@Mr0grog i have a question for you sir, if you have a moment
i was tinkering with the viewport and screenshot stuff - i'm sure there is a good reason, but before i go on the ol' goose chase, do you happen to know why a viewport >8k (in at least one dimension) will cause the texture manager to freak out?
Ross Hinkley
@rosshinkley
May 27 2016 21:00
hm... seems to be an opengl limitation
Timothy Obiso
@TimmyO18
May 27 2016 21:26
hey before I try to rewrite my code, in nightmare 1.8.2 will the yield syntax work>
?*
Ross Hinkley
@rosshinkley
May 27 2016 21:45
i don't think so
vo/co require a thennable, and i don't think that was introduced until after 1.8.2
Rob Brackett
@Mr0grog
May 27 2016 21:56
@rosshinkley possibly the --disable-gpu-rasterization flag might help
or --disable-gpu-memory-buffer-compositor-resources or --disable-gpu-compositing or just --disable-gpu
Rob Brackett
@Mr0grog
May 27 2016 22:10
anyway texture size limits ¯\_(ツ)_/¯
Ross Hinkley
@rosshinkley
May 27 2016 22:12
hahaha, yeah
i'll take a look at that later
i poked around segmentio/nightmare#616
and thought "this should be easy!"
and hit that issue, thought you'd have some insight
... got fifty kinds of sidetracked
aaaand now's now
Rob Brackett
@Mr0grog
May 27 2016 22:21
heh
Ross Hinkley
@rosshinkley
May 27 2016 22:21
that was all highly important.
:P
Rob Brackett
@Mr0grog
May 27 2016 22:22
yeah, rendering is… complicated
especially now that 3D compositing has become an important thing on the web
rosshinkley @rosshinkley nods
Ross Hinkley
@rosshinkley
May 27 2016 22:22
and not something i really wanted to get into on a friday afternoon with a partially bent brain
Rob Brackett
@Mr0grog
May 27 2016 22:23
yeah
Ross Hinkley
@rosshinkley
May 27 2016 22:23
i should have known better
Rob Brackett
@Mr0grog
May 27 2016 22:23
my very naive suspicion, being only vaguely familiar with the rendering pipeline on WebKit and not at all familiar with where Blink is now is that you might get farther if you can flip to software-only compositing
(i.e. turning off the GPU)
but that is a maybe at best
Ross Hinkley
@rosshinkley
May 27 2016 22:24
you're hedging a bet that the value for the max texture size is different
Rob Brackett
@Mr0grog
May 27 2016 22:25
things are in general, heavily optimized around the "we would never display a texture bigger than the physical screen" case because screens
Ross Hinkley
@rosshinkley
May 27 2016 22:25
it could be smaller for all we know :P
yeah, i understand that
and was my suspicion when it cropped up
"who has a wider-than-8k-screen?"
... okay, besides those bazillion dollar doodads at CES or what have you
Rob Brackett
@Mr0grog
May 27 2016 22:27
yeah
WebKit had tile sizes hard-coded for a while
Also, IIRC, that person was the same person who was having @2x problems
so they may actually be looking at a texture height of 16k
rosshinkley @rosshinkley nods
Ross Hinkley
@rosshinkley
May 27 2016 22:29
now that you mention it, i took a look at the chromium code
i'm pretty sure it is 16k, but i'm also on a high res box
aaaand.... i think i closed that window
rosshinkley @rosshinkley sighs.
Ross Hinkley
@rosshinkley
May 27 2016 22:32
i guess the qusetion still stands, how do you take a screenshot (with the current implementation) bigger than 8 or 16k in one dimension?
"take two and imagemagick 'em together"
;)
Rob Brackett
@Mr0grog
May 27 2016 22:37
:100:
Ross Hinkley
@rosshinkley
May 27 2016 22:38
looking at my history, i think this may have been what i saw?
my brain is quickly going to mush
i think i'm going to take off and go... do... something. Good day.
Rob Brackett
@Mr0grog
May 27 2016 22:41
looks like, based on that, switching to software won't help
if I'm understanding that conditional right, that limit is only being applied when rendering on software
heh
Yeah, slightly further on, if there is a compositor context, it pulls it from the hardware: https://code.google.com/p/chromium/codesearch#chromium/src/cc/resources/resource_provider.cc&sq=package:chromium&type=cs&l=458&rcl=1464366759
Ross Hinkley
@rosshinkley
May 27 2016 22:46
... but hten all bets are off, right?
... i know, i said i was going
thanks for taking a quick look :)
Rob Brackett
@Mr0grog
May 27 2016 22:48
well, I think the answer here is simple and unfortunate: Chromium won't render above an arbitrary size in each dimension
Ross Hinkley
@rosshinkley
May 27 2016 22:48
yyyeahhh
Rob Brackett
@Mr0grog
May 27 2016 22:48
we could screenshot, see if the result is empty, back it off to 16*1024 if so, and try again
there are still no guarantees there
it'll depend on your hardware
rosshinkley @rosshinkley nods
Rob Brackett
@Mr0grog
May 27 2016 22:49
but then the arbitrary hardcoded number might change at any point in the future, too
so not really anything you can do about that except have some heuristics
Ross Hinkley
@rosshinkley
May 27 2016 22:49
would also be partially hardware-dependent
unless we enforce software rendering to try to garner some level of guarantee
Rob Brackett
@Mr0grog
May 27 2016 22:50
enh, I don't think that's a great idea
Ross Hinkley
@rosshinkley
May 27 2016 22:51
i don't either
Rob Brackett
@Mr0grog
May 27 2016 22:51
we could check to make sure the clip rect is bigger than 0 and throw if the resulting image is empty with a message that you might be asking for an image that's too big
this feels like an area that's better to provide advice in case of failure rather than bend over backwards to try and ensure success (and potentially still fail)
Ross Hinkley
@rosshinkley
May 27 2016 22:52
mmm, at least on this box, the bad image size/buffer is nonzero
it looks like it's junk
but nonzero nonetheless
Rob Brackett
@Mr0grog
May 27 2016 22:53
is it valid PNG data?
Ross Hinkley
@rosshinkley
May 27 2016 22:54
certainly appears to be
... mind you, this is an image just outside the limit
1024 x 8193
i think if you start going beyond that, stranger things happen
Rob Brackett
@Mr0grog
May 27 2016 22:56
hahahaha
oh well
Ross Hinkley
@rosshinkley
May 27 2016 22:56
and then, no, the buffer is either empty or not valid
(at least in my limited test cases)
... so long as we're bandying possibly bad ideas, would it be worth listening to electron:stderr and looking for the gl exception of "dimensions out of range"?
Rob Brackett
@Mr0grog
May 27 2016 22:58
do you reliably get that message?
then we need to know if it comes immediately
rosshinkley @rosshinkley scrolls
Ross Hinkley
@rosshinkley
May 27 2016 22:59
looks like it always happens as soon as that page-touchup-to-force-render bit happens
and it looks like the time difference according to debug is ~115ms, give or take ten
Rob Brackett
@Mr0grog
May 27 2016 23:00
anything useful from calling image.isEmpty()? http://electron.atom.io/docs/api/native-image/#imageisempty
Ross Hinkley
@rosshinkley
May 27 2016 23:01
'ang on a minute
not for my screwball 8k-plus-a-mote case
isEmpty() (correctly) returns false
i wonder what the threshold is...
.... stand by
Ross Hinkley
@rosshinkley
May 27 2016 23:12
looks like i am well wrong. tested up to 32k, they are all (blank) pngs of the correct dimensions