NickJS was great but it's over now 🙏 We recommend you use Puppeteer or Playwright instead
google-chrome -enable-logging=stderr -v1 --headless google.com? (you can the —no-sandbox flag too)
Hello, I'm trying to scrape search results page-by-page, but I can't get nickjs to load pages past the first. Here is a code example:
(async () => {
let results: string[]; // @returns
let n; // number of pages to scrape.
for (let i = 0; i < n; i++) {
if (i > 0) {
try {
await tab.untilVisible(opts.next); // "Next Page" Button/link.
await tab.click(opts.next);
await tab.untilVisible(opts.previous); // "Previous Page" Button/link
} catch (e) {
HandleError(e);
}
}
await tab.untilVisible(opts.result);
await tab.inject("./node_modules/jquery/dist/jquery.min.js"); // We're going to use jQuery to scrape
try {
results = await tab.evaluate(opts, scrapeResult);
} catch (e) {
HandleError(e);
}
return results;
}
});Anyone know the solution? Even if I try waiting for a significant period of time it still doesn't work. Thanks for the help. :)
const completeIFrameForm = (arg, cb) => {
var iframe = document.querySelector("#Iframe_selector").contentWindow
const nameInput = iframe.document.querySelector("#name").value = arg.name //Selector from within the iFrame
cb(null, true)
}
const completeForm = await tab.evaluate(completeIFrameForm, {name})
Business Names from this url
Hi, im struggling to open a URL with POST. Im using the chrome driver with Google Chrome 62.0.3202.75, in the doc, it seems the options argument allows request method POST and data. for example:
var options = {
method: 'post',
data: {
test: 1
}
}
await tab.open(..., options)But the requests are not coming in as POST (and the data isnt sent as well)
open method documentation (you can find it here: https://hub.phantombuster.com/reference#nick-open) options argument is ignored when by chrome driver except the CasperJS one. Were you using the CasperJS driver?
Something went wrong: TypeError: tab.driver.client.Network.setRequestInterception is not a function
Hello @AbdesJSsetRequestInterception is marked as experimental (and deprecated) on the Chrome DevTools Protocol, it recommended to use Fetch.enable instead (https://chromedevtools.github.io/devtools-protocol/tot/Fetch#method-enable)
You can try to use the following example as a NickJS evaluate to send a POST request:
const sendXHR = (arg, cb) => {
const doXHR = ({ method, url, headers, data }) => {
return new Promise((resolve, reject) => {
let xhr = new XMLHttpRequest()
xhr.addEventListener("readystatechange", function() {
if (this.readyState === this.DONE) {
resolve(xhr.responseText)
}
})
xhr.onerror = function() {
reject({ status: this.status, statusText: xhr.statusText })
}
xhr.open(method, url)
if (headers) {
Object.keys(headers).forEach(el => xhr.setRequestHeader(el, headers[el]))
}
xhr.send(data)
})
}
cb(null, doXHR(arg))
}
<option value="0">Select Size</option>
<option value="3924119">XS</option>
<option value="3924120">S</option>
<option value="3924121">M</option>
<option value="3924122">L</option>
<option value="3924123">XL</option>
<option value="3924124" selected="selected">XXL</option>
<option value="3924125">XXXL</option></select>
Hello @PayFlic_ltd_twitter, you can use the following NickJS evaluate snippet to select a specific option by it’s value:
const selectOptionByContent = (arg, cb) => {
const optionValue = "XS"
let options = […document.querySelectorAll("select[name=\"size-dropdown\"] option")]
options = options.filter(el => el.textContent).filter(el => el.textContent.trim().toLowerCase() === optionValue.toLowerCase())
if (options.length < 1) {
return cb(`option ${optionValue} not found`)
}
const target = options[0]
target.selected = true
return cb(null, true)
}If you already know the option value tag to target you can simply use in a NickJS evaluate: document.querySelector("select[name=\"size-dropdown\"] option=\"3924119\"").selected = true
hello all, I'm trying to run chrome without headless mode for debug on my MacOS, but i can not run it, the code return this error
CHROME STDERR: [1129/143352.575017:ERROR:chrome_main_delegate.cc(617)] Web security may only be disabled if '--user-data-dir' is also specified.
CHROME STDOUT: Opening in existing browser session.
Fatal: Chrome subprocess exited with code 0and this is my config:
const nick = new Nick({
timeout: 60000,
headless: false,
debug: true,
});can someone help me solve this issues?