Category : puppeteer

Hello I am trying to scrap a web page and return all of the links inside example of the html element: <a href="#/item/2sDSXbG"> <a href="#/item/4ssaSXbG"> <a href="#/item/Sawd432"> Here is my code: let links = []; let elements2 = document.querySelectorAll(‘a’); for (var element2 of elements2) links.push(element2.textContent); After I return the value and print it I get ..

Read more

Puppeteer is very useful and I have been able to scrape many different parts of my site. I know its very easy easy to get by selection and xpath but how would I get by xpath or selection and print out "hey" <div title="hey">spoon</div> here is what I have come up with so far. let ..

Read more

so launching a chrome instance using these flags ; `chrome –remote-debugging-port=${port} –user-data-dir="C:UsersUSERAppDataLocalGoogleChromeUser Data –proxy-server=http://host:port` and the proxy is an private one which have username and password to be able to be used , and i tried connecting a puppeteer instance into that chrome and authenticate using puppeteer but that seems don’t work const browserURL = ..

Read more

So I went on Webshare (https://proxy.webshare.io/proxy/list?) and signed up for an account, which gives you access to 10 http and socks5 proxies for free. async function scrape() { try { const preparePageForTests = async (page) => { const userAgent = ‘Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36’;//’Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, ..

Read more