I am scraping a website where after submitting a form I get redirected to this –
Checking your browser before accessing <Website Name>. This process is automatic. Your browser will redirect to your requested content shortly. Please allow up to 5 seconds… DDoS protection by Cloudflare Ray ID: <Some ID>
Now, usually when I myself submit that form manually from the "real web browser" I get redirected to the main contents almost immediately after this browser checking stuff appears. But in puppeteer, it doesn’t.
I have tried to use
page.waitForNavigation() but I couldn’t get it to work.
Is there any way to actually get past this checking process? Or puppeteer is just getting blocked?
Thanks in advance!