We’ll use scrapeme.live as an example, a fake website prepared for scraping. We use Axios to get the HTML. Then we will pass the HTML to cheerio and query it as we would in a browser environment. We’ll query for the two things we want right now: paginator links and products.
For cases when we want to run JS, Playwright will do. Once everything is working fine, we will scale it by launching crawls async.
Web developer who has been working for startups for +10 years. Engineer turned entrepreneur.
Create your free account to unlock your custom reading experience.