Browser worker vs Code worker
Browser Workers:- Can simulate a user’s interaction with the website via a headless browser
- Handles complex scraping tasks like filling forms, and dynamic content loading.
- Roughly equivalent to doing a curl or python
requests.get(url) - Work by sending HTTP requests to the target website
- Can only work in situations that don’t require interacting with the website UI
Choose the optimal worker type
Choose the appropriate worker type based on the website’s technology and the navigation required to scrape your target data. Best practice: Start with the Code worker type and only switch if you can’t extract the data you need. When to use Browser worker:- You need to click elements to load additional data
- You need to scroll to load more content
- You need to capture network traffic from inside the browser (using
tag_scriptortag_response) - You need to type text to perform searches or trigger dynamic content”
Align your code with worker type
Some functions in our library are only available when using Browser workers and will throw an error if you try to use them with Code workers. Below is a list of functions that can only be used with Browser workers:wait_*(any wait function)scroll_*(any scroll function)tag_*(any tag function)typebrowser_sizeemulate_devicefreeze_pageclickhoverright_clickmouse_topress_keysolve_captchacapture_graphqlclose_popup