No credit card required

BeautifulSoup Scraper

apify/beautifulsoup-scraper

No credit card required

Crawls websites using raw HTTP requests. It parses the HTML with the BeautifulSoup library and extracts data from the pages using Python code. Supports both recursive crawling and lists of URLs. This Actor is a Python alternative to Cheerio Scraper.

Do you want to learn more about this Actor?

Get a demo

Back to issues Create new issue

Shopee scraper error

Closed

lojahub opened this issue

I'm trying to use the scraper to extract information about searching for items in the store, but without success. After some research, I noticed that the information on the page only loads after a few seconds. Any tips on how to resolve this?

URL example: https://shopee.com.br/search?keyword=chinelo

Jindřich Bär (jindrich.bar)

Hello and thank you for your interest in this Actor!

Actually, you just figured it out - this page is loading the content dynamically (with asynchronous requests through Javascript), which is something this scraper cannot do - that's also the reason why it's so fast (it doesn't execute client-side Javascript).

If you're looking for an Actor that can execute client-side scripts, check out our Web Scraper (https://console.apify.com/actors/moJRLRc85AitArpNN/). Cheers!

Add comment

Developer

Apify

Actor Metrics

18 monthly users
4 stars
99% runs succeeded
Created in Jul 2023
Modified a month ago

Categories

Developer tools

Linkedin Url Scrape

mikepowers/linkedin-url-scrape

Scrape Unlimited Linkedin Profile URL's

Mike Powers

1.1k

Lamudi.ph Real Estate Listings Scraper

pixelperfekt/lamudi-ph-real-estate-listings-scraper

This Python script uses Selenium and BeautifulSoup to scrape real estate listings from Lamudi.ph . It extracts details like property titles, prices, locations, bedrooms, and land size, storing the data in Apify's dataset for easy analysis.

PixelPerfekt

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

28.5k

719

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

5.7k

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts data from pages using JavaScript code. The Actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

Apify

72.3k

245

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

4.7k

Legacy PhantomJS Crawler

apify/legacy-phantomjs-crawler

Replacement for the legacy Apify Crawler product with a backward-compatible interface. The actor uses PhantomJS headless browser to recursively crawl websites and extract data from them using a piece of front-end JavaScript code.

Apify

1.6k

Extended GPT Scraper

drobnikj/extended-gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

Jakub Drobník

1.2k

Playwright Scraper

apify/playwright-scraper

Crawls websites with the headless Chromium, Chrome, or Firefox browser and Playwright library using a provided server-side Node.js code. Supports both recursive crawling and a list of URLs. Supports login to a website.

Apify

879

Monitoring Reporter Dashboard

apify/monitoring-reporter-dashboard

The monitoring reporter dashboard is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

Apify