Pricing

Pay per usage

Try for free

Go to Store

Web Scraper

Try for free

Developed by

Apify

Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

4.5 (23)

Pricing

Pay per usage

917

Total users

90K

Monthly users

Runs succeeded

>99%

Issues response

7.8 days

Last modified

2 months ago

Developer tools

Open source

Back to issues Create new issue

Unsure of how to get what I'm looking for

Closed

dej opened this issue

I'm looking for how I can xtract particluar elements from the webpage. I'd like to extract the email from the following element "yang@ee.ucla.edu" And also the name from this element "C.K. Ken Yang" But I'm unsure what exactly to include to get these values

Jindřich Bär (jindrich.bar)

Hello, and thank you for your interest in this Actor.

Your page function has syntax errors - the Actor cannot parse it correctly, since it's not valid JavaScript. You can use the injected jQuery library to collect data from the web page like this:

async function pageFunction(context) {
    const $ = context.jQuery;

    const results = $('#sp-resulthere .card').map(function() {
        const name = $(this).find('.people-title').text().trim();
        const email = $(this).find('.mailto-link').attr('href').slice("mailto:".length);
        return { name, email };
    }).get();
   
    return results;
}

To scrape the data correctly, you also need to make the Actor wait for networkidle0 event, not networkidle2. This is under Performance and limits > Navigation waits until. networkidle2 fires when there are at most 2 pending requests - on this page, it means that the actual data hasn't loaded yet.

See my fixed run here: https://console.apify.com/view/runs/odPkeMsv0IHDM3yPW

I'll close this issue now, but feel free to ask additional questions if you have any. Cheers!

Add comment

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

4.7

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

8.4K

5.0

Web Scraping API

zeeb0t/web-scraping-api---scrape-any-website

Web Scraping API that quickly and reliably scrapes any website—no selectors required. Premium proxies, CAPTCHA solving, JavaScript rendering, and automated structured data extraction are all included. It’s just $2 per 1,000 web pages scraped, with no minimum spend.

Anthony Ziebell

1.2K

5.0

Playwright Scraper

apify/playwright-scraper

Crawls websites with the headless Chromium, Chrome, or Firefox browser and Playwright library using a provided server-side Node.js code. Supports both recursive crawling and a list of URLs. Supports login to a website.

Apify

3.6

BeautifulSoup Scraper

apify/beautifulsoup-scraper

Crawls websites using raw HTTP requests. It parses the HTML with the BeautifulSoup library and extracts data from the pages using Python code. Supports both recursive crawling and lists of URLs. This Actor is a Python alternative to Cheerio Scraper.

Apify

870

4.2

HTML Scraper pro

scrapingxpert/html-scraper-pro

The HTML Scraper Pro is a powerful tool designed to extract the HTML source code and metadata from websites. It uses advanced web scraping techniques to retrieve the full HTML content of web pages,page title and HTTP status code.This tool is ideal for data extraction, website analysis, and archiving

scrapingxpert

100

RAG Web Browser

apify/rag-web-browser

Web browser for OpenAI Assistants, RAG pipelines, or AI agents, similar to a web browser in ChatGPT. It queries Google Search, scrapes the top N pages, and returns their content as Markdown for further processing by an LLM. It can also scrape individual URLs. Supports Model Context Protocol (MCP).

Apify

4.6K

4.4

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

64K

4.3

Cloudflare Web Scraper

ecomscrape/cloudflare-web-scraper

Advanced web scraper designed to extract data from Cloudflare-protected websites with CAPTCHA bypass, proxy rotation, and JavaScript execution capabilities.

ecomscrape

144

5.0

Instant web data scraper - Scrape any website

curious_coder/instant-web-scraper

Scrape any public and private website data by providing just URL and optionally cookies and proxy information. This scraper is similar to instant data scraper but runs on cloud and can be used as API too!