Instant web data scraper - Scrape any website

The Instant web data scraper actor is your all-in-one solution for effortlessly scraping data from any website of your choice. Say goodbye to manual data extraction, complex coding, and time-consuming setups. With this actor, you can easily collect the data you need, whether it's for research, analysis, or business insights.

Key Features

• Auto detect tables and data: No need to hunt for data manually; the actor does the heavy lifting. Browse through the rows to find a sample row that contains all the data you need, and let the actor scrape data in same format

• Dynamic web page support: The scraper recognizes dynamic data loads, ensuring you capture the most up-to-date information.

• Advanced Options: For added flexibility, you can pass cookies and set proxies if required. This enables you to navigate websites with restrictions or complex authentication processes.

• Customizable Delays: Control and randomise the crawling speed to your needs with minimum and maximum delay options

• Pagination Support: Easily navigate websites with pagination. The actor can automatically move to the next page via buttons or links.

• Infinite Scrolling: Perfect for websites with infinite scrolling, ensuring you don't miss any data.

How to Use

1. Enter the URL of the webpage you want to scrape.
2. Select the 'Find tables' action to identify tabular data.
3. Run the actor.
4. Review the output dataset and find a sample row containing the desired data.
5. Note the 'table number' and 'column numbers' of the data points to be scraped.
6. Define your data extraction preferences in the "Column mappings" section.
7. Enter the noted 'table number' from the previous run.
8. Find the next page button selector from the target web page. To get the selector, just right click on the element and click 'Inspect' option. Then Right click on the highlighted element on the developer tools and click 'Copy Selector' option.
9. Choose the 'Scrape data' action and run the actor again.
10. Optionally, pass cookies and set proxies if needed for advanced scraping scenarios.

Example use cases

1. Scraping wordpress directory

Input

1{
2    "action": "scrapeData",
3    "additionalColumns": false,
4    "maxDelay": 3,
5    "minDelay": 2,
6    "tableNumber": 1,
7    "nextPageSelector": ".next.page-numbers",
8    "columnMappings": [
9        {
10            "key": "name",
11            "value": "2"
12        },
13        {
14            "key": "link",
15            "value": "0"
16        },
17        {
18            "key": "ratings",
19            "value": "4"
20        },
21        {
22            "key": "authorName",
23            "value": "8"
24        },
25        {
26            "key": "usersCount",
27            "value": "9"
28        },
29        {
30            "key": "description",
31            "value": "6"
32        },
33        {
34            "key": "lastUpdated",
35            "value": "11"
36        },
37        {
38            "key": "pluginLogo",
39            "value": "1"
40        }
41    ],
42    "url": "https://wordpress.org/plugins/?plugin_business_model=commercial"
43}

Scraped data

1[
2	{
3		"name": "Elementor Website Builder – More than Just a Page Builder",
4		"link": "https://wordpress.org/plugins/elementor/",
5		"ratings": "6,586 total ratings",
6		"authorName": "Elementor.com",
7		"usersCount": "5+ million active installations",
8		"description": "The Elementor Website Builder has it all: drag and drop page builder, pixel perfect design,…",
9		"lastUpdated": "Updated 4 days ago",
10		"pluginLogo": "https://ps.w.org/elementor/assets/icon.svg?rev=2597493"
11	},
12	{
13		"name": "Yoast SEO",
14		"link": "https://wordpress.org/plugins/wordpress-seo/",
15		"ratings": "27,595 total ratings",
16		"authorName": "Team Yoast",
17		"usersCount": "5+ million active installations",
18		"description": "Improve your WordPress SEO: Write better content and have a fully optimized WordPress site using…",
19		"lastUpdated": "Updated 2 days ago",
20		"pluginLogo": "https://ps.w.org/wordpress-seo/assets/icon.svg?rev=2363699"
21	}
22]

2. Scraping Apify directory

Input

1{
2    "action": "scrapePage",
3    "additionalColumns": false,
4    "maxDelay": 3,
5    "minDelay": 2,
6    "tableNumber": 1,
7    "nextPageSelector": "button.data-tracking-actor-pagination-button-next-store-web",
8    "columnMappings": [
9        {
10            "key": "name",
11            "value": "2"
12        },
13        {
14            "key": "description",
15            "value": "4"
16        },
17        {
18            "key": "userName",
19            "value": "5"
20        },
21        {
22            "key": "usersCount",
23            "value": "8"
24        },
25        {
26            "key": "price",
27            "value": "7"
28        },
29        {
30            "key": "iconUrl",
31            "value": "1"
32        },
33        {
34            "key": "authorPicture",
35            "value": "6"
36        }
37    ],
38    "url": "https://apify.com/store"
39}

Scraped data

1[
2	{
3		"name": "Web Scraper",
4		"description": "Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.",
5		"userName": "Apify",
6		"usersCount": "46.5k",
7		"price": "Free",
8		"iconUrl": "https://images.apifyusercontent.com/J6UV1Nvv0A5RpAIUBaSPN_eBye_lDVpBYyoTg5Jl03I/rs:fill:76:76/aHR0cHM6Ly9hcGlmeS1pbWFnZS11cGxvYWRzLXByb2QuczMuYW1hem9uYXdzLmNvbS9tb0pSTFJjODVBaXRBcnBOTi9abjh2YldUaWthN2FuQ1FNbi1TRC0wMi0wMi5wbmc.webp",
9		"authorPicture": "https://images.apifyusercontent.com/FMPBaWpQ77oz0xJB2epDbiEacNO524JpP0IGc63WUdg/rs:fill:32:32/aHR0cHM6Ly9pbWFnZXMuYXBpZnl1c2VyY29udGVudC5jb20vUmZ6Z3ZXZW1WSjM3VHU3SnJnQjhBRjhmdk9PUVZNWTIwMVd3NnNoeGcxRS9yczpmaWxsOjMyOjMyL2FIUjBjSE02THk5aGNHbG1lUzFwYldGblpTMTFjR3h2WVdSekxYQnliMlF1Y3pNdVlXMWhlbTl1WVhkekxtTnZiUzlhYzJOTmQwWlNOVWczWlVOMFYzUjVhQzlaY1hSclVXMUZlRnB3YlUxa05tUktVUzFoY0dsbWVWOXplVzFpYjJ4ZmQyaHBkR1ZmWW1jdWNHNW4ucG5n.webp"
10	},
11	{
12		"name": "Google Search Results Scraper",
13		"description": "This Google Scraper enables you to scrape Google Search Engine Results Pages (SERPs) and extract organic and paid results, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Select country or language and extraction of custom attributes, and download your data, no coding needed.",
14		"userName": "Apify",
15		"usersCount": "34.2k",
16		"price": "Free",
17		"iconUrl": "https://images.apifyusercontent.com/aDW746gaT2abezCHdyJvcUvyfAcitg0wkspytcV8yf0/rs:fill:76:76/aHR0cHM6Ly9hcGlmeS1pbWFnZS11cGxvYWRzLXByb2QuczMuYW1hem9uYXdzLmNvbS9uRkpuZEZYQTV6akNUdXVkUC85S2pMeFlnRTRyS3ZLYnIzTS1Hb29nbGVfU2VhcmNoX1Jlc3VsdHNfU2NyYXBlci5wbmc.webp",
18		"authorPicture": "https://images.apifyusercontent.com/FMPBaWpQ77oz0xJB2epDbiEacNO524JpP0IGc63WUdg/rs:fill:32:32/aHR0cHM6Ly9pbWFnZXMuYXBpZnl1c2VyY29udGVudC5jb20vUmZ6Z3ZXZW1WSjM3VHU3SnJnQjhBRjhmdk9PUVZNWTIwMVd3NnNoeGcxRS9yczpmaWxsOjMyOjMyL2FIUjBjSE02THk5aGNHbG1lUzFwYldGblpTMTFjR3h2WVdSekxYQnliMlF1Y3pNdVlXMWhlbTl1WVhkekxtTnZiUzlhYzJOTmQwWlNOVWczWlVOMFYzUjVhQzlaY1hSclVXMUZlRnB3YlUxa05tUktVUzFoY0dsbWVWOXplVzFpYjJ4ZmQyaHBkR1ZmWW1jdWNHNW4ucG5n.webp"
19	}
20]