No credit card required

SuperScraper API

apify/super-scraper-api

No credit card required

Generic REST API for scraping websites: send a URL and get back HTML. This Actor is a drop-in replacement for ScrapingBee, ScrapingAnt, and ScraperAPI services. And it is open-source!

Back to issues Create new issue

How to manage page redirects

Closed

rocksoft opened this issue

ScrapingBee use Response Headers that are useful when the page redirects.

Spb-initial-status-code: The initial status code returned by the scraped page.
Spb-resolved-url: The resolved URL of the scraped page.

Pavlína Vencovská (paja)

Hi,

thanks for reaching out, we'll look into it and let you know what can be done.

Radoslav Chudovský (rado.ch)

Hi, for now, you can get both the initial status code and resolved URL in the response by setting the json_response parameter to true.

Pavlína Vencovská (paja)

Hi,

I'm going to close the issue for now, but if anything else come up, please, let us know.

Add comment

Developer

Apify

Actor Metrics

45 monthly users
32 bookmarks
78% runs succeeded
19 days response time
Created in May 2024
Modified 3 months ago

Categories

Your Super Scraper

mshopik/your-super-scraper

Scrape Your Super and extract data on food and drink from yoursuper.com. Our Your Super API lets you crawl product information and pricing. The saved data can be downloaded as HTML, JSON, CSV, Excel, and XML.

Mark Carter

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

36.8k

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

6.4k

121

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

Apify

76.2k

461

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

5.3k

Download HTML from URLs

mtrunkat/url-list-download-html

This actor takes a list of URLs and downloads HTML of each page.

Marek Trunkát

8.4k

Tiktok Shop Scraper

excavator/tiktok-shop-scraper

This is the Actor for crawling data from the TikTok shop product URLs. For example: https://shop.tiktok.com/view/product/XXXXXXXXXX These URLs are only available for TikTok Shop US. You can test it here: https://apify.com/excavator/tiktok-shop-product

Excavator

Legacy PhantomJS Crawler

apify/legacy-phantomjs-crawler

Replacement for the legacy Apify Crawler product with a backward-compatible interface. The actor uses PhantomJS headless browser to recursively crawl websites and extract data from them using a piece of front-end JavaScript code.

Apify

1.6k

Website Screenshot Generator

apify/screenshot-url

Create a screenshot of a website based on a specified URL. The screenshot is stored as the output in a key-value store. It can be used to monitor web changes regularly after setting up the scheduler.

Apify

3.1k

Extended GPT Scraper

drobnikj/extended-gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.