Asos Products Scraper

Extract fashion products from ASOS.com including clothing, shoes, and accessories with full details like brand, price, sizes, colors, images, and descriptions.

Important: Proxy Requirements

ASOS has aggressive bot detection. This scraper requires RESIDENTIAL proxies to work reliably. Datacenter proxies will be blocked with 403 errors.

To use this scraper successfully:

1. You must have access to Apify's RESIDENTIAL proxy group (paid subscription required)
2. Or configure your own residential proxy service in the proxy settings

Without residential proxies, requests will be blocked and the scraper will fail.

Features

• Complete Product Data - Extracts all product details including sizes, colors, images, and descriptions
• Smart Page Detection - Automatically detects listing pages vs product detail pages
• Pagination Handling - Handles both "Load More" buttons and URL-based pagination
• Advanced Anti-Bot Measures - Browser fingerprint randomization, realistic headers, stealth mode
• JSON-LD Extraction - Uses structured data when available for reliable extraction
• Flexible Input - Scrape category pages, search results, or individual product URLs
• Two Scraping Modes - Quick listing-only mode or full product detail mode

Quick Start

1. Set up proxies - Ensure RESIDENTIAL proxies are configured (required)
2. Add start URLs - Use category pages like https://www.asos.com/men/new-in/cat/?cid=6993
3. Set max items - Choose how many products to scrape
4. Run the scraper - Click Start and monitor the logs

Output Data

The actor extracts the following data fields for each product:

Field	Type	Description
productId	string	ASOS product ID
url	string	Full product page URL
brand	string	Brand name (e.g., ASOS DESIGN, Nike)
title	string	Product title/name
currentPrice	string	Current selling price
originalPrice	string	Original price if on sale
currency	string	Currency code (GBP, USD, etc.)
color	string	Selected/displayed color
sizes	array	Available sizes with availability status
images	array	High-resolution product image URLs
description	string	Product description/about text
category	string	Product category breadcrumb
productCode	string	ASOS product code
careInstructions	string	Care/washing instructions
composition	string	Material composition
scrapedAt	string	ISO timestamp of when data was scraped
dataType	string	"listing" or "product_detail"

Example Output

{
  "productId": "204123456",
  "url": "https://www.asos.com/asos-design/asos-design-oversized-t-shirt/prd/204123456",
  "brand": "ASOS DESIGN",
  "title": "oversized t-shirt in black",
  "currentPrice": "£15.00",
  "originalPrice": "£20.00",
  "currency": "GBP",
  "color": "Black",
  "sizes": [
    { "size": "XS", "available": true },
    { "size": "S", "available": true },
    { "size": "M", "available": false },
    { "size": "L", "available": true }
  ],
  "images": [
    "https://images.asos-media.com/products/...$XXL$",
    "https://images.asos-media.com/products/..."
  ],
  "description": "Crew neck. Short sleeves. Oversized fit.",
  "category": "Men > Clothing > T-Shirts & Vests",
  "productCode": "204123456",
  "careInstructions": "Machine wash according to instructions on care label",
  "composition": "Main: 100% Cotton",
  "scrapedAt": "2024-01-15T10:30:00.000Z",
  "dataType": "product_detail"
}

Usage

Using the Apify Console

1. Go to the actor's page on Apify Store
2. Click "Try for free"
3. Important: Ensure RESIDENTIAL proxy is selected in Proxy Configuration
4. Add your start URLs or use the defaults
5. Click "Start" to run the actor

Using the API

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('YOUR_USERNAME/asos-products-scraper').call({
    startUrls: [
        { url: 'https://www.asos.com/men/new-in/cat/?cid=6993' },
        { url: 'https://www.asos.com/women/sale/cat/?cid=7046' }
    ],
    maxItems: 100,
    scrapeProductDetails: true,
    maxConcurrency: 3,
    proxyConfiguration: {
        useApifyProxy: true,
        apifyProxyGroups: ['RESIDENTIAL']  // Required!
    }
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Scraped ${items.length} products`);

Input Parameters

Parameter	Type	Default	Description
startUrls	array	Men's & Women's New In	URLs to start scraping from (category, search, or product pages)
maxItems	integer	100	Maximum number of products to extract (1-10,000)
scrapeProductDetails	boolean	true	Visit each product page for full details, or extract only listing data
maxConcurrency	integer	3	Number of pages to process in parallel (1-10). Lower = safer
requestTimeout	integer	90	Timeout in seconds for each page (30-180)
proxyConfiguration	object	RESIDENTIAL proxy	Proxy settings - RESIDENTIAL proxies required

Recommended Start URLs

Category	URL
Men's New In	https://www.asos.com/men/new-in/cat/?cid=6993
Women's New In	https://www.asos.com/women/new-in/cat/?cid=15581
Men's Sale	https://www.asos.com/men/sale/cat/?cid=8409
Women's Sale	https://www.asos.com/women/sale/cat/?cid=7046
Men's T-Shirts	https://www.asos.com/men/t-shirts-vests/cat/?cid=7616
Women's Dresses	https://www.asos.com/women/dresses/cat/?cid=8799
Search Results	https://www.asos.com/search/?q=nike+trainers

Note: Always use www.asos.com in URLs (not just asos.com).

Scraping Modes

Full Details Mode (default)

With scrapeProductDetails: true, the scraper:

1. Visits listing/category pages
2. Extracts basic info and product URLs
3. Visits each product page for complete details (sizes, colors, full description, images)

Best for: Complete product data, inventory tracking, detailed analysis

Listing Only Mode

With scrapeProductDetails: false, the scraper:

1. Visits listing/category pages only
2. Extracts basic product info (brand, title, price, image)
3. Much faster but with limited data

Best for: Quick price checks, large-scale product discovery, trend monitoring

Troubleshooting

Common Issues and Solutions

Problem	Cause	Solution
403 Forbidden errors	Bot detection triggered	Ensure RESIDENTIAL proxies are configured
Timeout errors	Page loading slowly	Increase requestTimeout to 120-180 seconds
No products found	Wrong selectors or blocked	Check logs, try different category URL
Rate limited	Too many concurrent requests	Reduce maxConcurrency to 1-2
Empty data fields	ASOS updated their HTML	Report issue on GitHub

Best Practices

1. Always use RESIDENTIAL proxies - This is mandatory for ASOS
2. Start with low concurrency - Use 1-3 concurrent requests to avoid detection
3. Use specific category URLs - Don't start from homepage, use direct category links
4. Monitor your runs - Watch the logs for blocking patterns
5. Run during off-peak hours - Less traffic = less scrutiny
6. Limit batch sizes - Scrape 100-500 items per run rather than thousands

If You're Still Getting Blocked

1. Reduce maxConcurrency to 1
2. Try a different ASOS region (US, UK, EU have different detection)
3. Wait 10-15 minutes between runs
4. Try scraping a different category
5. Contact Apify support about proxy quality

Anti-Bot Measures

This scraper includes multiple anti-detection features:

• Browser Fingerprint Randomization - Randomizes browser characteristics
• User Agent Rotation - Cycles through modern browser user agents
• Viewport Randomization - Varies screen dimensions
• Realistic HTTP Headers - Includes all headers a real browser sends
• Automation Detection Bypass - Overrides navigator.webdriver and other flags
• Random Delays - Human-like pauses between actions
• Cookie Consent Handling - Automatically accepts cookie banners

Integrations

This actor can be integrated with:

• Zapier - Automate workflows with extracted data
• Make (Integromat) - Build complex automations
• Google Sheets - Export data directly to spreadsheets
• Slack - Get notifications when runs complete
• Webhooks - Custom integrations with your systems

Cost Estimation

The actor uses Apify platform resources. Typical costs:

Run Size	Estimated Cost	Notes
100 items	$0.50 - $1.00	RESIDENTIAL proxy adds cost
500 items	$2.00 - $4.00	Full details mode
1,000 items	$4.00 - $8.00	Full details mode
1,000 items	$1.00 - $2.00	Listing only mode

Note: Costs are higher than typical scrapers because RESIDENTIAL proxies are required. Datacenter proxies are much cheaper but will not work with ASOS.

Local Development

To run this scraper locally:

# Clone the repository
git clone https://github.com/YOUR_USERNAME/asos-products-scraper.git
cd asos-products-scraper

# Install dependencies
npm install

# Create input.json with your configuration
echo '{
  "startUrls": [{"url": "https://www.asos.com/men/new-in/cat/?cid=6993"}],
  "maxItems": 10,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}' > storage/key_value_stores/default/INPUT.json

# Set your Apify token (for proxy access)
export APIFY_TOKEN=your_token_here

# Run the scraper
npm start

Legal Notice

This actor extracts publicly available data only. Users are responsible for ensuring their use of extracted data complies with applicable laws and the target website's terms of service. This tool is intended for legitimate purposes such as:

• Price monitoring and comparison
• Market research and analysis
• Competitive intelligence
• Academic research
• Personal shopping assistance

Support

• Issues: Report bugs on GitHub Issues
• Questions: Ask in the Apify Discord community
• Feature Requests: Open a GitHub issue with the "enhancement" label

Changelog

v1.0.1

• Fixed preNavigationHooks syntax error
• Added browser fingerprint randomization
• Added user agent rotation
• Added realistic HTTP headers (Sec-Ch-Ua, Sec-Fetch-*)
• Improved automation detection bypass
• Changed default proxy to RESIDENTIAL (required)
• Reduced default concurrency from 5 to 3
• Increased default timeout from 60 to 90 seconds
• Fixed URL normalization (www.asos.com)

v1.0.0

• Initial release with full ASOS scraping functionality
• Product listing and detail page extraction
• Size and color availability tracking
• JSON-LD structured data extraction
• Anti-blocking measures with random delays
• Pagination handling (Load More + URL-based)
• Two scraping modes (full details vs listing only)