Contact

If you encounter any issues or need to exchange information, please feel free to contact us through the following link: My profile

Tesco.com Scraper: Extract Product Data & Pricing Intelligence

Introduction

Tesco operates as one of the UK's leading supermarket chains, offering an extensive online grocery shopping platform that serves millions of customers. With up to 50,000 products available online, including all major brands, Tesco's e-commerce platform represents a goldmine of valuable market data for businesses, researchers, and analysts seeking comprehensive grocery market insights.

The Tesco.com Product Search Scraper addresses the critical need for automated data extraction from this vast product catalog. Whether you're conducting competitive pricing analysis, monitoring product availability, tracking promotional campaigns, or building comprehensive market research datasets, manual data collection from Tesco's extensive inventory would be virtually impossible and highly time-consuming. This scraper provides an efficient solution for extracting structured product data at scale, enabling businesses to make data-driven decisions in the competitive UK grocery market.

Scraper Overview and Capabilities

The Tesco.com Product Search Scraper is a sophisticated data extraction tool designed specifically for navigating Tesco's complex e-commerce architecture. This scraper can extract comprehensive product details like UPC/GTIN, price, competitor price, category, availability, promotions, nutrition, and images from Tesco UK at scale. The tool supports automated pagination, allowing users to scrape entire product categories without manual intervention.

Key strengths of this scraper include its robust proxy integration, intelligent retry mechanisms, and comprehensive data field extraction. The tool is engineered to handle Tesco's anti-bot measures effectively while maintaining high success rates and data accuracy. It's particularly valuable for e-commerce businesses, market researchers, price comparison services, inventory management systems, and competitive intelligence teams who need reliable access to Tesco's product data.

The scraper is ideal for retail analysts tracking market trends, dropshipping businesses sourcing products, academic researchers studying consumer markets, and businesses developing price optimization strategies. Its scalable architecture ensures consistent performance whether extracting data from a single product category or conducting comprehensive market scans across multiple departments.

Input and Output Details

Example url 1: https://www.tesco.com/groceries/en-GB/shop/frozen-food/vegetables/all?sortBy=relevance&page=2

Example url 2: https://www.tesco.com/groceries/en-GB/shop/clothing-and-accessories/all

Example url 3: https://www.tesco.com/groceries/en-GB/shop/bakery/all

Example Screenshot of product information page:

Input Format

The scraper accepts configuration through a JSON object with several key parameters:

Scrape with URLs:

{
  "max_retries_per_url": 2, // Maximum number of retry attempts for each URL
  "proxy": { // Proxy configuration to avoid bot detection
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL" 
    ],
    "apifyProxyCountry": "GB" // Choose a country that matches your target data location
  },
  "max_items_per_url": 20, // Total number of items you want to scrape
  "urls": [ // Product list page URLs to scrape
    "https://www.tesco.com/groceries/en-GB/shop/frozen-food/vegetables/all?sortBy=relevance&page=2",
    "https://www.tesco.com/groceries/en-GB/shop/clothing-and-accessories/all",
    "https://www.tesco.com/groceries/en-GB/shop/bakery/all"
  ],
  "ignore_url_failures": true // Continue scraping even if some URLs fail
}

The urls parameter: List of product list page URLs that you want to scrape. You can add URLs one by one, or use the Bulk edit section to add a prepared list.

The ignore_url_failures parameter: If set to true, the scraper will continue running even if some URLs fail to be scraped after reaching the maximum number of retries. This ensures that one problematic URL doesn't stop your entire scraping job.

When you provide a list of URLs for scraping, all options in the "Scrape with search filters" section will be disabled. The system will only collect data from the URLs you specified.

Scrape with Search Filters:

{
  "max_retries_per_url": 2, // Maximum number of retry attempts for each search filter
  "proxy": { // Proxy configuration to avoid bot detection
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL" 
    ],
    "apifyProxyCountry": "GB" // Choose a country that matches your target data location
  },
  "max_items_per_url": 20, // Total number of items you want to scrape
  "keyword": "bread", // Search keyword to find products
  "sort_by": "price-ascending", // Sort products by specific criteria
  "page": 1 // Starting page number for search results
}

The keyword parameter: The search keyword to find products (e.g., "bread", "milk", "vegetables", "chicken").

The sort_by parameter: Sort products by various criteria:

• "relevance" - Relevance (most relevant first)
• "price-ascending" - Price: Low to High
• "price-descending" - Price: High to Low

The page parameter: Starting page number for scraping, useful for continuing interrupted scrapes or targeting specific result ranges.

When using search filters for scraping, you need to leave the urls field empty (or set it to null) in the "Scrape with URLs" configuration.

General Options:

The max_items_per_url parameter: Limits the number of products extracted from each product list page or search results page. The default value is 20, providing a manageable batch size while allowing for comprehensive data collection.

The max_retries_per_url parameter: Sets the maximum number of retry attempts for each URL or search filters if the scrape is detected as a bot or the page fails to load. The default value is 2, providing a good balance between thoroughness and efficiency.

The proxy parameter: Proxy configuration is essential for maintaining anonymity and avoiding detection. The residential proxy option ensures that your scraping activities appear as legitimate browsing, reducing the risk of being blocked or rate-limited. You should choose a country that matches the location of the website you're scraping (e.g., UK/GB for tesco.com).

Output Format

You get the output from the Tesco.com Product Search Scraper stored in a tab. The following is an example of the Information Fields collected after running the Actor.

[ // List of product information
  {
  "id": "317204044",
  "is_for_sale": true,
  "status": "AvailableForSale",
  "seller": null,
  "price": {
    "__typename": "PriceType",
    "actual": 3.25,
    "unit_price": 6.5,
    "unit_of_measure": "kg"
  },
  "promotions": [],
  "fulfilment": null,
  "context": null,
  "tpnb": "93550382",
  "tpnc": "317204044",
  "gtin": "05000227001717",
  "ad_id": "display_3S9ztzcW7HevtbeJ1vFLNU6z1z8KCgoIOTM1NTAzODISABoMCLbC4MUGEKTU3c8DIgIIAQ==",
  "base_product_id": "93550382",
  "title": "Aunt Bessie’s Green Vegetable Mix 500g ",
  "brand_name": "AUNT BESSIE'S",
  "short_description": null,
  "default_image_url": "https://digitalcontent.api.tesco.com/v2/media/ghs/b3294021-c10a-44c2-895f-1ed02cadab59/acda7e3a-4e64-4f90-90b9-5e711ef20066.jpeg?h=225&w=225",
  "super_department_id": "b;RnJvemVuJTIwRm9vZA==",
  "media": {
    "__typename": "ProductMediaType",
    "default_image": {
      "__typename": "ProductMediaDefaultImageType",
      "aspect_ratio": null
    }
  },
  "quantity_in_basket": null,
  "super_department_name": "Frozen Food",
  "department_id": "b;RnJvemVuJTIwRm9vZCU3Q1ZlZ2V0YWJsZXM=",
  "department_name": "Vegetables",
  "aisle_id": "b;RnJvemVuJTIwRm9vZCU3Q1ZlZ2V0YWJsZXMlN0NNaXhlZCUyMFZlZ2V0YWJsZXM=",
  "aisle_name": "Mixed Vegetables",
  "shelf_id": "b;RnJvemVuJTIwRm9vZCU3Q1ZlZ2V0YWJsZXMlN0NNaXhlZCUyMFZlZ2V0YWJsZXMlN0NGcm96ZW4lMjBWZWdldGFibGVzJTIwLSUyME1peGVkJTIwVmVnZXRhYmxlcw==",
  "shelf_name": "Frozen Vegetables - Mixed Vegetables",
  "display_type": "Quantity",
  "product_type": "SingleProduct",
  "average_weight": 0,
  "bulk_buy_limit": 25,
  "group_bulk_buy_limit": 0,
  "bulk_buy_limit_message": "You have reached the maximum quantity for this product",
  "bulk_buy_limit_group_id": "G00002499",
  "time_restricted_delivery": null,
  "restricted_delivery": null,
  "is_in_favourites": null,
  "is_new": false,
  "is_restricted_order_amendment": null,
  "max_weight": 0,
  "min_weight": 0,
  "increment": 0,
  "details": {
    "__typename": "ProductDetailsType",
    "components": [
      {
        "__typename": "AdditionalInfo",
        "is_low_everyday_pricing": false
      }
    ]
  },
  "catch_weight_list": null,
  "restrictions": [],
  "reviews": {
    "__typename": "ReviewsType",
    "stats": {
      "__typename": "ReviewStatsType",
      "no_of_reviews": 1,
      "overall_rating": 3,
      "overall_rating_range": 5
    }
  },
  "model_metadata": null,
  "from_url": "https://www.tesco.com/groceries/en-GB/shop/frozen-food/vegetables/all"
}, // ... Many other product details
]

The scraper returns comprehensive product data in multiple formats (JSON, CSV, Excel), with each product containing detailed information across these key fields:

Product Identification:

• ID: Unique product identifier within Tesco's system
• TPNB: Tesco Product Number Base - primary product identifier
• TPNC: Tesco Product Number Category - category-specific identifier
• GTIN: Global Trade Item Number for universal product identification
• Base Product ID: Core product identifier for variations

Commercial Information:

• For Sale: Boolean indicating product availability for purchase
• Status: Current product status (active, discontinued, out of stock)
• Seller: Product seller information (Tesco or third-party marketplace)
• Price: Current selling price with currency information
• Promotions: Active promotional offers and discount details
• Fulfilment: Delivery and collection options available

Product Details:

• Title: Full product name and description
• Brand Name: Manufacturer or brand information
• Short Description: Concise product summary
• Default Image URL: Primary product image for display purposes
• Details: Comprehensive product specifications and features

Categorization:

• Super Department Name/ID: Top-level category (e.g., "Food & Drink")
• Department Name/ID: Mid-level category (e.g., "Fresh Food")
• Aisle Name/ID: Specific product section (e.g., "Vegetables")
• Shelf Name/ID: Precise product location within category

Purchase Specifications:

• Average Weight: Typical product weight for variable-weight items
• Bulk Buy Limit: Maximum quantity purchasable in single transaction
• Min/Max Weight: Weight range for variable products
• Increment: Minimum purchase increment for bulk items

Enhanced Data:

• Reviews: Customer rating and review information
• Media: Additional product images and multimedia content
• Restrictions: Age restrictions, delivery limitations, or purchase constraints
• Is New: Flag indicating recently added products

This comprehensive data structure enables detailed market analysis, competitive pricing strategies, inventory management, and customer behavior insights.

Usage Guidelines and Best Practices

Step-by-Step Implementation:

Option A - URL-Based Scraping:

1. Target URL Selection: Begin by identifying specific Tesco category pages or search results that contain your desired products. Use Tesco's website navigation to find relevant sections and copy the complete URLs.

2. Proxy Configuration: Configure residential proxies from UK (GB) or nearby countries to avoid detection. The scraper's success rate significantly improves with proper proxy settings.

3. Parameter Optimization: Set appropriate limits for max_items_per_url based on your data requirements. Start with smaller limits (10-20) for testing, then scale up for production runs.

4. Execution Monitoring: Monitor scraping progress and adjust retry settings based on success rates. Higher retry counts improve data completeness but increase execution time.

Option B - Filter-Based Scraping:

1. Define Search Keywords: Configure the keyword parameter with specific product names or categories (e.g., "bread", "organic vegetables", "dairy products").

2. Set Sorting Preferences: Use the sort_by parameter to organize results:

   • "relevance" for most relevant matches
   • "price-ascending" to find budget-friendly options
   • "price-descending" to identify premium products

3. Configure Pagination: Set the starting page number to continue from where you left off or target specific result ranges.

4. Proxy Configuration: Configure UK (GB) residential proxies for optimal performance and to avoid detection.

5. Parameter Optimization: Set max_items_per_url based on your needs and enable ignore_url_failures for robust scraping.

6. Execution Monitoring: Monitor progress and refine your keyword and sorting parameters based on initial results.

Best Practices:

Method Selection:

• Use URL-based scraping for complex category hierarchies or when you need specific filtered results from Tesco's website
• Use filter-based scraping for simple keyword searches across the entire Tesco catalog

Scraping Strategy:

• Implement respectful scraping intervals to avoid overwhelming Tesco's servers
• Always use UK (GB) proxies to maintain geographic consistency and improve success rates
• Start with broad keywords and refine based on results when using filter-based scraping

Data Management:

• Regularly update URL patterns as Tesco may modify their website structure (for URL-based)
• Test keyword variations to ensure comprehensive product coverage (for filter-based)
• Store extracted data in structured formats for easy analysis and integration
• Validate data quality by cross-checking critical fields like prices and availability

Error Handling:

• Enable ignore_url_failures to prevent one failed URL or search from stopping your entire job
• Monitor for pattern changes in Tesco's website that might affect scraping accuracy
• Adjust max_retries_per_url based on network conditions and success rates

Performance Optimization:

• For price comparison: Use sort_by with "price-ascending" or "price-descending"
• For inventory monitoring: Use "relevance" sorting to get the most current matches
• For large catalogs: Break down into multiple smaller keyword-based searches rather than one large extraction

Common Issue Resolution:

• Access Denied Errors: Rotate proxy settings and reduce request frequency
• Incomplete Data: Increase retry limits and verify URL accessibility
• Structure Changes: Update selectors when Tesco modifies their page layout
• Rate Limiting: Implement delays between requests and use distributed proxy pools

Business Benefits and Applications

The Tesco.com Product Search Scraper delivers significant time savings compared to manual data collection, reducing weeks of research into hours of automated extraction. This solution provides accurate, real-time data extraction from Tesco and other eCommerce platforms, ensuring you have the most up-to-date information.

Key Applications:

• Competitive Analysis: Track competitor pricing strategies and promotional campaigns
• Market Research: Analyze product assortments, brand presence, and category trends
• Inventory Management: Monitor product availability and stock levels
• Price Intelligence: Develop dynamic pricing strategies based on market data
• Product Development: Identify market gaps and consumer demand patterns

The business value extends to improved decision-making speed, enhanced market positioning, and reduced manual research costs. Companies using this data can optimize their product offerings, adjust pricing strategies in real-time, and identify emerging market opportunities before competitors.

Conclusion

The Tesco.com Product Search Scraper represents a powerful solution for accessing comprehensive UK grocery market data efficiently and reliably. With its robust architecture, comprehensive data extraction capabilities, and business-ready output formats, this tool empowers organizations to leverage Tesco's extensive product catalog for strategic advantage. Whether you're conducting market research, optimizing inventory, or developing competitive strategies, this scraper provides the reliable data foundation necessary for informed business decisions.

Ready to unlock valuable market insights from Tesco's vast product catalog? Start extracting comprehensive grocery data today and transform your business intelligence capabilities.

Related Actors

• Tesco.com Product Details Scraper: A specialized data extraction solution engineered to harvest comprehensive product information from Tesco.com's.

Your feedback

We are always working to improve Actors' performance. So, if you have any technical feedback about Tesco.com Product Search Scraper or simply found a bug, please create an issue on the Actor's Issues tab in Apify Console.