10times - Top 100 Pages Crawler

Overview

This crawler extracts event data from 10times Top 100 pages across multiple categories. It provides comprehensive information about events including their rank, name, dates, location, category, and rating.

The crawler supports all major industry categories: Agriculture & Forestry, Animals & Pets, Apparel & Clothing, Arts & Crafts, Auto & Automotive, Baby, Kids & Maternity, Banking & Finance, Building & Construction, Business Services, Education & Training, Electric & Electronics, Entertainment & Media, Environment & Waste, Fashion & Beauty, Food & Beverages, Home & Office, Hospitality, IT & Technology, Industrial Engineering, Logistics & Transportation, Medical & Pharma, Miscellaneous, Packing & Packaging, Power & Energy, Science & Research, Security & Defense, Telecommunication, Travel & Tourism, Wellness, Health & Fitness.

Features

• Multi-Category Support: Extracts data from 30+ industry categories
• Comprehensive Event Details: Collects event names, dates, locations, and ratings
• Structured Output: Returns data in an organized, easy-to-use JSON format
• Configurable Input: Works with any category from the 10times Top 100 pages
• Advanced Anti-Detection: Implements sophisticated stealth techniques to avoid blocking
• Robust Error Handling: Comprehensive error handling and retry mechanisms
• Session Management: Intelligent session rotation and cookie management

Input Parameters

Parameter	Type	Description	Default
url	String	The category URL to scrape	https://10times.com/top100/travel-tourism
proxyConfiguration	Object	Proxy configuration settings	See below
maxRetries	Integer	Maximum retry attempts (1-10)	5
requestTimeout	Integer	Request timeout in seconds (30-600)	300

Example Input

{
  "url": "https://10times.com/top100/travel-tourism",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": ""
  },
  "maxRetries": 5,
  "requestTimeout": 300
}

Proxy Configuration

IMPORTANT: Proper proxy configuration is crucial for successful scraping. 10times.com has sophisticated anti-bot measures.

Recommended Settings:

{
  "useApifyProxy": true,
  "apifyProxyGroups": ["RESIDENTIAL"],
  "apifyProxyCountry": "IN"
}

Why These Settings Work Best:

1. RESIDENTIAL Proxy Group: Provides real residential IP addresses that are less likely to be blocked
2. India (IN) Country: 10times.com has better accessibility from Indian IP addresses
3. Session Rotation: The crawler automatically rotates sessions to avoid detection

Alternative Proxy Options:

• DATACENTER: Faster but more likely to be blocked
• RESIDENTIAL: Slower but more reliable (recommended)
• Other Countries: Try US, UK, or DE if IN doesn't work

Output Format

The crawler produces structured JSON data with the following format:

[
    {
        "rank": "1",
        "eventLink": "https://10times.com/magic",
        "eventName": "MAGIC LAS VEGAS",
        "eventDates": "10 - 12 Feb 2025",
        "dateStart": "20250210T000000Z",
        "dateEnd": "20250212T235959Z",
        "location": "Las Vegas",
        "category": "Apparel & Clothing",
        "rating": "4.3",
        "extractedAt": "2025-01-15T10:30:00.000Z",
        "sourceUrl": "https://10times.com/top100/travel-tourism"
    }
]

Available Categories

This crawler supports all categories listed on https://10times.com/top100/, including:

• Agriculture & Forestry
• Animals & Pets
• Apparel & Clothing
• Arts & Crafts
• Auto & Automotive
• Baby, Kids & Maternity
• Banking & Finance
• Building & Construction
• Business Services
• Education & Training
• Electric & Electronics
• Entertainment & Media
• Environment & Waste
• Fashion & Beauty
• Food & Beverages
• Home & Office
• Hospitality
• IT & Technology
• Industrial Engineering
• Logistics & Transportation
• Medical & Pharma
• Miscellaneous
• Packing & Packaging
• Power & Energy
• Science & Research
• Security & Defense
• Telecommunication
• Travel & Tourism
• Wellness, Health & Fitness

Troubleshooting

Common Issues and Solutions

1. 403 Forbidden Errors

Symptoms:

• Request blocked - received 403 status code
• Access denied - page blocked

Solutions:

• Use RESIDENTIAL proxy group instead of DATACENTER
• Set proxy country to India (IN)
• Increase delays between requests
• Check if the URL is valid and accessible

2. Execution Context Destroyed

Symptoms:

• page.evaluate: Execution context was destroyed, most likely because of a navigation

Solutions:

• This is usually caused by page redirects or anti-bot measures
• The crawler automatically handles this with retry logic
• Ensure proper proxy configuration

3. Tunnel Connection Failed

Symptoms:

• net::ERR_TUNNEL_CONNECTION_FAILED

Solutions:

• Check proxy configuration
• Try different proxy groups or countries
• Ensure sufficient proxy credits

4. No Data Extracted

Symptoms:

• No valid data extracted
• Event list table not found

Solutions:

• Verify the URL is correct and accessible
• Check if the page structure has changed
• Use different proxy settings
• Try a different category URL

5. Session Rotation Issues

Symptoms:

• Multiple failed requests with the same session
• Sessions being marked as bad repeatedly

Solutions:

• The crawler automatically handles session rotation
• Check proxy configuration
• Ensure sufficient proxy credits

Debug Information

The crawler generates debug files when issues occur:

• debug-{timestamp}.png: Screenshots when table not found
• error-{timestamp}.png: Screenshots when errors occur

These files help identify what the page looks like when issues occur.

Performance Optimization

1. Use RESIDENTIAL Proxies: More reliable but slower
2. Limit Concurrency: Keep at 1 for best results
3. Proper Delays: Built-in random delays prevent detection
4. Session Management: Automatic session rotation

Technical Details

Anti-Detection Features

• User Agent Rotation: Random selection from realistic browser user agents
• Viewport Randomization: Different screen sizes for each request
• Stealth Script Injection: Masks automation indicators
• Human-like Behavior: Random mouse movements and scrolling
• Cookie Management: Preserves successful session cookies
• Header Randomization: Realistic HTTP headers

Error Handling

• Automatic Retries: Configurable retry attempts
• Session Rotation: Bad sessions are automatically replaced
• Timeout Management: Configurable timeouts for different operations
• Graceful Degradation: Continues processing even if some requests fail

Legal Notice

By using this actor, you agree to:

1. Respect Website Terms: Comply with the terms of service of 10times.com
2. Data Ownership: Acknowledge that the data belongs to respective owners and is subject to copyright laws
3. Responsible Use: Use the actor only for lawful purposes without causing harm to the target website
4. Privacy Compliance: Handle any collected data in accordance with applicable privacy regulations
5. Liability: Accept responsibility for how the data is used and any consequences thereof

Support

If you encounter persistent issues:

1. Check the troubleshooting section above
2. Verify your proxy configuration
3. Try different category URLs
4. Ensure you have sufficient proxy credits
5. Check the debug screenshots for visual clues

The crawler is designed to be robust and handle most common issues automatically, but proper proxy configuration is essential for success.