Sympla Event Scraper

Overview

This code is a web scraper designed to extract event data from the Sympla event platform (https://www.sympla.com.br). It's built to be used within the Apify platform, which allows for scheduled or on-demand execution of web scraping tasks. The scraper gathers event information, including details like event name, date, location, organizer, and ticket information, for a specified city.

Features

• Event Listing: Fetches lists of events for a given city, with optional filtering by state and date range.
• Date Range Filtering: Supports filtering events by a start and end date.
• Event Details: Optionally fetches detailed information for each event, including descriptions, venue details, and category.
• Ticket Information: Retrieves ticket details for events, such as prices, availability, and sale dates.
• Pagination: Handles pagination of event listings to retrieve events from multiple pages.
• Error Handling: Includes error handling for API requests and data processing.
• Axios for HTTP Requests: Uses Axios for making HTTP requests.
• Apify Integration: Designed to run on the Apify platform.
• Session Management: (Partial) Attempts to use an Axios session to handle cookie-based access to event details and ticket information.
• Input Schema: Defines the expected input parameters for the scraper.

How it Works

The scraper operates in the following general steps:

1. Initialization: The script initializes the Apify actor and retrieves input parameters, including the target city, state, date range, and other options.
2. Event List Fetching:
   • The fetchEventsByCity function is used to retrieve a list of events from Sympla's API. This function constructs a URL with the necessary parameters, including the city, state, and date. It handles pagination to get all events.
   • The d parameter is always included in the request. If a date range is provided, d includes the start and end dates (YYYY-MM-DD,YYYY-MM-DD). If no date range is provided, d defaults to today's date (YYYY-MM-DD).
3. Event Details and Ticket Fetching (Optional):
   • If the fetch_details input parameter is set to true, the scraper fetches detailed information for each event.
   • It uses fetchEventPageDetails to get event descriptions, venue information, etc.
   • It uses fetchEventTickets to get ticket information.
   • A session using Axios is initialized to try to handle cookie requirements for fetching details. If session initialization fails, it falls back to non-session requests.
4. Data Formatting:
   • The formatEventData function is used to structure the event data into a consistent format.
   • The enhanceEventWithPageDetails and enhanceEventWithTicketsOnly functions add the detailed information and ticket data to the formatted event objects.
5. Data Output:
   • The scraper pushes the formatted event data to the Apify dataset, making it available for download or further processing.

Input Parameters

The scraper accepts the following input parameters:

{
    "city": {
        "title": "City",
        "type": "string",
        "description": "Name of the city to search for events",
        "default": "Florianopolis",
        "editor": "textfield"
    },
    "state": {
        "title": "State",
        "type": "string",
        "description": "State code (e.g., SC, SP, RJ)",
        "default": "SC",
        "editor": "textfield"
    },
    "use_date_range": {
        "title": "Use date range",
        "type": "boolean",
        "description": "Whether to filter events by date range",
        "default": true,
        "editor": "checkbox"
    },
    "start_date": {
        "title": "Start date",
        "type": "string",
        "description": "Start date in YYYY-MM-DD format (defaults to today if date range is enabled)",
        "editor": "textfield",
        "pattern": "^\\d{4}-\\d{2}-\\d{2}$",
        "example": "2025-04-15"
    },
    "end_date": {
        "title": "End date",
        "type": "string",
        "description": "End date in YYYY-MM-DD format (defaults to 30 days from start date if date range is enabled)",
        "editor": "textfield",
        "pattern": "^\\d{4}-\\d{2}-\\d{2}$",
        "example": "2025-04-30"
    },
    "fetch_details": {
        "title": "Fetch detailed information",
        "type": "boolean",
        "description": "Whether to fetch detailed information and ticket data for events",
        "default": true,
        "editor": "checkbox"
    },
    "max_events_for_details": {
        "title": "Maximum events for details",
        "type": "integer",
        "description": "Maximum number of events to fetch detailed information for",
        "default": 20,
        "minimum": 1,
        "maximum": 100,
        "editor": "number"
    },
    "max_pages": {
        "title": "Maximum pages",
        "type": "integer",
        "description": "Maximum number of pages to scrape",
        "default": 3,
        "minimum": 1,
        "maximum": 20,
        "editor": "number"
    },
    "request_delay": {
        "title": "Request delay (seconds)",
        "type": "integer",
        "description": "Delay between requests in seconds to avoid rate limiting",
        "default": 2,
        "minimum": 1,
        "maximum": 10,
        "editor": "number"
    },
     "detail_request_delay": {
        "title": "Detail request delay (milliseconds)",
        "type": "integer",
        "description": "Delay between detail requests in milliseconds",
        "default": 1000,
        "minimum": 500,
        "maximum": 5000,
        "editor": "number"
    }
}
DisclaimerAll data extracted by this scraper is publicly available on the Sympla website (https://www.sympla.com.br). This scraper only accesses and retrieves information that is already accessible to any internet user. It does not bypass any access restrictions or retrieve any non-public data.
sageApify Account: You'll need an account on the Apify platform (https://apify.com) to run this scraper.
Apify Actor: This code is designed to be deployed as an Apify actor. You can create a new actor on the Apify platform and upload this code.Input: Configure the actor's input parameters (city, state, date range, etc.) according to your needs.Run: Run the actor on the Apify platform. The scraped data will be stored in the actor's dataset.Output: You can then download the data from the Apify dataset in various formats (JSON, CSV, etc.).DependenciesaxiosapifyDate Parameter (d) BehaviorThe scraper constructs Sympla API requests, and a key parameter in these requests is d, which specifies the date or date range for events.  Here's how the scraper handles the d parameter:With Date Range: If you provide both start_date and end_date in the input, the scraper will include them in the d parameter in the format YYYY-MM-DD,YYYY-MM-DD (e.g., 2025-04-15,2025-04-30).With Only Start Date: If you provide only start_date, the scraper will include it in the d parameter in the format YYYY-MM-DD.Without Date Range: If you do not provide a date range (either by not setting use_date_range to true, or by not providing start/end dates), the scraper will default to including today's date in the d parameter, using the format YYYY-MM-DD.This ensures that the d parameter is *