Centris Broker Scraper

A production-ready scraper for extracting comprehensive real estate broker information from Centris.ca using PlaywrightCrawler. This scraper extracts detailed broker profiles, social media links, and optionally fetches their property listings with photos and metadata.

✨ Features

🏘️ Broker Information Extraction

• Personal Details: Name, professional title, professional photo
• Contact Information: Phone, email, contact URLs
• Identifiers: Broker ID, profile ID for unique identification
• Agency Information: Agency name, logo, and type

📱 Social Media Integration

• LinkedIn: Professional networking profiles
• Facebook: Business pages and personal profiles
• Twitter/X: Social media handles
• Instagram: Visual content accounts
• YouTube: Video content channels
• TikTok: Short-form video accounts

🏠 Property Listings (Optional)

When addBrokerProperties is enabled, extracts:

• Property URLs: Direct links to property detail pages
• Property IDs: Unique identifiers extracted from URLs
• Property Photos: Image galleries with full URLs
• Property Details: Titles, addresses, prices, and types
• Thumbnail Links: Uses .property-thumbnail-summary-link selectors

🔧 Custom Data Extraction

• extendedOutputFunction: Add custom data extraction logic
• Page Access: Full access to Playwright page object for advanced scraping
• Flexible Integration: Merge custom data with standard broker information
• Error Isolation: Custom function errors don't break the main scraping process

📋 Input Parameters

{
  "startUrls": ["https://www.centris.ca/en/real-estate-brokers"],
  "maxItems": 100,
  "endPage": 5,
  "addBrokerProperties": false,
  "proxy": true,
  "extendedOutputFunction": "return { customField: 'value', timestamp: new Date().toISOString() };"
}

Parameter Details

• startUrls (array): Starting URLs for broker discovery
• maxItems (number): Maximum brokers to scrape (default: 100)
• endPage (number): Maximum pages to process (default: 5)
• addBrokerProperties (boolean): Include property listings (default: false)
• proxy (boolean): Use Apify proxy for requests (default: true)
• extendedOutputFunction (string): Custom JavaScript function for additional data extraction

## 🔧 Custom Data Extraction

The `extendedOutputFunction` allows you to add custom data extraction logic. The function receives three parameters:
- `broker`: The extracted broker object
- `page`: Playwright page object for additional scraping
- `log`: Logger object for debugging

### Basic Example
```javascript
"extendedOutputFunction": "return { scrapedAt: new Date().toISOString(), source: 'centris' };"

Advanced Example - Extract Additional Contact Info

"extendedOutputFunction": `
  const additionalData = {};
  
  // Extract additional phone numbers
  const phones = await page.$$eval('a[href^="tel:"]', links => 
    links.map(link => link.href.replace('tel:', ''))
  );
  if (phones.length > 0) {
    additionalData.additionalPhones = phones;
  }
  
  // Extract office hours
  try {
    const hours = await page.$eval('.office-hours', el => el.textContent.trim());
    additionalData.officeHours = hours;
  } catch (e) {
    // Office hours not found
  }
  
  // Add custom metadata
  additionalData.extractedAt = new Date().toISOString();
  additionalData.brokerScore = Math.floor(Math.random() * 100); // Custom scoring
  
  return additionalData;
`

Example - Extract Broker Reviews/Ratings

"extendedOutputFunction": `
  const customData = {};
  
  try {
    // Extract star rating
    const rating = await page.$eval('.rating-stars', el => el.getAttribute('data-rating'));
    customData.rating = parseFloat(rating);
    
    // Extract review count
    const reviewCount = await page.$eval('.review-count', el => parseInt(el.textContent.match(/\\d+/)[0]));
    customData.reviewCount = reviewCount;
    
    // Extract recent reviews
    const reviews = await page.$$eval('.review-item', items => 
      items.slice(0, 3).map(item => ({
        text: item.querySelector('.review-text')?.textContent.trim(),
        date: item.querySelector('.review-date')?.textContent.trim(),
        author: item.querySelector('.review-author')?.textContent.trim()
      }))
    );
    customData.recentReviews = reviews;
    
  } catch (error) {
    log.info('Reviews not found for broker: ' + broker.name);
  }
  
  return customData;
`

Example - Market Area Analysis

"extendedOutputFunction": `
  // Extract served areas and specializations
  const areaData = {};
  
  try {
    // Extract service areas
    const serviceAreas = await page.$$eval('.service-area', areas => 
      areas.map(area => area.textContent.trim())
    );
    areaData.serviceAreas = serviceAreas;
    
    // Extract specializations
    const specializations = await page.$$eval('.specialization', specs => 
      specs.map(spec => spec.textContent.trim())
    );
    areaData.specializations = specializations;
    
    // Extract languages spoken
    const languages = await page.$$eval('.language', langs => 
      langs.map(lang => lang.textContent.trim())
    );
    areaData.languages = languages;
    
  } catch (error) {
    log.info('Extended area data not found for: ' + broker.name);
  }
  
  // Add processing timestamp
  areaData.processedAt = new Date().toISOString();
  
  return areaData;
`

📊 Output Structure

Standard Broker Record

{
  "url": "https://www.centris.ca/en/real-estate-broker~john-doe~agency-name/b1234",
  "name": "John Doe",
  "title": "Residential Real Estate Broker",
  "image": "https://mspublic.centris.ca/media.ashx?id=...",
  "brokerId": "123456",
  "profileId": "abc123...",
  "phone": "514-123-4567",
  "contactUrl": "https://www.centris.ca/en/contact-broker/b1234",
  "website": "https://johndoe-realestate.com",
  "socialMedia": {
    "linkedin": "https://linkedin.com/in/john-doe-realtor",
    "facebook": "https://facebook.com/johndoe.realestate",
    "twitter": "https://twitter.com/johndoe_realtor",
    "instagram": "https://instagram.com/johndoe_properties",
    "youtube": "https://youtube.com/c/johndoe-realestate",
    "tiktok": "https://tiktok.com/@johndoe_homes"
  },
  "agency": {
    "name": "Premium Real Estate Inc.",
    "image": "https://mspublic.centris.ca/media.ashx?id=...",
    "type": "Real Estate Agency"
  },
  "properties": [
    {
      "url": "https://www.centris.ca/en/property/12345-beautiful-home-montreal",
      "id": "12345",
      "title": "Beautiful 3-bedroom house in Montreal",
      "price": "$450,000",
      "type": "House",
      "photos": [
        "https://mspublic.centris.ca/media.ashx?id=...",
        "https://mspublic.centris.ca/media.ashx?id=..."
      ]
    }
  ]
}

Extended Broker Record (with custom function)

{
  "url": "https://www.centris.ca/en/real-estate-broker~john-doe~agency-name/b1234",
  "name": "John Doe",
  "title": "Residential Real Estate Broker",
  "brokerId": "123456",
  "phone": "514-123-4567",
  "socialMedia": { "..." },
  "agency": { "..." },
  "properties": [ "..." ],
  
  "rating": 4.8,
  "reviewCount": 127,
  "recentReviews": [
    {
      "text": "Excellent service, very professional",
      "date": "2024-01-15",
      "author": "Sarah M."
    }
  ],
  "serviceAreas": ["Montreal", "Laval", "Longueuil"],
  "specializations": ["Condos", "First-time buyers", "Investment properties"],
  "languages": ["English", "French", "Spanish"],
  "extractedAt": "2024-01-20T10:30:00.000Z",
  "processedAt": "2024-01-20T10:30:00.000Z"
}

⚖️ Compliance & Best Practices

• Rate Limiting: Built-in delays and concurrency limits
• Respectful Scraping: Follows robots.txt and site guidelines
• Data Privacy: No storage of sensitive personal information
• Error Handling: Robust fallbacks prevent service disruption
• Custom Code Safety: Extended functions are isolated and error-handled

📄 License

This project is for educational and research purposes. Ensure compliance with Centris.ca's terms of service when using this tool.

---

Note: This scraper extracts publicly available information from Centris.ca broker profiles and listings. Always verify data accuracy and comply with applicable data usage regulations.