Overview

The StreetEasy.com Scraper is a sophisticated tool designed to extract real estate listings data from StreetEasy.com, New York City's leading real estate marketplace. Perfect for real estate professionals, researchers, and analysts, this scraper automates the collection of detailed property information, including prices, amenities, building details, and agent information across all NYC boroughs.

Features

• Comprehensive Property Data Collection:
  • Sales and rental listings with complete property details
  • Historical price changes and time on market
  • Building information and amenities
  • Unit-specific details and floor plans
• Advanced Search Capabilities:
  • Filter by borough, neighborhood, or specific areas
  • Price range and property type filtering
  • Custom search criteria support (bedrooms, bathrooms, amenities)
• Market Intelligence:
  • Price per square foot calculations
  • Building history and property records
  • Agent and brokerage information
  • Market trends and pricing history
• Data Management:
  • Customizable data export formats (JSON, CSV, Excel)
  • Deduplication of listings
  • Incremental updates tracking
  • Historical data preservation

How to Use

1. Set Up: Ensure you have an Apify account and access to the Apify platform.
2. Provide Start URLs: Copy and paste the desired search link, specify the location you want to scrape from streeteasy.com (e.g., "https://streeteasy.com/for-rent/brooklyn"), or use a direct listing URL (e.g., "https://streeteasy.com/building/one-fifty/rental/4642882").
3. Adjust Scraper Settings: Modify settings like maxItems, maxConcurrency, and minConcurrency to suit your requirements.
4. Configure Proxy Settings: (Optional) Configure proxy settings for better reliability and to avoid IP blocks.
5. Run the Scraper: Execute the scraper on the Apify platform. The output will be available in various formats like JSON, CSV, or Excel, depending on your preference.

Input Configuration

Example configuration object:

1{
2    "startUrls": [
3        {
4            "url": "https://streeteasy.com/for-rent/brooklyn"
5        }
6    ],
7    "maxItems": 100,
8    "maxConcurrency": 10,
9    "minConcurrency": 1,
10    "maxRequestRetries": 100,
11    "proxy": {
12        "useApifyProxy": true,
13        "apifyProxyGroups": [
14            "RESIDENTIAL"
15        ]
16    }
17}

Input Fields Explanation

• Start URLs (startUrls):

  • Array of URLs to start scraping from
  • Uses the requestListSources editor
  • Example default URL: "https://streeteasy.com/for-rent/brooklyn"

• Max Items (maxItems):

  • Maximum number of listings to scrape per crawl
  • Type: Integer
  • Default: 100
  • Optional: If not defined, all items will be scraped

• Max Concurrency (maxConcurrency):

  • Maximum number of pages processed simultaneously
  • Type: Integer
  • Default: 10

• Min Concurrency (minConcurrency):

  • Minimum number of pages processed simultaneously
  • Type: Integer
  • Default: 1

• Max Request Retries (maxRequestRetries):

  • Number of retry attempts for failed requests
  • Type: Integer
  • Default: 100

• Proxy Configuration (proxy):

  • Configuration for proxy servers to hide scraper origin
  • Type: Object
  • Default configuration:
    • useApifyProxy: true
    • apifyProxyGroups: ["RESIDENTIAL"]
  • Uses the Apify Proxy editor for configuration

StreetEasy Scraper Output Structure

Core Fields

1{
2    "id": "string",
3    "areaName": "string",
4    "availableAt": "ISO 8601 date string",
5    "bedroomCount": "integer",
6    "buildingType": "string",
7    "fullBathroomCount": "integer",
8    "halfBathroomCount": "integer",
9    "furnished": "boolean",
10    "geoPoint": {
11        "latitude": "float",
12        "longitude": "float"
13    },
14    "hasTour3d": "boolean",
15    "hasVideos": "boolean",
16    "isNewDevelopment": "boolean",
17    "leaseTerm": "integer",
18    "livingAreaSize": "integer",
19    "monthsFree": "integer",
20    "noFee": "boolean",
21    "netEffectivePrice": "integer",
22    "offMarketAt": "ISO 8601 date string | null",
23    "price": "integer",
24    "priceChangedAt": "ISO 8601 date string",
25    "priceDelta": "integer",
26    "slug": "string | null",
27    "sourceGroupLabel": "string",
28    "sourceType": "string",
29    "state": "string",
30    "status": "string",
31    "street": "string",
32    "upcomingOpenHouse": "object | null",
33    "unit": "string",
34    "urlPath": "string",
35    "zipCode": "string"
36}

Rental Object Structure

1{
2    "rental": {
3        "__typename": "Rental",
4        "id": "string",
5        "address": {
6            "latitude": "string",
7            "longitude": "string"
8        },
9        "anyrooms": "integer",
10        "area": {
11            "id": "string",
12            "name": "string",
13            "borough": {
14                "name": "string"
15            }
16        },
17        "closed_at": "date | null",
18        "days_on_market": "integer",
19        "is_furnished": "boolean",
20        "is_featured": "boolean",
21        "is_sponsored": "boolean",
22        "is_viewed": "boolean",
23        "listed_at": "ISO 8601 date string",
24        "opaque": "string",
25        "status_date": "date | null",
26        "status_title": "string",
27        "source_group_id": "string",
28        "sourceuri": "string | null",
29        "quick_url": "string",
30        "smallImageURL": "string",
31        "mediumImageURL": "string",
32        "is_show_bsc_looker_report": "boolean",
33        "twilio_number": "string | null"
34    }
35}

Building Information

1{
2    "building": {
3        "id": "string",
4        "title": "string",
5        "subtitle": "string",
6        "is_new_development": "boolean",
7        "building_type": "string",
8        "year_built": "integer",
9        "open_rentals_count": "integer",
10        "open_sales_count": "integer",
11        "show_building_premium_page": "boolean",
12        "medium_image_uri": "string",
13        "residential_unit_count": "integer",
14        "landmark_name": "string",
15        "type": "string",
16        "status": "string",
17        "additionalDetails": {
18            "leasingStartDate": "ISO 8601 date string",
19            "salesStartDate": "ISO 8601 date string | null"
20        }
21    }
22}

Property Features and Amenities

1{
2    "propertyDetails": {
3        "address": {
4            "street": "string",
5            "houseNumber": "string",
6            "streetName": "string",
7            "city": "string",
8            "state": "string",
9            "zipCode": "string",
10            "unit": "string"
11        },
12        "roomCount": "integer",
13        "amenities": {
14            "list": ["string"],
15            "doormanTypes": ["string"],
16            "parkingTypes": ["string"],
17            "sharedOutdoorSpaceTypes": ["string"],
18            "storageSpaceTypes": ["string"]
19        },
20        "features": {
21            "list": ["string"],
22            "fireplaceTypes": ["string"],
23            "privateOutdoorSpaceTypes": ["string"],
24            "views": ["string"]
25        }
26    }
27}

Contact Information

1{
2    "contacts": [
3        {
4            "secondary_phone": "string",
5            "license": {
6                "display_type": "string"
7            },
8            "email": "string",
9            "id": "string",
10            "name": "string",
11            "primary_phone": "string",
12            "is_pro": "boolean",
13            "source_group": {
14                "label": "string",
15                "logo_url": "string"
16            },
17            "photo_uri": "string",
18            "profile": {
19                "url": "string"
20            },
21            "user_id": "string"
22        }
23    ]
24}

License Information

1{
2    "license": {
3        "business_name": "string",
4        "business_address_1": "string",
5        "business_address_2": "string | null",
6        "business_city": "string",
7        "business_state": "string",
8        "business_zip": "string",
9        "full_business_address": "string",
10        "license_type": {
11            "label": "string"
12        }
13    }
14}

Price History

1{
2    "price_histories": [
3        {
4            "date": "YYYY-MM-DD",
5            "price": "integer",
6            "description": "string",
7            "event": "string",
8            "closing_id": "string | null"
9        }
10    ]
11}

Media Content

1{
2    "media": {
3        "photos": [
4            {
5                "key": "string",
6                "description": "string",
7                "url": "string"
8            }
9        ],
10        "floorPlans": ["array"],
11        "videos": ["array"],
12        "tour3dUrl": "string | null",
13        "assetCount": "integer"
14    }
15}

Building Policies

1{
2    "policies": {
3        "list": ["string"],
4        "petPolicy": {
5            "catsAllowed": "boolean | null",
6            "dogsAllowed": "boolean | null",
7            "maxDogWeight": "integer | null",
8            "restrictedDogBreeds": ["string"]
9        }
10    }
11}

Nearby Information

1{
2    "nearby": {
3        "transitStations": [
4            {
5                "name": "string",
6                "distance": "float",
7                "routes": ["string"],
8                "geo": {
9                    "latitude": "float",
10                    "longitude": "float"
11                }
12            }
13        ]
14    }
15}

Schools Information

1{
2    "nearbySchools": [
3        {
4            "name": "string",
5            "district": "string",
6            "grades": ["string"],
7            "id": "string",
8            "idstr": "string",
9            "geoCenter": {
10                "latitude": "float",
11                "longitude": "float"
12            }
13        }
14    ]
15}

Property History

1{
2    "propertyHistory": [
3        {
4            "listingId": "string",
5            "sourceGroupLabel": "string",
6            "offMarketAt": "ISO 8601 date string | null",
7            "rentalEventsOfInterest": [
8                {
9                    "date": "YYYY-MM-DD",
10                    "price": "integer",
11                    "pricePercentChange": "float",
12                    "status": "string"
13                }
14            ]
15        }
16    ]
17}

Field Type Definitions

Status Values

• status: "ACTIVE", "DRAFT", etc.
• buildingType: "RENTAL", etc.

Building Amenities

Common values in amenities.list:

• BIKE_ROOM
• SHARED_OUTDOOR_SPACE
• DOORMAN
• ELEVATOR
• FIOS_AVAILABLE
• PARKING
• GYM
• LAUNDRY
• LIVE_IN_SUPER
• PACKAGE_ROOM
• WHEELCHAIR_ACCESS

Property Features

Common values in features.list:

• CENTRAL_AC
• VIEW
• DISHWASHER
• HARDWOOD_FLOORS
• WASHER_DRYER

View Types

Common values in features.views:

• CITY
• GARDEN
• PARK
• SKYLINE

Building Policies

Common values in policies.list:

• GUARANTORS_ACCEPTED
• PETS_ALLOWED
• SMOKE_FREE

Price Events

Common values for event in price histories:

• PRICE_DECREASE
• LISTED
• PRICE_INCREASE

Note: This documentation covers the main data structures and their fields. Some fields may be conditionally present or null depending on the listing type and available information.

Benefits

• Access comprehensive NYC real estate data programmatically
• Track market trends and price changes in real-time
• Generate detailed market analysis reports
• Monitor specific buildings or neighborhoods
• Export data in various formats for analysis
• Save countless hours of manual data collection

Why Choose This Scraper?

This StreetEasy.com scraper is specifically designed for the unique characteristics of the NYC real estate market. It handles the complexities of StreetEasy's data structure, including co-op and condo-specific information, building amenities, and NYC-specific property types. With built-in rate limiting and proxy support, it ensures reliable data collection while respecting the platform's usage guidelines.

Support

• GitHub Issues: Report issues here
• Documentation: Full documentation
• Contact:
  • Email: support@example.com
  • Twitter: @StreetEasyScraper

Additional Services

• Custom development and integration support
• Bulk data extraction services
• Market analysis report generation
• API access for real-time data
• Training and consulting services

---

Note: This scraper is not affiliated with, authorized, maintained, sponsored, or endorsed by StreetEasy.com or any of its affiliates or subsidiaries. This is an independent and unofficial software. Use at your own discretion.