Harvard Catalyst Profiles Scraper

Extract comprehensive researcher profiles from the Harvard Catalyst Profiles directory, including contact information and professional details.

What This Actor Does

This Actor collects detailed information about researchers from the Harvard Catalyst Profiles directory. Simply provide your search criteria, and it will automatically:

1. Search for researchers matching your keywords and filters
2. Collect complete profile information for each researcher

Perfect for:

• Academic research collaboration discovery
• Building researcher databases
• Analyzing institutional expertise
• Finding subject matter experts

Input Parameters

• Search Keywords - Terms to search for in profiles (e.g., "cancer research", "neuroscience")
• Department - Filter results by specific department (optional)
• Institution - Filter results by institution name (optional)
• Maximum Profiles - Number of profiles to collect (default: 10)
  • Start small (10-50) to verify results match your needs
  • Scale up gradually as needed

Note: To get all available profiles, use empty search keywords. The Actor automatically saves progress and can resume if interrupted.

Tips for Large-Scale Collection

• Enable "No timeout": Each profile is collected individually to ensure data accuracy, so large collections take extended time. Toggle the "No timeout" switch in the Run options before starting the Actor to avoid premature termination.
• Check your account balance: Verify your Apify account has sufficient credit before starting a large run.
• Data is saved continuously: Every profile is saved immediately — if a run is interrupted, your data is preserved and the Actor can resume from where it left off.
• Some profiles may be incomplete or skipped: Due to temporary server issues or page loading problems, a small number of profiles may not be fully collected. The Actor logs any skipped profiles so you can verify completeness.

Responsible Use

This Actor collects publicly available data from the Harvard Catalyst Profiles directory. Users should:

• Verify compliance with applicable laws and institutional policies
• Respect the Harvard Catalyst Profiles terms of service
• Use collected data ethically and appropriately

Output Data

Each profile includes:

• Basic Information: Name, ID, title, institution, department
• Contact Details: Full address, phone number, fax, email (when available)
• Professional Information: Faculty rank
• Profile URL: Direct link to the researcher's profile page
• Metadata: Collection timestamp and search query used

Quick Start

Using Prefill Configuration

The fastest way to start is using our prefill configuration which searches for cancer researchers:

{
    "searchKeywords": "cancer research",
    "department": "",
    "institution": "",
    "maxItems": 10
}

Custom Search Examples

Search by Keywords

{
    "searchKeywords": "machine learning healthcare",
    "maxItems": 10
}

Filter by Department

{
    "searchKeywords": "genomics",
    "department": "Genetics",
    "maxItems": 10
}

Institution-Specific Search

{
    "institution": "Harvard Medical School",
    "department": "Cell Biology",
    "maxItems": 10
}

Get All Available Profiles

{
    "searchKeywords": "",
    "department": "",
    "institution": "",
    "maxItems": 10000
}

Data Quality

• Structured JSON output in Apify Dataset format
• Automatically extracts email addresses from profile images when available. Since emails are read from images rather than text, occasional misreads may occur — we recommend verifying important addresses
• Comprehensive error handling with informative logging
• Clean data ready for analysis
• Progress tracking for large-scale data collection

Limitations

• Email addresses are extracted from images, so they may not be available for all profiles and occasional misreads are possible
• Some profiles may have incomplete information depending on source data
• A small number of profiles may fail to load due to temporary server issues — these are logged and marked in the output
• Large-scale collections require significant run time — always enable "No timeout" in Run options
• English language interface only
• No authentication required (public data only)

Troubleshooting

No Results Found

• Verify your search keywords are spelled correctly
• Try broader search terms
• Remove department/institution filters to expand results

Incomplete Profile Data

• Some researchers may not have all fields populated in their public profiles
• Email addresses are read from images on the profile page, so image quality affects accuracy
• Check the profile URL to verify data availability on the source website

Slow Performance

• Consider reducing the maxItems parameter for faster completion
• Check your Apify plan's memory allocation

Use Cases

Academic Collaboration Find researchers working on similar topics for potential collaborations and partnerships.

Grant Applications Identify experts in specific fields to support research proposals and grant applications.

Conference Planning Discover potential speakers and panelists in your field of interest.

Talent Recruitment Build a comprehensive database of researchers for academic recruitment purposes.

Data Privacy & Disclaimer

This Actor collects only publicly available information from the Harvard Catalyst Profiles directory. All data is already accessible through the public website. No authentication or login is required.

Educational & Research Use Only: This tool is provided strictly for educational and research purposes. It is intended to demonstrate web scraping techniques and data collection methodologies for learning and academic use.

No Warranty: The data collected may contain inaccuracies, omissions, or incomplete records. Email addresses are extracted from images and may occasionally be misread. Users should independently verify any data before relying on it for important decisions.

User Responsibility: Users are solely responsible for ensuring their use of this Actor and the collected data complies with all applicable laws, regulations, and the Harvard Catalyst Profiles terms of service. The developers assume no liability for misuse. Please use this tool responsibly and ethically.

Support

For issues or questions about this Actor:

1. Review the troubleshooting section above
2. Check your input parameters and configuration
3. Examine the run log for specific error messages and diagnostic information