University Faculty & Research Lab Scraper
Pricing
from $4.99 / 1,000 results
University Faculty & Research Lab Scraper
Scrapes university faculty and research directories, automatically detecting person listings and extracting names, titles, emails, departments, research interests, and profile URLs.
Pricing
from $4.99 / 1,000 results
Rating
0.0
(0)
Developer
Jamshaid Arif
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
11 days ago
Last modified
Categories
Share
Crawls university faculty and research-lab directory pages and returns one record per person: name, academic title, email, department and research interests. It works on any institution's directory because it auto-detects the person-card layout — no per-university configuration needed — and accepts your own CSS/XPath selectors when a site needs them.
How auto-detection works
The scraper groups every repeated container on the page by tag + class, scores each group
by how many of its members contain a person-name link, a mailto: and an academic title
keyword, and picks the winner. On near-ties it prefers the larger group, since the
tighter one usually drops the people who have no photo or no listed email.
Input
| Field | Default | Description |
|---|---|---|
start_urls | ["https://www.eecs.mit.edu/role/faculty/"] | Directory pages to crawl. |
selectors | {} | Optional overrides: item, name, title, email, profile_link, research_interests (CSS or XPath). |
follow_profiles | false | Fetch each profile page to fill in missing email / title / research interests. |
max_pages | 3 | Paginated listing pages to follow per start URL. |
limit | 100 | Max people to return. |
Runs with empty input (MIT EECS faculty, 100 people).
Output
Verified against
| Directory | Cards detected | Field fill (first 6) |
|---|---|---|
MIT EECS (eecs.mit.edu/role/faculty/) | 183 | name 6/6, title 6/6, email 6/6, profile 6/6 |
Cornell CS (cs.cornell.edu/people/faculty) | 20 | name 6/6, title 6/6, email 5/6, research 4/6 |
Two structurally different sites — MIT uses people-entry grid cells with heading links,
Cornell renders names as bare <span>s inside views-row — both handled by the same
heuristics with no selectors supplied.
