Proff.no Lead Scraper

An Apify Actor that scrapes business listings and company detail pages from proff.no using HTTPX and BeautifulSoup.

It collects company data such as name, categories, phone number, address, website, website status, email addresses, and social links (Facebook, Instagram, LinkedIn, X/Twitter, YouTube, TikTok, Pinterest).
When a company website is available and reachable, the Actor optionally crawls a few same-domain pages to discover additional contacts and social profiles.

---

Key Features

Listing → Detail Workflow

• Starts from Proff listing pages (e.g. https://www.proff.no/bransjesøk?q=Advokater%20og%20juridiske%20tjenester)
• Extracts company detail URLs and processes each profile
• Supports pagination via “Neste” and stops at a user-defined maximum number of results

Business Details Extraction

Extracts:

• name
• categories (breadcrumb path)
• phone
• address (with heuristics tuned for Norwegian addresses using 4-digit postnummer)
• website (from structured data first, then best-scored anchor links)
• website_details (ok, 404, unavailable, banned, or n/a)
• email1, email2, ...
• Social links (facebook, instagram, linkedin, x, youtube, tiktok, pinterest)

Email Detection

• Finds mailto: links and visible emails in page text
• Cleans and deduplicates with strict regex filtering

Tiny Same-Domain Crawl

• Visits a small number of pages on the company website (default 3)
• Prioritizes URLs containing kontakt, om, about, or similar contact-related paths
• Extracts additional emails and social links

Apify Integrations

• Uses RequestQueue for pagination and deduplication
• Stores structured results in Dataset
• Fully asynchronous using HTTPX

---

Input Configuration

Defined in input_schema.json.

Field	Type	Description
start_urls	array	Listing pages to start from (e.g. Proff bransjesøk URLs)
max_depth	integer	Crawl depth for generic link following (usually leave as 1)
max_results	integer	Maximum number of business records to save; auto-paginates until this limit
site_email_max_pages	integer	Pages to visit on company website for contacts/socials (default: 3)
timeout_seconds	integer	Per-request read timeout (default: 30)

Example input:

{
  "start_urls": [
    {"url": "https://www.proff.no/bransjesøk?q=Advokater%20og%20juridiske%20tjenester"}
  ],
  "max_depth": 1,
  "max_results": 50,
  "site_email_max_pages": 3,
  "timeout_seconds": 30
}

---

How It Works

1. Start URLs are enqueued as listing pages
2. Each listing page is fetched and parsed for company detail links
3. Each company detail page is scraped for core fields, emails, and a candidate website
4. If a valid website exists, the Actor checks its availability and crawls a few internal pages to find additional contacts and social profiles
5. Results are pushed to the default dataset

---

Example Output

{
  "source_url": "https://www.proff.no/selskap/frost-advokatfirma-as/kirkenes/advokater-og-juridiske-tjenester/IF9ORAU07U6",
  "name": "Frost Advokatfirma AS",
  "categories": "Advokater og juridiske tjenester, Finnmark",
  "phone": "+47 12 34 56 78",
  "address": "Eksempelveien 1, 9900 Kirkenes",
  "website": "https://www.frostdemo.no",
  "email1": "post@frostdemo.no",
  "website_details": "ok",
  "social_facebook": "https://www.facebook.com/frostdemo",
  "social_instagram": "n/a",
  "social_linkedin": "n/a",
  "social_x": "n/a",
  "social_youtube": "n/a",
  "social_tiktok": "n/a",
  "social_pinterest": "n/a"
}

---

Notes

• Minimum cost is enabled while in beta. Pricing may be updated later.
• Avoid overloading Proff.no by keeping reasonable request limits.
• For the most reliable results, ensure that the target URLs are valid Proff listing pages.

---

Disclaimer & License

This Apify Actor is provided "as is", without warranty of any kind — express or implied — including but not limited to the warranties of merchantability, fitness for a particular purpose, and non-infringement. Use it, modify it, break it, or improve it — but you do so at your own risk.

© 2025 SLSH. All rights reserved. Copying or modifying the source code is prohibited.