Domain Search
Pricing
from $3.12 / 1,000 api requests
Find every email for any domain the fastest, largest domain-wide discovery engine, crawling 2 billion pages every month, with advanced email extractors that return rich contact data and company profiles.
Pricing
from $3.12 / 1,000 api requests
Rating
0.0
(0)
Developer
Tomba io
Maintained by CommunityActor stats
2
Bookmarked
44
Total users
6
Monthly active users
16 days ago
Last modified
Categories
Share
1[settings]2idiomatic_version_file_enable_tools = []3
4[tools."github:apify/apify-cli"]5version = "latest"6bin = "apify"7
8[tools."github:apify/apify-cli".platforms]9linux-x64 = { asset_pattern = "apify-*-linux-x64" }10linux-arm64 = { asset_pattern = "apify-*-linux-arm64" }11macos-x64 = { asset_pattern = "apify-*-darwin-x64" }12macos-arm64 = { asset_pattern = "apify-*-darwin-arm64" }13windows-x64 = { asset_pattern = "apify-*-windows-x64.exe" }14windows-arm64 = { asset_pattern = "apify-*-windows-arm64.exe" }.prettierignore{ "printWidth": 120, "singleQuote": true, "tabWidth": 4}1# Development2
3Notes for maintainers of the Tomba Domain Search Actor. The README is the end-user page shown on Apify Store; [USAGE.md](USAGE.md) covers running the Actor locally and troubleshooting.4
5## Requirements6
7- Node.js 20+8- [Apify CLI](https://docs.apify.com/cli) for deployment9
10## Scripts11
12```bash13npm install14npm run build # compile TypeScript to dist/15npm run lint # ESLint (src and test)16npm run format # Prettier17npm test # unit + end-to-end tests (node:test)18npm start # run locally with tsx19```20
21## Credentials22
23The Actor uses our Tomba account. Credentials come from environment variables, never from the input:24
25| Variable | Description |26| -------------------- | -------------------------------------------------- |27| `TOMBA_API_KEY` | Tomba API key (`ta_…`) |28| `TOMBA_API_SECRET` | Tomba secret (`ts_…`) |29| `TOMBA_API_ENDPOINT` | Optional API base URL; only used by the test suite |3031`.actor/actor.json` maps the variables to Apify secrets:32
33```bash34apify secrets add tombaApiKey ta_xxxxxxxxxxxxxxxxxxxx35apify secrets add tombaApiSecret ts_xxxxxxxxxxxxxxxxxxxx36apify push37```38
39Run locally:40
41```bash42TOMBA_API_KEY=ta_… TOMBA_API_SECRET=ts_… npm start43```44
45If the variables are missing the run fails with "Actor is misconfigured" before any request is made.46
47## Pricing (pay per event)48
49In **Apify Console → Publication → Monetization**, choose **Pay per event** and add:50
51| Event | Price | Charged when |52| --------------- | -------- | ------------------------------------------------------------ |53| `tomba-request` | $0.00312 | Once per credit of a billable Domain Search page (see below) |5455Domain Search costs credits, charged as `tomba-request` events with `count`:56
57- `ceil(limit / 10)` credits per page (limit 10 = 1, 20 = 2, 50 = 5)58- plus `PHONE_CREDITS` (5) credits per returned address with phone data, only when `enrichMobile` is on: an address whose `phone_data` array is non-empty (`hasPhoneData()` in `src/tomba.ts`). The `emails[].phone_number` boolean is only Tomba's "has a phone" flag and is never billed59
60`domainSearchCredits()` in `src/main.ts` computes this from the whole returned page (even if fewer addresses are kept because of `maxEmailsPerDomain`) and passes it to `callTomba()` as the charge count. Each page is a separate Tomba request and is charged on its own; the item's `chargedCredits` is the sum.61
62`isBillable()` in `src/tomba.ts` mirrors Tomba's billing:63
64| Tomba outcome | Charged |65| ---------------------------------------------------------------------- | ------- |66| JSON with non-empty `data`, including an organization with zero emails | Yes |67| Error status (4xx, 5xx, including 422 and 429) | No |68| Success with empty or null `data` | No |69| Success with an `errors` object | No |70| Non-JSON body (reported as 502) | No |71| Cache hit | No |7273There is no client-side rate limit; throughput is controlled by `maxConcurrency`.74
75## Architecture76
77- `src/tomba.ts`: shared helper, identical in every Tomba Actor. It handles credentials, caching (per-Actor `tomba-cache-<actorId>` key-value store; falls back to an in-run cache if it can't be opened), retries with exponential backoff, pay-per-event charging, budget reservation, the concurrency pool and resume state.78- `src/main.ts`: input handling, paging and output mapping.79 - Input is `domains` and/or `companies` (at least one required; the run fails before any request otherwise). Domains are normalized (`normalizeDomain`) and deduplicated; company names are trimmed (inner whitespace collapsed) and deduplicated case-insensitively. A domain is sent as `domain=`, a company name as `company=`.80 - Resume-state keys are the plain domain for domains and `company:<lowercased name>` for companies; cache keys differ automatically because the request params differ (`domain` vs `company`).81 - Optional query parameters: `department`, `country`, `enrich_mobile=true` (only when `enrichMobile` is on) and `webhook_url` (from `webhookUrl`, must start with `http://` or `https://`). `education` and `healthcare` departments are not in the SDK type but are accepted by the API, so the params are cast.82 - Searches run in parallel (`runPool`); the pages of one search run sequentially, starting at `page`.83 - Paging stops when `maxEmailsPerDomain` is reached, at a short page, at `meta.total_pages`, when `meta.total` emails were collected, at a non-billable page, or when the charge limit is reached (the domain keeps the pages already fetched).84 - If the charge limit is reached before the first page of a domain, the domain is not marked done, so a resumed run processes it.85 - One dataset item per search: `domain` or `company` (+ `domain` from `meta.params.domain` / `organization.website_url` when Tomba reports it), `organization` (if `includeCompanyInfo`), `emails` (passed through, so `phone_data` is included with `enrichMobile`), `phoneNumbers` (sum of `phone_data` lengths), `meta` (of the last page), `pages`, `chargedRequests`, `chargedCredits`, `charged`, `cached`; failures are `{ domain | company, charged: false, cached: false, error }`.86 - A `SUMMARY` record is written to the default key-value store (batch runs only).87 - `outputFormat` is accepted for backwards compatibility but does not change the output.88- The `tomba` SDK v1.1.1 resolves every call to `{ data, rateLimit }`, where `data` is the response body. Its `.d.ts` types still declare the old return type, so always go through `callTomba()`.89
90## Tests91
92- `test/tomba.test.ts`: unit tests for the shared helper (identical in every Actor)93- `test/main.test.ts`: end-to-end tests that run `src/main.ts` against a local mock Tomba API (paging, billing incl. phone credits, `companies`, `enrich_mobile` / `webhook_url`, cache, retries, charge limit and resume, concurrency, credentials, Standby HTTP API). The mock adds `phone_data` only when the request has `enrich_mobile=true` (every third address gets one number)94- `test/helpers.ts`: mock server and Actor runner (identical in every Actor)95
96Locally, the Apify SDK prices every event at $1 when `ACTOR_TEST_PAY_PER_EVENT=true`, so the tests use `maxTotalChargeUsd` as an event count.97
98## Standby mode (real-time API)99
100`.actor/actor.json` sets `usesStandbyMode: true` and `webServerSchema: ./web_server_schema.json` (OpenAPI 3).101
102- `src/standby.ts` (shared, identical in every Actor): `runActor()` runs a batch job, or, when `APIFY_META_ORIGIN=STANDBY`, starts an HTTP server on `Actor.config.get('containerPort')`.103 - `GET /` with the `x-apify-container-server-readiness-probe` header, or with no query: readiness / usage.104 - `GET /?…`: input built by `fromQuery()` in `src/main.ts` (`domain`/`domains`, `company`/`companies` as repeated or comma-separated values, `maxEmailsPerDomain`, `includeCompanyInfo`, `page`, `limit`, `department`, `country`, `enrichMobile`, `webhookUrl`).105 - `POST /`: the same JSON input as a batch run.106 - Responses: `200 { items }`, `400` invalid input (`InputError`), `402` max charge limit reached, `404`, `405`.107- `run(input, ctx)` is shared by both modes: `ctx.push()` writes to the dataset in batch runs and to the HTTP response in Standby; `ctx.isDone()`/`ctx.markDone()` persist resume state only in batch runs. The `SUMMARY` record is only written in batch runs.108- Caching and pay-per-event charging work the same in both modes.109
110Try it locally:111
112```bash113APIFY_META_ORIGIN=STANDBY ACTOR_WEB_SERVER_PORT=8080 TOMBA_API_KEY=ta_… TOMBA_API_SECRET=ts_… npm start114curl "localhost:8080/?domain=stripe.com"115```116
117## Key-value store schema118
119`.actor/key_value_store_schema.json` documents the default key-value store records (`INPUT`, `TOMBA_STATE`, `SUMMARY`). The cross-run cache lives in the separate named store `tomba-cache-<actorId>`, one per Actor: under limited permissions an Actor can only open named storages it created itself, so the Tomba Actors must not share one store. If the store can't be opened, the run logs a warning and caches for this run only.120
121## Memory122
123`defaultMemoryMbytes` is 256: the Actor only makes HTTP calls, so more memory just costs more.124
125## Deployment126
127```bash128apify login129apify push130```131
132Or link the Git repository from the [Actor creation page](https://console.apify.com/actors/new).133
134## License135
136ISC{ "extends": "./tsconfig.json", "compilerOptions": { "noEmit": true }, "include": ["./src/**/*", "./test/**/*"]}# configurations.idea.vscode.zed
# crawlee and apify storage foldersapify_storagecrawlee_storagestorage
# installed filesnode_modules
# git folder.git
# dist folderdisttest1# Usage Guide (maintainers)2
3How to run the Tomba Domain Search Actor locally and what to check when something goes wrong. See [Development.md](Development.md) for credentials, pricing and architecture, and the README for the end-user documentation.4
5## Run locally6
71. Put an input in `storage/key_value_stores/default/INPUT.json` (no Tomba credentials in the input):8
9 ```json10 {11 "domains": ["stripe.com"],12 "maxEmailsPerDomain": 20,13 "limit": "10"14 }15 ```16
172. Run the Actor with the Tomba credentials in the environment:18
19 ```bash20 TOMBA_API_KEY=ta_xxx TOMBA_API_SECRET=ts_xxx npm start21 ```22
23On the Apify platform the credentials come from the `tombaApiKey` / `tombaApiSecret` Apify secrets (see `.actor/actor.json`), so users never provide them.24
25## Input26
27- `domains`: domains to search; normalized and deduplicated28- `companies`: company names to search (sent as `company=`); trimmed and deduplicated. At least one of `domains` / `companies` is required29- `maxEmailsPerDomain` (default 10, max 100): the Actor pages automatically to reach it30- `limit` (`"10"`, `"20"` or `"50"`, default `"10"`): emails per page request; every billable page costs `ceil(limit / 10)` `tomba-request` events plus, with `enrichMobile`, 5 per returned address with non-empty `phone_data`31- `page` (default 1): first page to fetch32- `department`, `country`: Tomba filters, passed through as query parameters33- `enrichMobile` (default false): sends `enrich_mobile=true`; emails then carry `phone_data`34- `webhookUrl`: sent as `webhook_url`; Tomba also POSTs the result there (not for cached pages)35- `includeCompanyInfo` (default true): include `organization` in the output36- `outputFormat`: accepted but currently has no effect37- `maxConcurrency` (default 10), `maxRetries` (default 3), `useCache` / `cacheTtlHours` (default on, 24 hours)38
39## Output40
41One dataset item per domain or company name (company searches have `company` and, when Tomba reports it, `domain`):42
43```json44{45 "domain": "stripe.com",46 "organization": { "organization": "Stripe", "...": "..." },47 "emails": [{ "email": "jane@stripe.com", "first_name": "Jane", "phone_data": [], "...": "..." }],48 "phoneNumbers": 0,49 "meta": { "total": 1250, "pageSize": 10, "current": 2, "total_pages": 125 },50 "pages": 2,51 "chargedRequests": 2,52 "chargedCredits": 2,53 "charged": true,54 "cached": false55}56```57
58A search without results is saved as `{ "domain" | "company", "charged": false, "cached": false, "error" }`.59
60## Error handling61
62- **Missing credentials**: the run fails with "Actor is misconfigured" when `TOMBA_API_KEY` / `TOMBA_API_SECRET` are not set; no request is made63- **Network errors, 429 and 5xx**: retried with exponential backoff (`maxRetries`), never charged64- **Invalid domains or filters (4xx)**: saved as an item with an `error` field and `charged: false`, not retried65- **A later page fails**: the domain keeps the pages already fetched66- **Max cost per run reached**: the Actor stops cleanly; a resumed run skips finished domains67
68## Troubleshooting69
70### Authentication errors71
72```73Error: Please enter a valid KEY74```75
76- Check the `TOMBA_API_KEY` / `TOMBA_API_SECRET` environment variables (or the Apify secrets they map to)77- Check the Tomba account status of those credentials78
79### No results80
81- The domain may not have indexed emails, or the `department` / `country` filters are too narrow82- Check that the domain is spelled correctly83
84### Slow runs85
86- Raise `maxConcurrency`; there is no client-side rate limit87- Use `limit: "50"` so fewer page requests are needed (the credit cost per result is the same)88- Keep `useCache` on for re-runs# This file tells Git which files shouldn't be added to source control
.idea.vscode.zedstorageapify_storagecrawlee_storagenode_modulesdisttsconfig.tsbuildinfoexamples/
# Added by Apify CLI.venv1import prettier from 'eslint-config-prettier';2
3import apify from '@apify/eslint-config/ts.js';4import globals from 'globals';5import tsEslint from 'typescript-eslint';6
7// eslint-disable-next-line import/no-default-export8export default [9 { ignores: ['**/dist', 'eslint.config.mjs', 'examples/**'] },10 ...apify,11 prettier,12 {13 languageOptions: {14 parser: tsEslint.parser,15 parserOptions: {16 project: 'tsconfig.eslint.json',17 },18 globals: {19 ...globals.node,20 ...globals.jest,21 },22 },23 plugins: {24 '@typescript-eslint': tsEslint.plugin,25 },26 rules: {27 'no-console': 0,28 },29 },30 {31 // node:test's describe/it return promises by design.32 files: ['test/**/*.ts'],33 rules: {34 '@typescript-eslint/no-floating-promises': 0,35 },36 },37];root = true
[*]indent_style = spaceindent_size = 4charset = utf-8trim_trailing_whitespace = trueinsert_final_newline = trueend_of_line = lf{ "extends": "@apify/tsconfig", "compilerOptions": { "module": "NodeNext", "moduleResolution": "NodeNext", "target": "ES2022", "outDir": "dist", "noUnusedLocals": false, "skipLibCheck": true, "lib": ["DOM"] }, "include": ["./src/**/*"]}{ "name": "@tomba-io/domain-search", "version": "1.0.0", "type": "module", "description": "An Apify Actor that uses Tomba API to find email addresses associated with a given domain.", "engines": { "node": ">=18.0.0" }, "dependencies": { "apify": "^3.7.2", "tomba": "^1.1.1" }, "devDependencies": { "@apify/eslint-config": "^1.0.0", "@apify/tsconfig": "^0.1.1", "@types/node": "^22.15.32", "ajv": "^8.20.0", "eslint": "^9.29.0", "eslint-config-prettier": "^10.1.5", "globals": "^16.2.0", "prettier": "^3.5.3", "standard-version": "^9.5.0", "tsx": "^4.20.3", "typescript": "^5.8.3", "typescript-eslint": "^8.34.1" }, "scripts": { "start": "npm run start:dev", "start:prod": "node dist/main.js", "start:dev": "tsx src/main.ts", "build": "tsc", "lint": "eslint src test", "lint:fix": "eslint src test --fix", "format": "prettier --write .", "format:check": "prettier --check .", "release": "standard-version && git push --follow-tags origin main", "test": "node --import tsx --test test/*.test.ts" }, "author": "It's not you it's me", "license": "ISC"}# Specify the base Docker image. You can read more about# the available images at https://docs.apify.com/sdk/js/docs/guides/docker-images# You can also use any other image from Docker Hub.FROM apify/actor-node:22 AS builder
# Check preinstalled packagesRUN npm ls crawlee apify puppeteer playwright
# Copy just package.json and package-lock.json# to speed up the build using Docker layer cache.COPY package*.json ./
# Install all dependencies. Don't audit to speed up the installation.RUN npm install --include=dev --audit=false
# Next, copy the source files using the user set# in the base image.COPY . ./
# Install all dependencies and build the project.# Don't audit to speed up the installation.RUN npm run build
# Create final imageFROM apify/actor-node:22
# Check preinstalled packagesRUN npm ls crawlee apify puppeteer playwright
# Copy just package.json and package-lock.json# to speed up the build using Docker layer cache.COPY package*.json ./
# Install NPM packages, skip optional and development dependencies to# keep the image small. Avoid logging too much and print the dependency# tree for debuggingRUN npm --quiet set progress=false \ && npm install --omit=dev --omit=optional \ && echo "Installed NPM packages:" \ && (npm list --omit=dev --all || true) \ && echo "Node.js version:" \ && node --version \ && echo "NPM version:" \ && npm --version \ && rm -r ~/.npm
# Copy built JS files from builder imageCOPY /usr/src/app/dist ./dist
# Next, copy the remaining files and directories with the source code.# Since we do this after NPM install, quick build will be really fast# for most source file changes.COPY . ./
# Run the image.CMD npm run start:prod --silent{ "actorSpecification": 1, "fields": { "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "company": { "type": "string", "description": "The company name that was searched (company searches only)" }, "domain": { "type": "string", "description": "The domain that was searched (for a company search: the domain Tomba resolved, when reported)" }, "organization": { "type": ["object", "null"], "description": "Organization information for the domain", "properties": { "website_url": { "type": ["string", "null"], "description": "Company website URL" }, "organization": { "type": ["string", "null"], "description": "Company name" }, "location": { "type": ["object", "null"], "description": "Company location details", "properties": { "country": { "type": ["string", "null"], "description": "Country code" }, "city": { "type": ["string", "null"], "description": "City name" }, "state": { "type": ["string", "null"], "description": "State/Province" }, "street_address": { "type": ["string", "null"], "description": "Street address" }, "postal_code": { "type": ["string", "null"], "description": "Postal/ZIP code" } } }, "social_links": { "type": ["object", "null"], "description": "Company social media links", "properties": { "twitter_url": { "type": ["string", "null"], "description": "Twitter profile URL" }, "facebook_url": { "type": ["string", "null"], "description": "Facebook page URL" }, "linkedin_url": { "type": ["string", "null"], "description": "LinkedIn company page URL" } } }, "phone_number": { "type": ["boolean", "string", "null"], "description": "Company phone number" }, "industries": { "type": ["string", "null"], "description": "Company industry" }, "founded": { "type": ["string", "null"], "description": "Year company was founded" }, "company_size": { "type": ["string", "null"], "description": "Company size range" }, "company_type": { "type": ["string", "null"], "description": "Type of company (publicly traded, privately held, etc.)" }, "revenue": { "type": ["string", "null"], "description": "Company revenue range" }, "description": { "type": ["string", "null"], "description": "Company description" }, "whois": { "type": ["object", "null"], "description": "Domain WHOIS information", "properties": { "registrar_name": { "type": ["string", "null"], "description": "Domain registrar" }, "created_date": { "type": ["string", "null"], "description": "Domain creation date" } } } } }, "emails": { "type": ["array", "null"], "description": "Array of email addresses found for the domain", "items": { "type": "object", "properties": { "email": { "type": ["string", "null"], "description": "Email address" }, "first_name": { "type": ["string", "null"], "description": "First name of the person" }, "last_name": { "type": ["string", "null"], "description": "Last name of the person" }, "position": { "type": ["string", "null"], "description": "Job position" }, "department": { "type": ["string", "null"], "description": "Department" }, "full_name": { "type": ["string", "null"], "description": "Full name of the person" }, "gender": { "type": ["string", "null"], "description": "Gender of the person" }, "phone_number": { "type": ["boolean", "string", "null"], "description": "Whether Tomba has a phone number for this person (flag only; the numbers themselves are in phone_data when enrichMobile is on)" }, "phone_data": { "type": ["array", "null"], "description": "The person's phone numbers (only returned when enrichMobile is on)", "items": { "type": "object" } }, "type": { "type": ["string", "null"], "description": "Email type (personal, generic, etc.)" }, "country": { "type": ["string", "null"], "description": "Country of the person" }, "seniority": { "type": ["string", "null"], "description": "Seniority level" }, "twitter": { "type": ["string", "null"], "description": "Twitter profile URL" }, "linkedin": { "type": ["string", "null"], "description": "LinkedIn profile URL" }, "score": { "type": ["number", "null"], "description": "Confidence score for the email" }, "verification": { "type": ["object", "null"], "description": "Email verification status", "properties": { "date": { "type": ["string", "null"], "description": "Verification date" }, "status": { "type": ["string", "null"], "description": "Verification status (valid, invalid, etc.)" } } }, "sources": { "type": ["array", "null"], "description": "Sources where the email was found", "items": { "type": "object", "properties": { "uri": { "type": ["string", "null"], "description": "Source URI" }, "website_url": { "type": ["string", "null"], "description": "Website URL where found" }, "extracted_on": { "type": ["string", "null"], "description": "Extraction date" }, "last_seen_on": { "type": ["string", "null"], "description": "Last seen date" }, "still_on_page": { "type": ["boolean", "null"], "description": "Whether still on page" } } } } } } }, "phoneNumbers": { "type": "integer", "description": "Total phone numbers returned across all emails (emails[].phone_data); only filled when enrichMobile is on" }, "meta": { "type": ["object", "null"], "description": "Metadata about the search results", "properties": { "total": { "type": ["number", "null"], "description": "Total number of emails found" }, "params": { "type": ["object", "null"], "description": "Search parameters used" } } }, "pages": { "type": "integer", "description": "Number of page requests that returned results for this domain" }, "chargedRequests": { "type": "integer", "description": "Number of page requests charged for this domain (pay-per-event)" }, "chargedCredits": { "type": "integer", "description": "Credits billed for this search ($0.00312 each): 1 per 10 results requested with limit (rounded up) per page, plus 5 per returned address with phone data (enrichMobile only)" }, "charged": { "type": "boolean", "description": "Whether this request was charged (pay-per-event)" }, "cached": { "type": "boolean", "description": "Whether the result came from the cache (free)" }, "error": { "type": ["string", "null"], "description": "Error message if the search failed" } } }, "views": { "overview": { "title": "Domain Search Results", "description": "Overview of email addresses found for each domain", "transformation": { "fields": [ "company", "domain", "organization.website_url", "organization.organization", "organization.location.country", "organization.location.city", "organization.location.state", "organization.industries", "organization.founded", "organization.company_size", "organization.company_type", "organization.revenue", "organization.phone_number", "organization.description", "organization.social_links.linkedin_url", "organization.social_links.twitter_url", "emails", "phoneNumbers", "meta.total", "meta.params", "pages", "chargedRequests", "chargedCredits", "charged", "cached", "error" ], "flatten": ["organization", "organization.location", "organization.social_links", "meta"] }, "display": { "component": "table", "properties": { "company": { "label": "Company", "format": "text" }, "domain": { "label": "Domain", "format": "text" }, "organization.website_url": { "label": "Website", "format": "url" }, "organization.organization": { "label": "Company Name", "format": "text" }, "organization.location.country": { "label": "Country", "format": "text" }, "organization.location.city": { "label": "City", "format": "text" }, "organization.location.state": { "label": "State", "format": "text" }, "organization.industries": { "label": "Industry", "format": "text" }, "organization.founded": { "label": "Founded", "format": "text" }, "organization.company_size": { "label": "Company Size", "format": "text" }, "organization.company_type": { "label": "Company Type", "format": "text" }, "organization.revenue": { "label": "Revenue", "format": "text" }, "organization.phone_number": { "label": "Phone", "format": "text" }, "organization.description": { "label": "Description", "format": "text" }, "organization.social_links.linkedin_url": { "label": "LinkedIn", "format": "url" }, "organization.social_links.twitter_url": { "label": "Twitter", "format": "url" }, "emails": { "label": "Email Results", "format": "array" }, "phoneNumbers": { "label": "Phone numbers", "format": "number" }, "meta.total": { "label": "Total Found", "format": "number" }, "meta.params": { "label": "Search Parameters", "format": "object" }, "pages": { "label": "Pages", "format": "number" }, "chargedRequests": { "label": "Charged Requests", "format": "number" }, "chargedCredits": { "label": "Credits charged", "format": "number" }, "charged": { "label": "Charged", "format": "boolean" }, "cached": { "label": "Cached", "format": "boolean" }, "error": { "label": "Error", "format": "text" } } } }, "emails": { "title": "Individual Emails", "description": "Detailed view of individual email addresses", "transformation": { "fields": ["company", "domain", "emails"], "unwind": ["emails"] }, "display": { "component": "table", "properties": { "company": { "label": "Company", "format": "text" }, "domain": { "label": "Domain", "format": "text" }, "email": { "label": "Email", "format": "text" }, "first_name": { "label": "First Name", "format": "text" }, "last_name": { "label": "Last Name", "format": "text" }, "full_name": { "label": "Full Name", "format": "text" }, "position": { "label": "Position", "format": "text" }, "department": { "label": "Department", "format": "text" }, "seniority": { "label": "Seniority", "format": "text" }, "type": { "label": "Email Type", "format": "text" }, "country": { "label": "Person Country", "format": "text" }, "phone_data": { "label": "Phone numbers", "format": "array" }, "linkedin": { "label": "LinkedIn", "format": "url" }, "score": { "label": "Confidence Score", "format": "number" }, "verification": { "label": "Verification", "format": "object" }, "sources": { "label": "Sources", "format": "array" } } } }, "company": { "title": "Company Information", "description": "Detailed company information for each domain", "transformation": { "fields": [ "company", "domain", "organization.website_url", "organization.organization", "organization.location.country", "organization.location.city", "organization.location.state", "organization.location.street_address", "organization.location.postal_code", "organization.industries", "organization.founded", "organization.company_size", "organization.company_type", "organization.revenue", "organization.phone_number", "organization.description", "organization.social_links.linkedin_url", "organization.social_links.twitter_url", "organization.social_links.facebook_url", "organization.whois.registrar_name", "organization.whois.created_date" ], "flatten": ["organization", "organization.location", "organization.social_links", "organization.whois"] }, "display": { "component": "table", "properties": { "company": { "label": "Company", "format": "text" }, "domain": { "label": "Domain", "format": "text" }, "organization.website_url": { "label": "Website", "format": "url" }, "organization.organization": { "label": "Company Name", "format": "text" }, "organization.description": { "label": "Description", "format": "text" }, "organization.industries": { "label": "Industry", "format": "text" }, "organization.founded": { "label": "Founded", "format": "text" }, "organization.company_size": { "label": "Company Size", "format": "text" }, "organization.company_type": { "label": "Company Type", "format": "text" }, "organization.revenue": { "label": "Revenue", "format": "text" }, "organization.location.country": { "label": "Country", "format": "text" }, "organization.location.city": { "label": "City", "format": "text" }, "organization.location.state": { "label": "State/Province", "format": "text" }, "organization.location.street_address": { "label": "Address", "format": "text" }, "organization.location.postal_code": { "label": "Postal Code", "format": "text" }, "organization.phone_number": { "label": "Phone", "format": "text" }, "organization.social_links.linkedin_url": { "label": "LinkedIn", "format": "url" }, "organization.social_links.twitter_url": { "label": "Twitter", "format": "url" }, "organization.social_links.facebook_url": { "label": "Facebook", "format": "url" }, "organization.whois.registrar_name": { "label": "Domain Registrar", "format": "text" }, "organization.whois.created_date": { "label": "Domain Created", "format": "text" } } } } }}{ "openapi": "3.0.0", "info": { "title": "Tomba Domain Search API", "version": "1.0.0", "description": "Real-time API (Apify Standby mode). Find the email addresses (and optionally phone numbers) of a company by domain or company name. One credit costs $0.00312: each page costs 1 credit per 10 results requested with `limit` (10 = 1 credit, 20 = 2, 50 = 5), plus 5 credits for each returned address with phone data when `enrichMobile` is on. Unknown domains, errors and cached results are free." }, "paths": { "/": { "get": { "summary": "Search emails for domains or company names", "description": "Without query parameters, returns a short usage note (also used as the readiness probe).", "parameters": [ { "name": "domain", "in": "query", "description": "Domain to search. Repeat the parameter or separate values with commas for several domains. URLs are cleaned up. Provide `domain`, `company`, or both.", "required": false, "schema": { "type": "string" }, "example": "stripe.com" }, { "name": "domains", "in": "query", "description": "Alias of `domain`.", "required": false, "schema": { "type": "string" } }, { "name": "company", "in": "query", "description": "Company name to search instead of a domain (Tomba resolves it to its domain). Repeat the parameter or separate values with commas for several companies.", "required": false, "schema": { "type": "string" }, "example": "Stripe" }, { "name": "companies", "in": "query", "description": "Alias of `company`.", "required": false, "schema": { "type": "string" } }, { "name": "maxEmailsPerDomain", "in": "query", "description": "Maximum number of emails per domain or company (default 10). The Actor fetches as many pages as needed.", "required": false, "schema": { "type": "integer", "minimum": 1, "maximum": 100, "default": 10 }, "example": 10 }, { "name": "includeCompanyInfo", "in": "query", "description": "Include the company profile (`organization`) in each result (default true).", "required": false, "schema": { "type": "boolean", "default": true } }, { "name": "page", "in": "query", "description": "First page to fetch (default 1).", "required": false, "schema": { "type": "integer", "minimum": 1, "default": 1 } }, { "name": "limit", "in": "query", "description": "Emails per page request: 10, 20 or 50. Costs 1, 2 or 5 credits per page.", "required": false, "schema": { "type": "string", "enum": ["10", "20", "50"], "default": "10" } }, { "name": "department", "in": "query", "description": "Only return people working in this department.", "required": false, "schema": { "type": "string", "enum": [ "accounting", "administrative", "communication", "diversity", "education", "engineering", "executive", "facilities", "finance", "healthcare", "hr", "it", "legal", "management", "marketing", "operations", "pr", "sales", "security", "software", "support", "warehouse" ] }, "example": "engineering" }, { "name": "country", "in": "query", "description": "Two-letter country code to filter email addresses.", "required": false, "schema": { "type": "string", "pattern": "^[A-Z]{2}$" }, "example": "US" }, { "name": "enrichMobile", "in": "query", "description": "Also return each person's phone numbers in `emails[].phone_data` (+5 credits per returned address with phone data).", "required": false, "schema": { "type": "boolean", "default": false } }, { "name": "webhookUrl", "in": "query", "description": "Optional URL that Tomba also POSTs each search result to.", "required": false, "schema": { "type": "string", "pattern": "^https?://" } } ], "responses": { "200": { "$ref": "#/components/responses/Results" }, "400": { "$ref": "#/components/responses/Error" }, "402": { "$ref": "#/components/responses/Error" } } }, "post": { "summary": "Search emails using a JSON input", "description": "Accepts the same JSON input as a normal Actor run. Provide `domains`, `companies`, or both.", "requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "properties": { "domains": { "type": "array", "description": "Company domains to search for email addresses, e.g. stripe.com. URLs are cleaned up and duplicates removed. Provide domains, company names, or both.", "items": { "type": "string" }, "example": ["stripe.com", "tomba.io"] }, "companies": { "type": "array", "description": "Company names to search instead of a domain, e.g. Stripe. Tomba resolves the company to its domain. Provide domains, company names, or both.", "items": { "type": "string" } }, "maxEmailsPerDomain": { "type": "integer", "description": "Maximum number of emails to find per domain (applies to domain search)", "minimum": 1, "maximum": 100, "default": 10 }, "includeCompanyInfo": { "type": "boolean", "description": "Include additional company information when available", "default": true }, "outputFormat": { "type": "string", "description": "Format of the output data", "default": "detailed", "enum": ["detailed", "simple"] }, "page": { "type": "integer", "description": "Page number for pagination", "minimum": 1, "default": 1 }, "limit": { "type": "string", "description": "Specifies the max number of email addresses to return", "default": "10", "enum": ["10", "20", "50"] }, "department": { "type": "string", "description": "Get only email addresses for people working in the selected department", "enum": [ "accounting", "administrative", "communication", "diversity", "education", "engineering", "executive", "facilities", "finance", "healthcare", "hr", "it", "legal", "management", "marketing", "operations", "pr", "sales", "security", "software", "support", "warehouse" ] }, "country": { "type": "string", "description": "Two-letter country code to filter email addresses (e.g., 'US', 'UK', 'CA')", "pattern": "^[A-Z]{2}$" }, "enrichMobile": { "type": "boolean", "description": "Also return each person's phone numbers (in emails[].phone_data). Costs extra: 5 additional credits ($0.0156) for every returned address that comes with phone data. Addresses without phone data cost nothing extra.", "default": false }, "webhookUrl": { "type": "string", "description": "Optional URL that Tomba also POSTs each search result to. Results served from the cache are not sent again.", "pattern": "^https?://" }, "maxConcurrency": { "type": "integer", "description": "How many Tomba requests run in parallel. There is no rate limit, so raise it for large lists.", "minimum": 1, "maximum": 50, "default": 10 }, "maxRetries": { "type": "integer", "description": "How many times to retry a request after a network error, 429 or 5xx response (exponential backoff). Failed requests are never charged.", "minimum": 0, "maximum": 10, "default": 3 }, "useCache": { "type": "boolean", "description": "Reuse results from previous runs for identical requests. Cached results are free.", "default": true }, "cacheTtlHours": { "type": "integer", "description": "How long cached results stay valid. 0 disables the cross-run cache.", "minimum": 0, "maximum": 720, "default": 24 } } } } } }, "responses": { "200": { "$ref": "#/components/responses/Results" }, "400": { "$ref": "#/components/responses/Error" }, "402": { "$ref": "#/components/responses/Error" } } } } }, "components": { "responses": { "Results": { "description": "One row per domain or company, in the same format as the dataset items.", "content": { "application/json": { "schema": { "type": "object", "properties": { "items": { "type": "array", "items": { "$ref": "#/components/schemas/DomainSearchResult" } } } } } } }, "Error": { "description": "Invalid input (400) or the run's max charge limit was reached (402).", "content": { "application/json": { "schema": { "type": "object", "properties": { "error": { "type": "string" } } } } } } }, "schemas": { "DomainSearchResult": { "type": "object", "properties": { "company": { "type": "string", "description": "Company name searched (company searches only)" }, "domain": { "type": "string", "description": "Domain searched, or the domain Tomba resolved for a company" }, "organization": { "type": "object", "description": "Company profile (when includeCompanyInfo is true)", "additionalProperties": true }, "emails": { "type": "array", "items": { "type": "object", "properties": { "email": { "type": "string" }, "first_name": { "type": "string", "nullable": true }, "last_name": { "type": "string", "nullable": true }, "full_name": { "type": "string", "nullable": true }, "position": { "type": "string", "nullable": true }, "department": { "type": "string", "nullable": true }, "seniority": { "type": "string", "nullable": true }, "type": { "type": "string" }, "country": { "type": "string", "nullable": true }, "linkedin": { "type": "string", "nullable": true }, "phone_number": { "type": "boolean" }, "phone_data": { "type": "array", "items": { "type": "object", "additionalProperties": true } }, "score": { "type": "integer" }, "verification": { "type": "object", "additionalProperties": true }, "sources": { "type": "array", "items": { "type": "object", "additionalProperties": true } } }, "additionalProperties": true } }, "phoneNumbers": { "type": "integer" }, "meta": { "type": "object", "additionalProperties": true }, "pages": { "type": "integer" }, "chargedRequests": { "type": "integer" }, "chargedCredits": { "type": "integer" }, "charged": { "type": "boolean" }, "cached": { "type": "boolean" }, "error": { "type": "string" } } } } }}{ "actorOutputSchemaVersion": 1, "title": "Domain Search Output", "description": "Output from Tomba Domain Search Actor - email addresses discovered within specific domains", "type": "object", "properties": { "results": { "type": "string", "title": "Domain Search Results", "description": "Email addresses found within the domain with contact information", "template": "{{links.apiDefaultDatasetUrl}}/items" } }}{ "actorSpecification": 1, "name": "domain-search", "title": "Tomba Domain Search", "description": "An Apify Actor that uses Tomba API to find email addresses (and, optionally, phone numbers) associated with given domains or company names.", "version": "1.0", "buildTag": "latest", "meta": { "templateId": "ts-empty" }, "dockerfile": "../Dockerfile", "input": "./input_schema.json", "output": "./output_schema.json", "storages": { "dataset": "./dataset_schema.json", "keyValueStore": "./key_value_store_schema.json" }, "environmentVariables": { "TOMBA_API_KEY": "@tombaApiKey", "TOMBA_API_SECRET": "@tombaApiSecret" }, "usesStandbyMode": true, "webServerSchema": "./web_server_schema.json", "defaultMemoryMbytes": 256}{ "actorKeyValueStoreSchemaVersion": 1, "title": "Tomba Domain Search storage", "description": "Records the Actor keeps in its default key-value store. Results are in the dataset.", "collections": { "input": { "title": "Input", "description": "The input of this run.", "key": "INPUT", "contentTypes": ["application/json"] }, "progress": { "title": "Progress", "description": "Domains and company names already searched, so a migrated or restarted run continues without charging again.", "key": "TOMBA_STATE", "contentTypes": ["application/json"], "jsonSchema": { "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "done": { "type": "object", "description": "Processed searches (a domain, or `company:<lowercased name>`) mapped to true", "additionalProperties": { "type": "boolean" } } }, "required": ["done"] } }, "summary": { "title": "Summary", "description": "Totals and settings of the run, written when a batch run finishes.", "key": "SUMMARY", "contentTypes": ["application/json"], "jsonSchema": { "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "totalDomains": { "type": "integer", "description": "Unique domains in the input" }, "totalCompanies": { "type": "integer", "description": "Unique company names in the input" }, "tombaRequests": { "type": "integer", "description": "Requests sent to Tomba (cache hits excluded)" }, "chargedRequests": { "type": "integer", "description": "Charged events (credits)" }, "cacheHits": { "type": "integer", "description": "Pages served from the cache" }, "processedAt": { "type": "string", "format": "date-time" }, "settings": { "type": "object", "description": "Search settings used for the run", "properties": { "maxEmailsPerDomain": { "type": "integer" }, "includeCompanyInfo": { "type": "boolean" }, "outputFormat": { "type": "string" }, "page": { "type": "integer" }, "limit": { "type": "string" }, "department": { "type": "string" }, "country": { "type": "string" }, "enrichMobile": { "type": "boolean" }, "webhookUrl": { "type": "string" } } } }, "required": ["totalDomains", "totalCompanies", "processedAt"] } } }}{ "title": "Tomba Domain Search Input", "type": "object", "schemaVersion": 1, "properties": { "domains": { "title": "Domains", "type": "array", "description": "Company domains to search for email addresses, e.g. stripe.com. URLs are cleaned up and duplicates removed. Provide domains, company names, or both.", "items": { "type": "string" }, "editor": "stringList", "prefill": ["stripe.com", "tomba.io"] }, "companies": { "title": "Company names", "type": "array", "description": "Company names to search instead of a domain, e.g. Stripe. Tomba resolves the company to its domain. Provide domains, company names, or both.", "items": { "type": "string" }, "editor": "stringList" }, "maxEmailsPerDomain": { "title": "Max Emails Per Domain", "type": "integer", "description": "Maximum number of emails to find per domain (applies to domain search)", "minimum": 1, "maximum": 100, "default": 10 }, "includeCompanyInfo": { "title": "Include Company Info", "type": "boolean", "description": "Include additional company information when available", "default": true }, "outputFormat": { "title": "Output Format", "type": "string", "description": "Format of the output data", "enum": ["detailed", "simple"], "enumTitles": ["Detailed", "Simple"], "default": "detailed" }, "page": { "title": "Page Number", "type": "integer", "description": "Page number for pagination", "minimum": 1, "default": 1 }, "limit": { "title": "Results Limit", "type": "string", "description": "Specifies the max number of email addresses to return", "enum": ["10", "20", "50"], "enumTitles": ["10 emails", "20 emails", "50 emails"], "default": "10" }, "department": { "title": "Department Filter", "type": "string", "description": "Get only email addresses for people working in the selected department", "enum": [ "accounting", "administrative", "communication", "diversity", "education", "engineering", "executive", "facilities", "finance", "healthcare", "hr", "it", "legal", "management", "marketing", "operations", "pr", "sales", "security", "software", "support", "warehouse" ], "enumTitles": [ "Accounting", "Administrative", "Communication", "Diversity", "Education", "Engineering", "Executive", "Facilities", "Finance", "Healthcare", "HR", "IT", "Legal", "Management", "Marketing", "Operations", "PR", "Sales", "Security", "Software", "Support", "Warehouse" ] }, "country": { "title": "Country Filter", "type": "string", "description": "Two-letter country code to filter email addresses (e.g. 'US', 'GB', 'CA'). Lowercase is accepted.", "maxLength": 2, "minLength": 2, "pattern": "^[A-Za-z]{2}$", "editor": "textfield" }, "enrichMobile": { "title": "Enrich with mobile phone numbers", "type": "boolean", "description": "Also return each person's phone numbers (in emails[].phone_data). Costs extra: 5 additional credits ($0.0156) for every returned address that comes with phone data. Addresses without phone data cost nothing extra.", "default": false, "editor": "checkbox" }, "webhookUrl": { "title": "Webhook URL", "type": "string", "description": "Optional URL that Tomba also POSTs each search result to. Results served from the cache are not sent again.", "pattern": "^https?://", "editor": "textfield" }, "maxConcurrency": { "title": "Max concurrency", "type": "integer", "description": "How many Tomba requests run in parallel. There is no rate limit, so raise it for large lists.", "minimum": 1, "maximum": 50, "default": 10, "editor": "number", "sectionCaption": "Run options", "sectionDescription": "Concurrency, retry and cache settings. The defaults work well for most runs." }, "maxRetries": { "title": "Max retries", "type": "integer", "description": "How many times to retry a request after a network error, 429 or 5xx response (exponential backoff). Failed requests are never charged.", "minimum": 0, "maximum": 10, "default": 3, "editor": "number" }, "useCache": { "title": "Use cache", "type": "boolean", "description": "Reuse results from previous runs for identical requests. Cached results are free.", "default": true, "editor": "checkbox" }, "cacheTtlHours": { "title": "Cache TTL (hours)", "type": "integer", "description": "How long cached results stay valid. 0 disables the cross-run cache.", "minimum": 0, "maximum": 720, "default": 24, "editor": "number", "unit": "hours" } }}1// Shared runner for the Tomba Apify Actors: batch runs and Standby (real-time HTTP API) mode.2// Keep this file identical across all Actors.3import type { IncomingMessage, ServerResponse } from 'node:http';4import { createServer } from 'node:http';5
6import { Actor, log } from 'apify';7
8import type { RunOptions } from './tomba.js';9import { isStopped, logSummary, setupTomba, useRunState } from './tomba.js';10
11/** Invalid input: a failed batch run, or `400 Bad Request` in Standby mode. */12export class InputError extends Error {}13
14export type Item = Record<string, unknown>;15
16export interface RunContext {17 /** Save result items (dataset in batch runs, HTTP response in Standby mode). */18 push: (items: Item | Item[]) => Promise<void>;19 /** True when this input was already processed (batch runs resume after a migration or restart). */20 isDone: (key: string) => boolean;21 /** Record a processed input so a resumed run skips it. */22 markDone: (key: string) => void;23 standby: boolean;24}25
26/** Processes one Actor input. Throws InputError for invalid input. */27export type ActorRun<I> = (input: I, ctx: RunContext) => Promise<void>;28
29export interface ActorDefinition<I> {30 /** Name used in logs. */31 title: string;32 run: ActorRun<I>;33 /** Builds the Actor input from `GET /?…` query parameters in Standby mode. */34 fromQuery: (query: URLSearchParams) => I;35 /** Number of inputs, for the batch summary log. */36 count?: (input: I) => number;37}38
39const MAX_BODY_BYTES = 1_000_000;40
41export function isStandby(): boolean {42 return Actor.config.get('metaOrigin') === 'STANDBY';43}44
45/** Entry point: runs the Actor as a batch job, or as an HTTP server in Standby mode. */46export async function runActor<I extends RunOptions>(definition: ActorDefinition<I>): Promise<void> {47 await Actor.init();48
49 if (isStandby()) {50 await setupTomba();51 await startStandbyServer(definition);52 return;53 }54
55 const input = ((await Actor.getInput<I>()) ?? {}) as I;56 await setupTomba(input);57 const state = await useRunState();58 const startedAt = Date.now();59 try {60 await definition.run(input, {61 push: async (items) => Actor.pushData(items),62 isDone: (key) => Boolean(state.done[key]),63 markDone: (key) => {64 state.done[key] = true;65 },66 standby: false,67 });68 } catch (err) {69 if (err instanceof InputError) {70 await Actor.fail(err.message);71 return;72 }73 throw err;74 }75 logSummary(definition.title, definition.count?.(input) ?? 0, startedAt);76 await Actor.exit();77}78
79async function startStandbyServer<I>(definition: ActorDefinition<I>): Promise<void> {80 const server = createServer((req, res) => {81 handleRequest(definition, req, res).catch((err: Error) => {82 log.exception(err, 'Standby request failed');83 send(res, 500, { error: 'Internal error' });84 });85 });86 const port = Actor.config.get('containerPort');87 await new Promise<void>((resolve) => {88 server.listen(port, resolve);89 });90 log.info(`${definition.title} is ready for real-time requests`, { port });91}92
93async function handleRequest<I>(definition: ActorDefinition<I>, req: IncomingMessage, res: ServerResponse) {94 if (req.headers['x-apify-container-server-readiness-probe']) {95 send(res, 200, { status: 'ready' });96 return;97 }98
99 const url = new URL(req.url ?? '/', 'http://localhost');100 if (url.pathname !== '/') {101 send(res, 404, { error: 'Not found. Use GET / with query parameters or POST / with a JSON input.' });102 return;103 }104
105 let input: I;106 try {107 if (req.method === 'GET') {108 if ([...url.searchParams.keys()].length === 0) {109 send(res, 200, {110 status: 'ready',111 usage: 'GET /?<parameters> or POST / with the same JSON input as a normal run.',112 });113 return;114 }115 input = definition.fromQuery(url.searchParams);116 } else if (req.method === 'POST') {117 input = (await readJson(req)) as I;118 } else {119 send(res, 405, { error: 'Method not allowed. Use GET or POST.' });120 return;121 }122 } catch (err) {123 send(res, 400, { error: (err as Error).message });124 return;125 }126
127 if (isStopped()) {128 send(res, 402, { error: 'Max charge limit reached for this Actor run.' });129 return;130 }131
132 const items: Item[] = [];133 try {134 await definition.run(input, {135 push: async (pushed) => {136 items.push(...(Array.isArray(pushed) ? pushed : [pushed]));137 },138 isDone: () => false,139 markDone: () => undefined,140 standby: true,141 });142 } catch (err) {143 if (err instanceof InputError) {144 send(res, 400, { error: err.message });145 return;146 }147 throw err;148 }149 send(res, 200, { items });150}151
152async function readJson(req: IncomingMessage): Promise<unknown> {153 const chunks: Buffer[] = [];154 let size = 0;155 for await (const chunk of req) {156 size += (chunk as Buffer).length;157 if (size > MAX_BODY_BYTES) throw new InputError('Request body is too large (max 1 MB).');158 chunks.push(chunk as Buffer);159 }160 const text = Buffer.concat(chunks).toString('utf8').trim();161 if (!text) return {};162 try {163 const body = JSON.parse(text);164 if (typeof body !== 'object' || body === null || Array.isArray(body)) {165 throw new InputError('Request body must be a JSON object.');166 }167 return body;168 } catch (err) {169 if (err instanceof InputError) throw err;170 throw new InputError('Request body must be valid JSON.');171 }172}173
174function send(res: ServerResponse, status: number, body: unknown): void {175 if (res.headersSent) return;176 res.writeHead(status, { 'content-type': 'application/json; charset=utf-8' });177 res.end(JSON.stringify(body));178}179
180// --- Query string helpers for `fromQuery` ---------------------------------------------------181
182/** Values of a list parameter: repeated (`?domain=a&domain=b`) and/or comma-separated (`?domains=a,b`). */183export function queryList(query: URLSearchParams, ...names: string[]): string[] {184 return names185 .flatMap((name) => query.getAll(name))186 .flatMap((value) => value.split(','))187 .map((value) => value.trim())188 .filter(Boolean);189}190
191export function queryString(query: URLSearchParams, name: string): string | undefined {192 const value = query.get(name)?.trim();193 return value || undefined;194}195
196export function queryBool(query: URLSearchParams, name: string): boolean | undefined {197 const value = query.get(name)?.trim().toLowerCase();198 if (value === undefined || value === '') return undefined;199 if (['1', 'true', 'yes'].includes(value)) return true;200 if (['0', 'false', 'no'].includes(value)) return false;201 throw new InputError(`Query parameter "${name}" must be true or false.`);202}203
204export function queryInt(query: URLSearchParams, name: string): number | undefined {205 const value = query.get(name)?.trim();206 if (value === undefined || value === '') return undefined;207 const number = Number(value);208 if (!Number.isInteger(number)) throw new InputError(`Query parameter "${name}" must be an integer.`);209 return number;210}1// Shared Tomba helper for the Tomba Apify Actors.2// Keep this file identical across all Actors.3import { createHash } from 'node:crypto';4
5import type { KeyValueStore } from 'apify';6import { Actor, log } from 'apify';7import { TombaClient, TombaException } from 'tomba';8
9/** Pay-per-event event names (configured in Apify Console → Monetization). */10export const EVENT_REQUEST = 'tomba-request';11export const EVENT_PHONE_FINDER = 'phone-finder-request';12
13/**14 * Tomba credits added per phone number (or per address with phone data, depending on the endpoint)15 * returned when `enrich_mobile=true`. Every credit is charged as one `tomba-request` event.16 */17export const PHONE_CREDITS = 5;18
19/** Number of phone numbers in a record's `phone_data` array (0 when missing or empty). */20export function phoneDataCount(record: unknown): number {21 if (!isObject(record)) return 0;22 return Array.isArray(record.phone_data) ? record.phone_data.length : 0;23}24
25/** True when a record has at least one phone number in `phone_data`. */26export function hasPhoneData(record: unknown): boolean {27 return phoneDataCount(record) > 0;28}29
30export interface RunOptions {31 maxConcurrency?: number;32 maxRetries?: number;33 useCache?: boolean;34 cacheTtlHours?: number;35}36
37export interface TombaCallResult {38 /** Raw JSON body returned by Tomba (`{ data, meta, ... }`). */39 body?: Record<string, unknown>;40 /** Shortcut to `body.data`. */41 data?: unknown;42 /** HTTP status (0 for network errors, 502 for a non-JSON body). */43 status: number;44 charged: boolean;45 /** Number of events charged for this call (0 when not charged). */46 chargedCount?: number;47 cached: boolean;48 /** True when the call was not made because the user's max charge limit was reached. */49 skipped?: boolean;50 error?: string;51}52
53export const stats = {54 requests: 0,55 charged: 0,56 cached: 0,57 failed: 0,58 retries: 0,59};60
61const CACHE_STORE_PREFIX = 'tomba-cache';62const BASE_DELAY_MS = 1000;63const MAX_DELAY_MS = 30000;64const NETWORK_ERROR = /ECONN|ETIMEDOUT|ENOTFOUND|EAI_AGAIN|EPIPE|socket hang up|timeout|network/i;65
66let client: TombaClient | undefined;67let options: Required<RunOptions> = { maxConcurrency: 10, maxRetries: 3, useCache: true, cacheTtlHours: 24 };68let cacheStore: KeyValueStore | undefined;69const memoryCache = new Map<string, Record<string, unknown>>();70const inFlight = new Map<string, number>();71let stopped = false;72
73/** Initialise the Tomba client from the Actor's environment variables and apply the run options. */74export async function setupTomba(runOptions: RunOptions = {}): Promise<TombaClient> {75 const key = process.env.TOMBA_API_KEY;76 const secret = process.env.TOMBA_API_SECRET;77 if (!key || !secret) {78 await Actor.fail('Actor is misconfigured: Tomba credentials are missing. Please contact the Actor developer.');79 throw new Error('unreachable');80 }81
82 options = {83 maxConcurrency: clamp(runOptions.maxConcurrency ?? 10, 1, 50),84 maxRetries: clamp(runOptions.maxRetries ?? 3, 0, 10),85 useCache: runOptions.useCache ?? true,86 cacheTtlHours: Math.max(0, runOptions.cacheTtlHours ?? 24),87 };88
89 client = new TombaClient();90 // Verification and enrichment can take over a minute, so keep the SDK's 120 s timeout.91 client.setKey(key).setSecret(secret).setTimeout(120_000);92 // Only used by the test suite to point the client at a local mock server.93 if (process.env.TOMBA_API_ENDPOINT) client.setEndpoint(process.env.TOMBA_API_ENDPOINT);94
95 if (options.useCache && options.cacheTtlHours > 0) {96 try {97 cacheStore = await Actor.openKeyValueStore(cacheStoreName());98 } catch (err) {99 // E.g. "Insufficient permissions" under limited permissions: never fail the run because of the cache.100 log.warning('Cross-run cache is unavailable; results are cached for this run only.', {101 error: (err as Error).message,102 });103 cacheStore = undefined;104 }105 }106
107 return client;108}109
110/**111 * Name of the cross-run cache store. It is per Actor: with limited permissions an Actor can only open112 * named storages it created itself, so the Tomba Actors must not share one store.113 */114export function cacheStoreName(): string {115 const actorId = Actor.getEnv().actorId ?? process.env.ACTOR_ID;116 return actorId ? `${CACHE_STORE_PREFIX}-${actorId}` : CACHE_STORE_PREFIX;117}118
119/** The Tomba client created by setupTomba(). */120export function getClient(): TombaClient {121 if (!client) throw new Error('setupTomba() must be called first');122 return client;123}124
125export function getOptions(): Required<RunOptions> {126 return options;127}128
129/** True once the user's max-charge budget is exhausted; workers should stop picking up new items. */130export function isStopped(): boolean {131 return stopped;132}133
134export function stop(): void {135 stopped = true;136}137
138/**139 * Tomba billing rules: charge only when Tomba returns a JSON body without an `errors` object140 * and with a non-empty `data` (a negative answer such as an undeliverable verification141 * or a domain search with no emails is still a non-empty `data` object).142 */143export function isBillable(body: unknown): boolean {144 if (!isObject(body)) return false;145 if (body.errors !== undefined && body.errors !== null) return false;146 const { data } = body;147 if (data === null || data === undefined) return false;148 if (Array.isArray(data)) return data.length > 0;149 // An object whose values are all null (e.g. `{ person: null, company: null }`) is an empty result.150 if (isObject(data)) return Object.values(data).some((v) => v !== null && v !== undefined);151 return data !== '' && data !== false;152}153
154/**155 * Perform one Tomba API call with caching, retry and pay-per-event charging.156 *157 * @param endpoint short endpoint name used for the cache key (e.g. 'technology')158 * @param params request parameters used for the cache key159 * @param fn the SDK call160 * @param eventName pay-per-event name to charge when the response is billable161 * @param count events to charge for a billable response: a number, or a function of the response body162 */163export async function callTomba(164 endpoint: string,165 params: Record<string, unknown>,166 fn: (tomba: TombaClient) => Promise<unknown>,167 eventName: string = EVENT_REQUEST,168 count: number | ((body: Record<string, unknown>) => number) = 1,169): Promise<TombaCallResult> {170 if (!client) throw new Error('setupTomba() must be called first');171
172 const cacheKey = hashKey(endpoint, params);173 const cachedBody = await readCache(cacheKey);174 if (cachedBody) {175 stats.cached++;176 return { body: cachedBody, data: cachedBody.data, status: 200, charged: false, cached: true };177 }178
179 // Reserve budget before calling Tomba so concurrent workers never exceed the user's max charge.180 // A fixed count is known up front, so reserve all of it; a body-dependent count reserves at least 1.181 const reserved = typeof count === 'number' ? Math.max(1, Math.ceil(count)) : 1;182 if (!reserve(eventName, reserved)) {183 stop();184 return { status: 0, charged: false, cached: false, skipped: true, error: 'Max charge limit reached' };185 }186
187 try {188 for (let attempt = 0; ; attempt++) {189 try {190 stats.requests++;191 const body = unwrap(await fn(client));192
193 if (!isObject(body)) {194 stats.failed++;195 return { status: 502, charged: false, cached: false, error: 'Invalid response from Tomba' };196 }197
198 if (!isBillable(body)) {199 const error = errorMessage(body) ?? undefined;200 return { body, data: body.data, status: 200, charged: false, cached: false, error };201 }202
203 const units = Math.max(1, Math.ceil(typeof count === 'function' ? count(body) : count));204 const chargedCount = await charge(eventName, units);205 const charged = chargedCount > 0;206 await writeCache(cacheKey, body);207 return { body, data: body.data, status: 200, charged, chargedCount, cached: false };208 } catch (err) {209 const { status, message, retryAfter } = parseError(err);210 const retryable = status === 0 || status === 429 || status >= 500;211 if (retryable && attempt < options.maxRetries) {212 stats.retries++;213 const delay = retryAfter ?? Math.min(MAX_DELAY_MS, BASE_DELAY_MS * 2 ** attempt);214 const jitter = Math.random() * 250;215 log.warning(`Tomba ${endpoint} returned ${status || 'network error'}, retrying in ${delay}ms`, {216 attempt: attempt + 1,217 });218 await sleep(delay + jitter);219 continue;220 }221 stats.failed++;222 return { status, charged: false, cached: false, error: message };223 }224 }225 } finally {226 release(eventName, reserved);227 }228}229
230/**231 * Run `worker` over `items` with bounded concurrency. Stops picking new items once the charge limit232 * is reached (`isStopped()`) or `shouldStop()` returns true (e.g. maxResults reached for this run).233 */234export async function runPool<T>(235 items: T[],236 worker: (item: T, index: number) => Promise<void>,237 concurrency = options.maxConcurrency,238 shouldStop: () => boolean = () => false,239): Promise<void> {240 let next = 0;241 const runners = Array.from({ length: Math.min(concurrency, items.length) }, async () => {242 while (!stopped && !shouldStop() && next < items.length) {243 const index = next++;244 await worker(items[index], index);245 }246 });247 await Promise.all(runners);248}249
250/** Persisted run state used to resume after a migration, restart or resurrection. */251export async function useRunState(): Promise<{ done: Record<string, true> }> {252 return Actor.useState<{ done: Record<string, true> }>('TOMBA_STATE', { done: {} });253}254
255export function logSummary(title: string, total: number, startedAt: number): void {256 const seconds = Math.round((Date.now() - startedAt) / 1000);257 log.info(`${title} finished`, {258 inputs: total,259 tombaRequests: stats.requests,260 charged: stats.charged,261 cacheHits: stats.cached,262 failed: stats.failed,263 retries: stats.retries,264 seconds,265 stoppedByChargeLimit: stopped,266 });267}268
269export function normalizeDomain(value: string): string {270 return value271 .trim()272 .toLowerCase()273 .replace(/^https?:\/\//, '')274 .replace(/^www\./, '')275 .replace(/\/.*$/, '');276}277
278export function normalizeEmail(value: string): string {279 return value.trim().toLowerCase();280}281
282export function unique<T>(items: T[], key: (item: T) => string = String): T[] {283 const seen = new Set<string>();284 return items.filter((item) => {285 const k = key(item);286 if (!k || seen.has(k)) return false;287 seen.add(k);288 return true;289 });290}291
292// ---------------------------------------------------------------------------293
294async function charge(eventName: string, count: number): Promise<number> {295 const result = await Actor.charge({ eventName, count });296 if (result.chargedCount > 0) stats.charged += result.chargedCount;297 if (result.eventChargeLimitReached) {298 log.info('Max charge limit reached, finishing current items and stopping.');299 stop();300 }301 return result.chargedCount;302}303
304function reserve(eventName: string, units: number): boolean {305 if (stopped) return false;306 const manager = Actor.getChargingManager();307 const remaining = manager.calculateMaxEventChargeCountWithinLimit(eventName);308 const current = inFlight.get(eventName) ?? 0;309 if (remaining - current < units) return false;310 inFlight.set(eventName, current + units);311 return true;312}313
314function release(eventName: string, units: number): void {315 inFlight.set(eventName, Math.max(0, (inFlight.get(eventName) ?? units) - units));316}317
318async function readCache(key: string): Promise<Record<string, unknown> | undefined> {319 if (!options.useCache) return undefined;320 const hit = memoryCache.get(key);321 if (hit) return hit;322 if (!cacheStore) return undefined;323 const entry = await cacheStore.getValue<{ savedAt: number; body: Record<string, unknown> }>(key);324 if (!entry) return undefined;325 if (Date.now() - entry.savedAt > options.cacheTtlHours * 3600_000) return undefined;326 memoryCache.set(key, entry.body);327 return entry.body;328}329
330async function writeCache(key: string, body: Record<string, unknown>): Promise<void> {331 if (!options.useCache) return;332 memoryCache.set(key, body);333 if (cacheStore) {334 try {335 await cacheStore.setValue(key, { savedAt: Date.now(), body });336 } catch (err) {337 log.debug('Failed to write cache entry', { error: (err as Error).message });338 }339 }340}341
342function hashKey(endpoint: string, params: Record<string, unknown>): string {343 const sorted = Object.keys(params)344 .sort()345 .map((k) => [k, params[k]]);346 return `${endpoint}-${createHash('sha256').update(JSON.stringify(sorted)).digest('hex').slice(0, 40)}`;347}348
349/** tomba >= 1.1.0 resolves to `{ data: body, rateLimit }`; older versions resolve to the body itself. */350function unwrap(response: unknown): unknown {351 if (isObject(response) && 'rateLimit' in response && 'data' in response) return response.data;352 return response;353}354
355function parseError(err: unknown): { status: number; message: string; retryAfter?: number } {356 if (err instanceof TombaException) {357 const response = err.code as { status?: number; headers?: Record<string, string> } | undefined;358 // Without an HTTP response it is either a network error (retryable, status 0)359 // or the SDK's own input validation (not retryable, treated as 400).360 const networkError = NETWORK_ERROR.test(err.message ?? '');361 let status = networkError ? 0 : 400;362 if (typeof response?.status === 'number') status = response.status;363 const retryAfterHeader = Number(response?.headers?.['retry-after']);364 const retryAfter =365 Number.isFinite(retryAfterHeader) && retryAfterHeader > 0 ? retryAfterHeader * 1000 : undefined;366 const message = errorMessage(err.response) ?? err.message ?? 'Tomba request failed';367 return { status, message: status ? `${status}: ${message}` : message, retryAfter };368 }369 return { status: 0, message: err instanceof Error ? err.message : String(err) };370}371
372function errorMessage(body: unknown): string | undefined {373 if (!isObject(body)) return typeof body === 'string' && body.length < 300 ? body : undefined;374 const errors = body.errors as Record<string, unknown> | undefined;375 if (isObject(errors) && typeof errors.message === 'string') return errors.message;376 return undefined;377}378
379function isObject(value: unknown): value is Record<string, unknown> {380 return typeof value === 'object' && value !== null && !Array.isArray(value);381}382
383function clamp(value: number, min: number, max: number): number {384 return Math.min(max, Math.max(min, Math.floor(value)));385}386
387async function sleep(ms: number): Promise<void> {388 return new Promise((resolve) => {389 setTimeout(resolve, ms);390 });391}1import { Actor, log } from 'apify';2import { Domain } from 'tomba';3
4import { InputError, queryBool, queryInt, queryList, queryString, runActor } from './standby.js';5import type { RunOptions } from './tomba.js';6import {7 callTomba,8 EVENT_REQUEST,9 getClient,10 hasPhoneData,11 isBillable,12 normalizeDomain,13 PHONE_CREDITS,14 phoneDataCount,15 runPool,16 stats,17 unique,18} from './tomba.js';19
20/** Departments documented by Tomba (the SDK type lacks `education` and `healthcare`, the API accepts them). */21type Department =22 | 'accounting'23 | 'administrative'24 | 'communication'25 | 'diversity'26 | 'education'27 | 'engineering'28 | 'executive'29 | 'facilities'30 | 'finance'31 | 'healthcare'32 | 'hr'33 | 'it'34 | 'legal'35 | 'management'36 | 'marketing'37 | 'operations'38 | 'pr'39 | 'sales'40 | 'security'41 | 'software'42 | 'support'43 | 'warehouse';44
45interface ActorInput extends RunOptions {46 domains?: string[];47 companies?: string[];48 maxEmailsPerDomain?: number;49 includeCompanyInfo?: boolean;50 outputFormat?: 'detailed' | 'simple';51 // Domain search query parameters52 page?: number;53 limit?: '10' | '20' | '50';54 department?: Department;55 country?: string; // Two-letter country code (e.g., "US")56 enrichMobile?: boolean;57 webhookUrl?: string;58}59
60interface SearchBody {61 data?: { organization?: { website_url?: unknown }; emails?: unknown[] };62 meta?: { total?: number; total_pages?: number; params?: { domain?: unknown } };63}64
65/** One search: by domain (`domain=`) or by company name (`company=`). */66interface Target {67 kind: 'domain' | 'company';68 value: string;69 /** Resume-state key. Domains keep their plain value; company names are prefixed so they never collide. */70 key: string;71}72
73type DomainSearchParams = Parameters<Domain['domainSearch']>[0];74
75/**76 * Domain Search pricing: 1 credit per 10 results requested with `limit` (rounded up),77 * plus 5 credits per returned address with phone data (a non-empty `phone_data` array, only78 * returned with `enrich_mobile=true`). Each credit is one `tomba-request` event.79 */80function domainSearchCredits(requestLimit: number, emails: unknown, enrichMobile: boolean): number {81 const base = Math.ceil(requestLimit / 10);82 if (!enrichMobile || !Array.isArray(emails)) return base;83 return base + PHONE_CREDITS * emails.filter(hasPhoneData).length;84}85
86/** The domain Tomba resolved for a company search, if it reports one. */87function resolvedDomain(body: SearchBody): string | undefined {88 const candidates = [body.meta?.params?.domain, body.data?.organization?.website_url];89 for (const candidate of candidates) {90 if (typeof candidate === 'string' && candidate.trim()) return normalizeDomain(candidate);91 }92 return undefined;93}94
95await runActor<ActorInput>({96 title: 'Domain Search',97 count: (input) => (input.domains?.length ?? 0) + (input.companies?.length ?? 0),98 fromQuery: (query) => ({99 domains: queryList(query, 'domain', 'domains'),100 companies: queryList(query, 'company', 'companies'),101 maxEmailsPerDomain: queryInt(query, 'maxEmailsPerDomain'),102 includeCompanyInfo: queryBool(query, 'includeCompanyInfo'),103 page: queryInt(query, 'page'),104 limit: queryString(query, 'limit') as ActorInput['limit'],105 department: queryString(query, 'department') as Department | undefined,106 country: queryString(query, 'country'),107 enrichMobile: queryBool(query, 'enrichMobile'),108 webhookUrl: queryString(query, 'webhookUrl'),109 }),110 run: async (input, { push, isDone, markDone, standby }) => {111 const {112 domains: rawDomains = [],113 companies: rawCompanies = [],114 maxEmailsPerDomain = 10,115 includeCompanyInfo = true,116 outputFormat,117 page: startPage = 1,118 limit = '10',119 department,120 country: rawCountry,121 enrichMobile = false,122 webhookUrl: rawWebhookUrl,123 } = input;124 const country =125 typeof rawCountry === 'string' && rawCountry.trim() ? rawCountry.trim().toUpperCase() : undefined;126
127 const domains = unique(128 (Array.isArray(rawDomains) ? rawDomains : []).filter((d) => typeof d === 'string').map(normalizeDomain),129 );130 const companies = unique(131 (Array.isArray(rawCompanies) ? rawCompanies : [])132 .filter((c) => typeof c === 'string')133 .map((c) => c.trim().replace(/\s+/g, ' '))134 .filter(Boolean),135 (c) => c.toLowerCase(),136 );137 if (!domains.length && !companies.length) {138 throw new InputError(139 'Input must contain at least one domain in "domains" or one company name in "companies".',140 );141 }142
143 const webhookUrl = rawWebhookUrl?.trim() || undefined;144 if (webhookUrl && !/^https?:\/\//i.test(webhookUrl)) {145 throw new InputError('"webhookUrl" must start with http:// or https://.');146 }147
148 const domainService = new Domain(getClient());149
150 const targets: Target[] = [151 ...domains.map((value): Target => ({ kind: 'domain', value, key: value })),152 ...companies.map((value): Target => ({ kind: 'company', value, key: `company:${value.toLowerCase()}` })),153 ];154 const pending = targets.filter((t) => !isDone(t.key));155 if (pending.length < targets.length) {156 log.info(`Resuming: ${targets.length - pending.length} searches already processed.`);157 }158
159 const pageSize = Number(limit);160 const maxEmails = Math.max(1, maxEmailsPerDomain);161 if (!standby) {162 log.info(`Searching emails for ${pending.length} domains/companies`, {163 maxEmailsPerDomain: maxEmails,164 limit,165 department,166 country,167 enrichMobile,168 });169 }170
171 await runPool(pending, async (target) => {172 const label = target.kind === 'domain' ? target.value : `company "${target.value}"`;173 const id = target.kind === 'domain' ? { domain: target.value } : { company: target.value };174 let organization: unknown;175 let meta: unknown;176 let domain: string | undefined;177 const emails: unknown[] = [];178 let pages = 0;179 let chargedPages = 0;180 let chargedCredits = 0;181 let allCached = true;182 let error: string | undefined;183
184 // Pages of one search are fetched sequentially; each page is one billable request.185 for (let page = Math.max(1, startPage); emails.length < maxEmails; page++) {186 const params: Record<string, unknown> = { ...id, page, limit };187 if (department) params.department = department;188 if (country) params.country = country;189 if (enrichMobile) params.enrich_mobile = true;190 if (webhookUrl) params.webhook_url = webhookUrl;191 const res = await callTomba(192 'domain-search',193 params,194 async () => domainService.domainSearch(params as DomainSearchParams),195 EVENT_REQUEST,196 (body) => domainSearchCredits(pageSize, (body.data as SearchBody['data'])?.emails, enrichMobile),197 );198 if (res.skipped) {199 // Max charge limit reached before the first page: leave the search for a later run.200 if (pages === 0) return;201 break;202 }203
204 if (!isBillable(res.body)) {205 if (pages === 0) error = res.error ?? 'No results found';206 else log.warning(`${label}: page ${page} returned no results`, { error: res.error });207 break;208 }209
210 const body = res.body as SearchBody;211 pages++;212 if (res.charged) chargedPages++;213 chargedCredits += res.chargedCount ?? 0;214 if (!res.cached) allCached = false;215 organization ??= body.data?.organization;216 domain ??= target.kind === 'company' ? resolvedDomain(body) : undefined;217 meta = body.meta;218
219 const pageEmails = Array.isArray(body.data?.emails) ? body.data.emails : [];220 emails.push(...pageEmails.slice(0, maxEmails - emails.length));221
222 const totalPages = body.meta?.total_pages;223 const total = body.meta?.total;224 const noMore =225 pageEmails.length < pageSize ||226 (typeof totalPages === 'number' && page >= totalPages) ||227 (typeof total === 'number' && emails.length >= total);228 if (noMore) break;229 }230
231 if (pages === 0) {232 await push({ ...id, charged: false, cached: false, error });233 log.info(`${label}: ${error}`);234 } else {235 const phoneNumbers = emails.reduce<number>((sum, e) => sum + phoneDataCount(e), 0);236 await push({237 ...id,238 ...(domain ? { domain } : {}),239 ...(includeCompanyInfo ? { organization } : {}),240 emails,241 phoneNumbers,242 meta,243 pages,244 chargedRequests: chargedPages,245 chargedCredits,246 charged: chargedPages > 0,247 cached: allCached,248 });249 log.info(250 `${label}: ${emails.length} emails${enrichMobile ? `, ${phoneNumbers} phone numbers` : ''} from ${pages} page(s)${allCached ? ' (cached)' : ''}`,251 );252 }253
254 markDone(target.key);255 });256
257 // The run summary only makes sense for a batch run; Standby requests share one process.258 if (!standby) {259 await Actor.setValue('SUMMARY', {260 totalDomains: domains.length,261 totalCompanies: companies.length,262 tombaRequests: stats.requests,263 chargedRequests: stats.charged,264 cacheHits: stats.cached,265 processedAt: new Date().toISOString(),266 settings: {267 maxEmailsPerDomain: maxEmails,268 includeCompanyInfo,269 outputFormat,270 page: startPage,271 limit,272 department,273 country,274 enrichMobile,275 webhookUrl,276 },277 });278 }279 },280});1// Shared test harness for the Tomba Apify Actors.2// Keep this file identical across all Actors.3import { spawn } from 'node:child_process';4import { readFileSync } from 'node:fs';5import { mkdir, mkdtemp, readdir, readFile, rm, writeFile } from 'node:fs/promises';6import type { IncomingMessage, Server, ServerResponse } from 'node:http';7import { createServer, request as httpRequest } from 'node:http';8import type { AddressInfo } from 'node:net';9import { tmpdir } from 'node:os';10import { join } from 'node:path';11
12import { Ajv } from 'ajv';13
14export interface MockRequest {15 method: string;16 path: string;17 query: Record<string, string>;18 headers: IncomingMessage['headers'];19 body: unknown;20}21
22export interface MockResponse {23 status?: number;24 body?: unknown;25 /** Raw (non-JSON) body. */26 raw?: string;27 headers?: Record<string, string>;28}29
30export type MockHandler = (req: MockRequest) => MockResponse | Promise<MockResponse>;31
32export interface MockServer {33 url: string;34 requests: MockRequest[];35 close: () => Promise<void>;36}37
38/** Start a local HTTP server that impersonates the Tomba API. */39export async function startMockTomba(handler: MockHandler): Promise<MockServer> {40 const requests: MockRequest[] = [];41 const server: Server = createServer((req: IncomingMessage, res: ServerResponse) => {42 const chunks: Buffer[] = [];43 req.on('data', (chunk: Buffer) => chunks.push(chunk));44 req.on('end', async () => {45 const url = new URL(req.url ?? '/', 'http://localhost');46 const text = Buffer.concat(chunks).toString();47 let body: unknown = text;48 try {49 body = text ? JSON.parse(text) : undefined;50 } catch {51 // keep raw text52 }53 const request: MockRequest = {54 method: req.method ?? 'GET',55 path: url.pathname,56 query: Object.fromEntries(url.searchParams),57 headers: req.headers,58 body,59 };60 requests.push(request);61 try {62 const out = await handler(request);63 const status = out.status ?? 200;64 if (out.raw !== undefined) {65 res.writeHead(status, { 'content-type': 'text/html', ...out.headers });66 res.end(out.raw);67 } else {68 res.writeHead(status, { 'content-type': 'application/json', ...out.headers });69 res.end(JSON.stringify(out.body ?? {}));70 }71 } catch (err) {72 res.writeHead(500, { 'content-type': 'application/json' });73 res.end(JSON.stringify({ errors: { message: (err as Error).message } }));74 }75 });76 });77 await new Promise<void>((resolve) => {78 server.listen(0, '127.0.0.1', resolve);79 });80 const { port } = server.address() as AddressInfo;81 return {82 url: `http://127.0.0.1:${port}`,83 requests,84 close: async () =>85 new Promise<void>((resolve) => {86 server.close(() => resolve());87 }),88 };89}90
91export interface RunOptions {92 input: Record<string, unknown>;93 /** Mock server URL; omit to run without an endpoint override. */94 endpoint?: string;95 /** Reuse a storage directory (to test cache and resume). */96 storageDir?: string;97 /** Keep the default storages from the previous run in `storageDir`. */98 keepStorage?: boolean;99 /** Max total charge in USD (locally every event costs $1). */100 maxTotalChargeUsd?: number;101 /** Set to false to run without Tomba credentials. */102 withCredentials?: boolean;103 timeoutMs?: number;104}105
106export interface ChargeLogEntry {107 eventName: string;108 chargedCount?: number;109 [key: string]: unknown;110}111
112export interface RunResult {113 code: number | null;114 output: string;115 items: Record<string, unknown>[];116 charges: ChargeLogEntry[];117 storageDir: string;118 /** Number of charged events per event name. */119 chargeCounts: Record<string, number>;120}121
122/** Run the Actor (`src/main.ts`) in a child process against local storage. */123export async function runActor(options: RunOptions): Promise<RunResult> {124 const storageDir = options.storageDir ?? (await mkdtemp(join(tmpdir(), 'tomba-actor-test-')));125 const inputDir = join(storageDir, 'key_value_stores', 'default');126 await mkdir(inputDir, { recursive: true });127 await writeFile(join(inputDir, 'INPUT.json'), JSON.stringify(options.input));128
129 const env: NodeJS.ProcessEnv = {130 PATH: process.env.PATH,131 HOME: process.env.HOME,132 CRAWLEE_STORAGE_DIR: storageDir,133 CRAWLEE_PURGE_ON_START: options.keepStorage ? '0' : '1',134 APIFY_LOG_LEVEL: 'INFO',135 ACTOR_TEST_PAY_PER_EVENT: 'true',136 ACTOR_USE_CHARGING_LOG_DATASET: 'true',137 };138 if (options.withCredentials !== false) {139 env.TOMBA_API_KEY = 'ta_test_key';140 env.TOMBA_API_SECRET = 'ts_test_secret';141 }142 if (options.endpoint) env.TOMBA_API_ENDPOINT = options.endpoint;143 if (options.maxTotalChargeUsd !== undefined) env.ACTOR_MAX_TOTAL_CHARGE_USD = String(options.maxTotalChargeUsd);144
145 const { code, output } = await new Promise<{ code: number | null; output: string }>((resolve, reject) => {146 const child = spawn(process.execPath, ['--import', 'tsx', 'src/main.ts'], { env, cwd: process.cwd() });147 let out = '';148 child.stdout.on('data', (d: Buffer) => {149 out += d.toString();150 });151 child.stderr.on('data', (d: Buffer) => {152 out += d.toString();153 });154 const timer = setTimeout(() => {155 child.kill('SIGKILL');156 reject(new Error(`Actor run timed out\n${out}`));157 }, options.timeoutMs ?? 60_000);158 child.on('close', (c) => {159 clearTimeout(timer);160 resolve({ code: c, output: out });161 });162 });163
164 const items = await readDataset(storageDir, 'default');165 assertMatchesDatasetSchema(items);166 const charges = (await readDataset(storageDir, 'charging_log')) as ChargeLogEntry[];167 const chargeCounts: Record<string, number> = {};168 for (const c of charges) chargeCounts[c.eventName] = (chargeCounts[c.eventName] ?? 0) + (c.chargedCount ?? 1);169
170 return { code, output, items, charges, storageDir, chargeCounts };171}172
173export async function readDataset(storageDir: string, name: string): Promise<Record<string, unknown>[]> {174 const dir = join(storageDir, 'datasets', name);175 let files: string[];176 try {177 files = (await readdir(dir)).filter((f) => f.endsWith('.json')).sort();178 } catch {179 return [];180 }181 return Promise.all(files.map(async (f) => JSON.parse(await readFile(join(dir, f), 'utf8'))));182}183
184export async function removeStorage(storageDir: string): Promise<void> {185 await rm(storageDir, { recursive: true, force: true });186}187
188/** Total number of charged events across all event names. */189export function totalCharges(result: RunResult): number {190 return Object.values(result.chargeCounts).reduce((a, b) => a + b, 0);191}192
193export interface StandbyActor {194 /** Base URL of the Actor's HTTP server. */195 url: string;196 storageDir: string;197 /** GET or POST a request to the Actor and parse the JSON response. */198 call: (199 path: string,200 init?: { method?: string; body?: unknown; headers?: Record<string, string> },201 ) => Promise<{ status: number; body: Record<string, unknown> }>;202 /** Stop the Actor and return its log output and charges. */203 stop: () => Promise<{ output: string; chargeCounts: Record<string, number> }>;204}205
206async function freePort(): Promise<number> {207 const server = createServer();208 await new Promise<void>((resolve) => {209 server.listen(0, '127.0.0.1', resolve);210 });211 const { port } = server.address() as AddressInfo;212 await new Promise<void>((resolve) => {213 server.close(() => resolve());214 });215 return port;216}217
218async function httpCall(219 url: string,220 init: { method?: string; body?: unknown; headers?: Record<string, string> } = {},221): Promise<{ status: number; body: Record<string, unknown> }> {222 let payload: string | undefined;223 if (typeof init.body === 'string') payload = init.body;224 else if (init.body !== undefined) payload = JSON.stringify(init.body);225 return new Promise((resolve, reject) => {226 const req = httpRequest(227 url,228 {229 method: init.method ?? (payload ? 'POST' : 'GET'),230 headers: { 'content-type': 'application/json', ...init.headers },231 },232 (res) => {233 const chunks: Buffer[] = [];234 res.on('data', (c: Buffer) => chunks.push(c));235 res.on('end', () => {236 const text = Buffer.concat(chunks).toString();237 let body: Record<string, unknown> = {};238 try {239 body = JSON.parse(text);240 } catch {241 body = { raw: text };242 }243 resolve({ status: res.statusCode ?? 0, body });244 });245 },246 );247 req.on('error', reject);248 if (payload) req.write(payload);249 req.end();250 });251}252
253/** Start the Actor in Standby mode (real-time HTTP API) and wait until it is ready. */254export async function startStandbyActor(255 options: { endpoint?: string; maxTotalChargeUsd?: number; withCredentials?: boolean } = {},256): Promise<StandbyActor> {257 const storageDir = await mkdtemp(join(tmpdir(), 'tomba-standby-test-'));258 const port = await freePort();259 const env: NodeJS.ProcessEnv = {260 PATH: process.env.PATH,261 HOME: process.env.HOME,262 CRAWLEE_STORAGE_DIR: storageDir,263 APIFY_LOG_LEVEL: 'INFO',264 APIFY_META_ORIGIN: 'STANDBY',265 ACTOR_WEB_SERVER_PORT: String(port),266 ACTOR_TEST_PAY_PER_EVENT: 'true',267 ACTOR_USE_CHARGING_LOG_DATASET: 'true',268 };269 if (options.withCredentials !== false) {270 env.TOMBA_API_KEY = 'ta_test_key';271 env.TOMBA_API_SECRET = 'ts_test_secret';272 }273 if (options.endpoint) env.TOMBA_API_ENDPOINT = options.endpoint;274 if (options.maxTotalChargeUsd !== undefined) env.ACTOR_MAX_TOTAL_CHARGE_USD = String(options.maxTotalChargeUsd);275
276 const child = spawn(process.execPath, ['--import', 'tsx', 'src/main.ts'], { env, cwd: process.cwd() });277 let output = '';278 child.stdout.on('data', (d: Buffer) => {279 output += d.toString();280 });281 child.stderr.on('data', (d: Buffer) => {282 output += d.toString();283 });284 const exited = new Promise<void>((resolve) => {285 child.on('close', () => resolve());286 });287
288 const url = `http://127.0.0.1:${port}`;289 const deadline = Date.now() + 30_000;290 for (;;) {291 if (child.exitCode !== null) throw new Error(`Standby Actor exited early\n${output}`);292 try {293 const res = await httpCall(`${url}/`, { headers: { 'x-apify-container-server-readiness-probe': '1' } });294 if (res.status === 200) break;295 } catch {296 // not listening yet297 }298 if (Date.now() > deadline) {299 child.kill('SIGKILL');300 throw new Error(`Standby Actor did not become ready\n${output}`);301 }302 await new Promise((r) => {303 setTimeout(r, 100);304 });305 }306
307 return {308 url,309 storageDir,310 call: async (path, init) => httpCall(`${url}${path}`, init),311 stop: async () => {312 child.kill('SIGTERM');313 await exited;314 const charges = (await readDataset(storageDir, 'charging_log')) as ChargeLogEntry[];315 const chargeCounts: Record<string, number> = {};316 for (const c of charges)317 chargeCounts[c.eventName] = (chargeCounts[c.eventName] ?? 0) + (c.chargedCount ?? 1);318 await removeStorage(storageDir);319 return { output, chargeCounts };320 },321 };322}323
324let datasetValidator: ReturnType<Ajv['compile']> | undefined;325
326/**327 * Apify validates every pushed item against `.actor/dataset_schema.json` and fails the run on a mismatch,328 * while local runs don't. Check the items the same way so schema mismatches fail the tests.329 */330export function assertMatchesDatasetSchema(items: Record<string, unknown>[]): void {331 if (!datasetValidator) {332 const { fields } = JSON.parse(readFileSync(join(process.cwd(), '.actor/dataset_schema.json'), 'utf8'));333 delete fields.$schema;334 datasetValidator = new Ajv({ allErrors: true, strict: false }).compile(fields);335 }336 for (const [index, item] of items.entries()) {337 if (!datasetValidator(item)) {338 const errors = (datasetValidator.errors ?? [])339 .map((e) => `${e.instancePath || '/'} ${e.message}`)340 .join('; ');341 throw new Error(342 `Dataset item ${index} doesn't match .actor/dataset_schema.json: ${errors}\n${JSON.stringify(item)}`,343 );344 }345 }346}347
348/**349 * Validate an input against `.actor/input_schema.json` (Apify validates the input before a run or API call350 * starts). Returns the error messages; empty when the input is accepted.351 */352export function inputSchemaErrors(input: Record<string, unknown>): string[] {353 const schema = JSON.parse(readFileSync(join(process.cwd(), '.actor/input_schema.json'), 'utf8'));354 const validate = new Ajv({ allErrors: true, strict: false }).compile({355 type: 'object',356 properties: schema.properties,357 required: schema.required ?? [],358 });359 return validate(input) ? [] : (validate.errors ?? []).map((e) => `${e.instancePath || '/'} ${e.message}`);360}1// Unit tests for src/tomba.ts. Keep this file identical across all Actors.2import assert from 'node:assert/strict';3import { mkdtemp } from 'node:fs/promises';4import { tmpdir } from 'node:os';5import { join } from 'node:path';6import { after, before, describe, it } from 'node:test';7
8process.env.CRAWLEE_STORAGE_DIR = await mkdtemp(join(tmpdir(), 'tomba-unit-'));9process.env.ACTOR_TEST_PAY_PER_EVENT = 'true';10process.env.TOMBA_API_KEY = 'ta_unit';11process.env.TOMBA_API_SECRET = 'ts_unit';12
13const { Actor } = await import('apify');14const { TombaException } = await import('tomba');15const T = await import('../src/tomba.js');16
17const ok = (body: unknown) => async () => ({ data: body, rateLimit: {} });18const httpError = (status: number, body: unknown = {}, headers: Record<string, string> = {}) =>19 new TombaException('error', { status, headers }, body);20const charged = (event = T.EVENT_REQUEST) => Actor.getChargingManager().getChargedEventCount(event);21
22let seq = 0;23const params = () => ({ n: ++seq });24
25before(async () => {26 await Actor.init();27 await T.setupTomba({ maxRetries: 1, useCache: true, cacheTtlHours: 1, maxConcurrency: 4 });28});29
30after(async () => {31 await Actor.exit({ exit: false });32});33
34describe('isBillable', () => {35 const cases: [string, unknown, boolean][] = [36 ['data array', { data: [{ a: 1 }] }, true],37 ['data object', { data: { email: 'a@b.co' } }, true],38 ['negative answer (object with empty list)', { data: { organization: {}, emails: [] } }, true],39 ['negative answer (undeliverable)', { data: { email: { status: 'invalid' } } }, true],40 ['empty array', { data: [] }, false],41 ['empty object', { data: {} }, false],42 ['object with only null values', { data: { person: null, company: null } }, false],43 ['negative answer (email null, other fields set)', { data: { email: null, first_name: 'John' } }, true],44 ['null data', { data: null }, false],45 ['missing data', { meta: {} }, false],46 ['errors object', { data: { a: 1 }, errors: { message: 'x' } }, false],47 ['string body', '<html>', false],48 ['null body', null, false],49 ['array body', [1, 2], false],50 ];51 for (const [name, body, expected] of cases) {52 it(name, () => assert.equal(T.isBillable(body), expected));53 }54});55
56describe('normalizers', () => {57 it('normalizeDomain strips protocol, www, path and case', () => {58 assert.equal(T.normalizeDomain(' https://www.Stripe.com/about?x=1 '), 'stripe.com');59 assert.equal(T.normalizeDomain('http://tomba.io'), 'tomba.io');60 assert.equal(T.normalizeDomain('sub.example.org'), 'sub.example.org');61 });62
63 it('normalizeEmail trims and lowercases', () => {64 assert.equal(T.normalizeEmail(' John@Stripe.COM '), 'john@stripe.com');65 });66
67 it('unique keeps first occurrence and drops empty keys', () => {68 assert.deepEqual(T.unique(['a', 'b', 'a', '', 'c']), ['a', 'b', 'c']);69 assert.deepEqual(70 T.unique([{ id: 1 }, { id: 2 }, { id: 1 }], (x) => String(x.id)),71 [{ id: 1 }, { id: 2 }],72 );73 });74});75
76describe('phone data', () => {77 it('counts phone numbers in phone_data', () => {78 assert.equal(T.phoneDataCount({ phone_data: [{ number: '+1' }, { number: '+2' }] }), 2);79 assert.equal(T.phoneDataCount({ phone_data: [] }), 0);80 assert.equal(T.phoneDataCount({ phone_number: true }), 0);81 assert.equal(T.phoneDataCount(null), 0);82 assert.equal(T.phoneDataCount('x'), 0);83 });84
85 it('detects records with phone data', () => {86 assert.equal(T.hasPhoneData({ phone_data: [{ number: '+1' }] }), true);87 assert.equal(T.hasPhoneData({ phone_data: [] }), false);88 assert.equal(T.hasPhoneData({}), false);89 });90
91 it('prices phone data at 5 credits', () => {92 assert.equal(T.PHONE_CREDITS, 5);93 });94});95
96describe('cacheStoreName', () => {97 it("is unique per Actor so limited-permission runs never open another Actor's store", () => {98 const previous = process.env.ACTOR_ID;99 try {100 process.env.ACTOR_ID = 'AbC123xyz';101 assert.equal(T.cacheStoreName(), 'tomba-cache-AbC123xyz');102 delete process.env.ACTOR_ID;103 assert.equal(T.cacheStoreName(), 'tomba-cache');104 } finally {105 if (previous === undefined) delete process.env.ACTOR_ID;106 else process.env.ACTOR_ID = previous;107 }108 });109});110
111describe('setupTomba without cache permissions', () => {112 it('keeps working with an in-run cache when the cache store cannot be opened', async () => {113 const original = Actor.openKeyValueStore.bind(Actor);114 Actor.openKeyValueStore = async () => {115 throw new Error('Insufficient permissions for the key-value store.');116 };117 try {118 await T.setupTomba({ maxRetries: 1, useCache: true, cacheTtlHours: 1, maxConcurrency: 4 });119 const p = { cachePermissionTest: true };120 let calls = 0;121 const fn = async () => {122 calls++;123 return { data: { data: [1] }, rateLimit: {} };124 };125 const first = await T.callTomba('perm', p, fn);126 const second = await T.callTomba('perm', p, fn);127 assert.equal(first.charged, true);128 assert.equal(second.cached, true);129 assert.equal(calls, 1);130 } finally {131 Actor.openKeyValueStore = original;132 await T.setupTomba({ maxRetries: 1, useCache: true, cacheTtlHours: 1, maxConcurrency: 4 });133 }134 });135});136
137describe('runPool', () => {138 it('processes every item with bounded concurrency', async () => {139 let active = 0;140 let peak = 0;141 const seen: number[] = [];142 await T.runPool(143 Array.from({ length: 20 }, (_, i) => i),144 async (item) => {145 active++;146 peak = Math.max(peak, active);147 await new Promise((r) => {148 setTimeout(r, 5);149 });150 seen.push(item);151 active--;152 },153 3,154 );155 assert.equal(seen.length, 20);156 assert.deepEqual(157 [...seen].sort((a, b) => a - b),158 Array.from({ length: 20 }, (_, i) => i),159 );160 assert.ok(peak <= 3, `peak concurrency ${peak}`);161 });162
163 it('handles an empty list', async () => {164 await T.runPool([], async () => assert.fail('should not run'));165 });166});167
168describe('callTomba', () => {169 it('charges a response with data and unwraps the SDK { data, rateLimit } shape', async () => {170 const start = charged();171 const res = await T.callTomba('test', params(), ok({ data: [{ name: 'React' }] }));172 assert.equal(res.status, 200);173 assert.equal(res.charged, true);174 assert.equal(res.cached, false);175 assert.deepEqual(res.data, [{ name: 'React' }]);176 assert.equal(charged(), start + 1);177 });178
179 it('accepts an SDK that returns the body directly', async () => {180 const res = await T.callTomba('test', params(), async () => ({ data: { x: 1 } }));181 assert.equal(res.charged, true);182 assert.deepEqual(res.data, { x: 1 });183 });184
185 it('charges a negative answer', async () => {186 const res = await T.callTomba('test', params(), ok({ data: { organization: {}, emails: [] } }));187 assert.equal(res.charged, true);188 });189
190 it('serves a repeated request from cache without charging', async () => {191 const p = params();192 let calls = 0;193 const fn = async () => {194 calls++;195 return { data: { data: { v: 1 } }, rateLimit: {} };196 };197 await T.callTomba('test', p, fn);198 const start = charged();199 const res = await T.callTomba('test', { ...p }, fn);200 assert.equal(res.cached, true);201 assert.equal(res.charged, false);202 assert.deepEqual(res.data, { v: 1 });203 assert.equal(calls, 1);204 assert.equal(charged(), start);205 });206
207 it('uses a different cache entry for different params or endpoints', async () => {208 const p = params();209 await T.callTomba('a', p, ok({ data: [1] }));210 const other = await T.callTomba('b', p, ok({ data: [2] }));211 assert.equal(other.cached, false);212 assert.deepEqual(other.data, [2]);213 });214
215 it('does not charge or cache empty data', async () => {216 const p = params();217 const start = charged();218 const res = await T.callTomba('test', p, ok({ data: [] }));219 assert.equal(res.charged, false);220 assert.equal(res.status, 200);221 const again = await T.callTomba('test', p, ok({ data: [] }));222 assert.equal(again.cached, false);223 assert.equal(charged(), start);224 });225
226 it('does not charge a success with an errors object', async () => {227 const res = await T.callTomba('test', params(), ok({ data: { a: 1 }, errors: { message: 'Bad thing' } }));228 assert.equal(res.charged, false);229 assert.equal(res.error, 'Bad thing');230 });231
232 it('reports a non-JSON body as 502 without charging', async () => {233 const res = await T.callTomba('test', params(), ok('<html>oops</html>'));234 assert.equal(res.status, 502);235 assert.equal(res.charged, false);236 });237
238 it('does not retry or charge a 422', async () => {239 let calls = 0;240 const res = await T.callTomba('test', params(), async () => {241 calls++;242 throw httpError(422, { errors: { message: 'Invalid domain' } });243 });244 assert.equal(calls, 1);245 assert.equal(res.status, 422);246 assert.equal(res.charged, false);247 assert.match(res.error ?? '', /Invalid domain/);248 });249
250 it('does not retry SDK input validation errors', async () => {251 let calls = 0;252 const res = await T.callTomba('test', params(), async () => {253 calls++;254 throw new TombaException('Domain is required.');255 });256 assert.equal(calls, 1);257 assert.equal(res.status, 400);258 assert.equal(res.charged, false);259 });260
261 it('retries a 429 honoring Retry-After, then charges the success', async () => {262 let calls = 0;263 const res = await T.callTomba('test', params(), async () => {264 calls++;265 if (calls === 1) throw httpError(429, {}, { 'retry-after': '1' });266 return { data: { data: { ok: true } }, rateLimit: {} };267 });268 assert.equal(calls, 2);269 assert.equal(res.charged, true);270 });271
272 it('retries network errors', async () => {273 let calls = 0;274 const res = await T.callTomba('test', params(), async () => {275 calls++;276 if (calls === 1) throw new TombaException('connect ECONNREFUSED 127.0.0.1:9');277 return { data: { data: [1] }, rateLimit: {} };278 });279 assert.equal(calls, 2);280 assert.equal(res.charged, true);281 });282
283 it('gives up after maxRetries on 5xx without charging', async () => {284 let calls = 0;285 const res = await T.callTomba('test', params(), async () => {286 calls++;287 throw httpError(503);288 });289 assert.equal(calls, 2); // 1 attempt + maxRetries (1)290 assert.equal(res.status, 503);291 assert.equal(res.charged, false);292 });293
294 it('charges the given event name', async () => {295 const start = charged(T.EVENT_PHONE_FINDER);296 await T.callTomba('phone', params(), ok({ data: { phone: '+1' } }), T.EVENT_PHONE_FINDER);297 assert.equal(charged(T.EVENT_PHONE_FINDER), start + 1);298 });299
300 it('charges a fixed count of events', async () => {301 const start = charged();302 const res = await T.callTomba('test', params(), ok({ data: [1] }), T.EVENT_REQUEST, 3);303 assert.equal(res.chargedCount, 3);304 assert.equal(charged(), start + 3);305 });306
307 it('charges a count computed from the response body', async () => {308 const start = charged();309 const res = await T.callTomba(310 'test',311 params(),312 ok({ data: { items: [1, 2, 3, 4] } }),313 T.EVENT_REQUEST,314 (body) => (body.data as { items: unknown[] }).items.length,315 );316 assert.equal(res.chargedCount, 4);317 assert.equal(charged(), start + 4);318 });319
320 it('does not charge a count for a non-billable response', async () => {321 const start = charged();322 const res = await T.callTomba('test', params(), ok({ data: [] }), T.EVENT_REQUEST, 5);323 assert.equal(res.charged, false);324 assert.equal(charged(), start);325 });326
327 it('tracks stats', () => {328 assert.ok(T.stats.requests > 0);329 assert.ok(T.stats.charged > 0);330 assert.ok(T.stats.cached > 0);331 assert.ok(T.stats.retries > 0);332 assert.ok(T.stats.failed > 0);333 });334});1// End-to-end tests: run the Actor against a mock Tomba API.2import assert from 'node:assert/strict';3import { after, afterEach, describe, it } from 'node:test';4
5import type { MockHandler, MockServer } from './helpers.js';6import {7 inputSchemaErrors,8 removeStorage,9 runActor,10 startMockTomba,11 startStandbyActor,12 totalCharges,13} from './helpers.js';14
15function organization(domain: string) {16 return {17 website_url: domain,18 organization: 'Stripe',19 location: {20 country: 'US',21 city: 'San Francisco',22 state: 'California',23 street_address: '354 Oyster Point Blvd',24 postal_code: '94080',25 },26 social_links: {27 twitter_url: 'https://twitter.com/stripe',28 facebook_url: null,29 linkedin_url: 'https://www.linkedin.com/company/stripe',30 },31 phone_number: '+1 415 298 5539',32 industries: 'Information Technology and Services',33 founded: '2010',34 company_size: '5K-10K',35 company_type: 'privately held',36 revenue: '$100M-$250M',37 accept_all: false,38 description: 'Stripe is a payment processing platform.',39 pattern: '{first}',40 disposable: false,41 webmail: false,42 };43}44
45const PHONE = { number: '+14155550123', type: 'mobile' };46
47function email(domain: string, n: number, enrichMobile = false) {48 return {49 email: `person${n}@${domain}`,50 first_name: `First${n}`,51 last_name: `Last${n}`,52 full_name: `First${n} Last${n}`,53 gender: 'female',54 // Tomba's "has a phone number" flag: informational only, it does not affect pricing.55 phone_number: domain === 'phones.com' && n % 3 === 0,56 // With enrich_mobile=true every third address carries its phone numbers.57 ...(enrichMobile ? { phone_data: n % 3 === 0 ? [PHONE] : [] } : {}),58 type: 'personal',59 country: 'US',60 position: 'Financial Crimes Analyst',61 department: 'finance',62 seniority: 'senior',63 twitter: null,64 linkedin: `https://www.linkedin.com/in/person${n}`,65 accept_all: false,66 pattern: '{first}',67 score: 99,68 verification: { date: '2025-09-13T00:00:00+02:00', status: 'valid' },69 last_updated_at: '2025-09-13T00:00:00+02:00',70 sources: [71 {72 uri: `https://${domain}/about`,73 website_url: domain,74 extracted_on: '2024-01-01T00:00:00+02:00',75 last_seen_on: '2025-09-01T00:00:00+02:00',76 still_on_page: true,77 },78 ],79 };80}81
82/** Number of emails Tomba knows for each test domain (default 25). */83const EMAIL_COUNTS: Record<string, number> = { 'exact.com': 20, 'nometa.com': 15, 'nomail.com': 0 };84
85/** A realistic paged Domain Search response. */86function searchPage(query: Record<string, string>, withMeta = true) {87 // A company search resolves the company name to its domain.88 const domain =89 query.domain ??90 `${String(query.company)91 .toLowerCase()92 .replace(/[^a-z0-9]/g, '')}.com`;93 const enrichMobile = query.enrich_mobile === 'true';94 const total = EMAIL_COUNTS[domain] ?? 25;95 const pageSize = Number(query.limit ?? 10);96 const page = Number(query.page ?? 1);97 const first = (page - 1) * pageSize;98 const emails = Array.from({ length: Math.max(0, Math.min(pageSize, total - first)) }, (_, i) =>99 email(domain, first + i + 1, enrichMobile),100 );101 return {102 data: { organization: organization(domain), emails },103 ...(withMeta104 ? {105 meta: {106 total,107 pageSize,108 current: page,109 total_pages: Math.ceil(total / pageSize),110 params: { domain, page, limit: pageSize },111 },112 }113 : {}),114 };115}116
117/** Default Tomba behaviour. */118const tomba: MockHandler = (req) => {119 assert.equal(req.method, 'GET');120 assert.equal(req.path, '/domain-search');121 const { domain, page } = req.query;122 if (domain === 'empty.com') return { body: { data: null } };123 if (domain === 'invalid.com') return { status: 422, body: { errors: { message: 'Invalid domain' } } };124 if (domain === 'html.com') return { raw: '<html>Bad gateway</html>' };125 if (domain === 'brokenpage.com' && page === '2') return { status: 422, body: { errors: { message: 'Bad page' } } };126 return { body: searchPage(req.query, domain !== 'nometa.com') };127};128
129const servers: MockServer[] = [];130const dirs: string[] = [];131
132async function mock(handler: MockHandler = tomba): Promise<MockServer> {133 const server = await startMockTomba(handler);134 servers.push(server);135 return server;136}137
138async function run(...args: Parameters<typeof runActor>) {139 const result = await runActor(...args);140 dirs.push(result.storageDir);141 return result;142}143
144const pagesRequested = (server: MockServer) => server.requests.map((r) => `${r.query.domain}#${r.query.page}`);145
146afterEach(async () => {147 await Promise.all(servers.splice(0).map(async (s) => s.close()));148});149
150after(async () => {151 await Promise.all(dirs.map(removeStorage));152});153
154describe('domain-search', () => {155 it('returns one item per domain and charges one event per billable page', async () => {156 const server = await mock();157 const result = await run({ input: { domains: ['stripe.com'] }, endpoint: server.url });158
159 assert.equal(result.code, 0, result.output);160 assert.equal(result.items.length, 1);161 const expected = searchPage({ domain: 'stripe.com', page: '1', limit: '10' });162 assert.deepEqual(result.items[0], {163 domain: 'stripe.com',164 organization: expected.data.organization,165 emails: expected.data.emails,166 phoneNumbers: 0,167 meta: expected.meta,168 pages: 1,169 chargedRequests: 1,170 chargedCredits: 1,171 charged: true,172 cached: false,173 });174 assert.deepEqual(server.requests[0].query, { domain: 'stripe.com', page: '1', limit: '10' });175 assert.deepEqual(result.chargeCounts, { 'tomba-request': 1 });176 });177
178 describe('pricing: 1 credit per 10 results requested, +5 per address with phone data', () => {179 for (const [limit, credits] of [180 ['10', 1],181 ['20', 2],182 ['50', 5],183 ] as const) {184 it(`limit ${limit} costs ${credits} credit(s) per page`, async () => {185 const server = await mock();186 const result = await run({187 input: { domains: ['stripe.com'], limit, maxEmailsPerDomain: Number(limit) },188 endpoint: server.url,189 });190 assert.equal(server.requests.length, 1);191 assert.equal(result.items[0].chargedCredits, credits);192 assert.deepEqual(result.chargeCounts, { 'tomba-request': credits });193 });194 }195
196 it('does not charge extra for the phone_number flag alone', async () => {197 const server = await mock();198 const result = await run({199 input: { domains: ['phones.com'], limit: '10', maxEmailsPerDomain: 10 },200 endpoint: server.url,201 });202 const emails = result.items[0].emails as { phone_number: boolean; phone_data?: unknown }[];203 assert.equal(emails.filter((e) => e.phone_number).length, 3);204 assert.ok(emails.every((e) => e.phone_data === undefined));205 assert.equal(result.items[0].phoneNumbers, 0);206 assert.equal(result.items[0].chargedCredits, 1);207 assert.deepEqual(result.chargeCounts, { 'tomba-request': 1 });208 });209
210 it('with enrichMobile adds 5 credits per returned address with phone data', async () => {211 const server = await mock();212 const result = await run({213 input: { domains: ['phones.com'], limit: '10', maxEmailsPerDomain: 10, enrichMobile: true },214 endpoint: server.url,215 });216 // 10 addresses, 3 with phone data (3, 6, 9): 1 + 3 x 5 = 16 credits.217 const emails = result.items[0].emails as { phone_data: unknown[] }[];218 assert.equal(emails.filter((e) => e.phone_data.length > 0).length, 3);219 assert.equal(result.items[0].chargedCredits, 16);220 assert.deepEqual(result.chargeCounts, { 'tomba-request': 16 });221 });222
223 it('charges the phone data of the whole returned page, even when fewer addresses are kept', async () => {224 const server = await mock();225 const result = await run({226 input: { domains: ['phones.com'], limit: '20', maxEmailsPerDomain: 2, enrichMobile: true },227 endpoint: server.url,228 });229 // limit 20 returns 20 addresses (6 with phone data) even though only 2 are kept: 2 + 6 x 5 = 32.230 assert.equal(result.items[0].chargedCredits, 32);231 assert.equal(result.items[0].phoneNumbers, 0);232 });233
234 it('charges every page by its own results', async () => {235 const server = await mock();236 const result = await run({237 input: { domains: ['phones.com'], limit: '10', maxEmailsPerDomain: 25, enrichMobile: true },238 endpoint: server.url,239 });240 // Pages of 10, 10 and 5 addresses with 3, 3 and 2 phones: (1+15) + (1+15) + (1+10) = 43.241 assert.equal(server.requests.length, 3);242 assert.equal(result.items[0].chargedCredits, 43);243 assert.deepEqual(result.chargeCounts, { 'tomba-request': 43 });244 });245
246 it('charges the base credits for a company with no addresses', async () => {247 const server = await mock();248 const result = await run({ input: { domains: ['nomail.com'], limit: '50' }, endpoint: server.url });249 assert.equal(result.items[0].chargedCredits, 5);250 assert.deepEqual(result.chargeCounts, { 'tomba-request': 5 });251 });252
253 it('charges nothing for cached pages', async () => {254 const server = await mock();255 const input = { domains: ['phones.com'], limit: '10', maxEmailsPerDomain: 10, enrichMobile: true };256 const first = await run({ input, endpoint: server.url });257 assert.equal(first.items[0].chargedCredits, 16);258 const second = await run({259 input,260 endpoint: server.url,261 storageDir: first.storageDir,262 });263 assert.equal(second.items[0].chargedCredits, 0);264 assert.equal(second.items[0].cached, true);265 assert.deepEqual(second.chargeCounts, {});266 });267 });268
269 it('sends the built-in credentials to Tomba', async () => {270 const server = await mock();271 await run({ input: { domains: ['stripe.com'] }, endpoint: server.url });272 assert.equal(server.requests[0].headers['x-tomba-key'], 'ta_test_key');273 assert.equal(server.requests[0].headers['x-tomba-secret'], 'ts_test_secret');274 });275
276 it('normalizes and deduplicates domains', async () => {277 const server = await mock();278 await run({279 input: { domains: ['https://www.Stripe.com/jobs', 'stripe.com', 'STRIPE.COM '] },280 endpoint: server.url,281 });282 assert.deepEqual(283 server.requests.map((r) => r.query.domain),284 ['stripe.com'],285 );286 });287
288 it('passes the filters and the start page to Tomba', async () => {289 const server = await mock();290 await run({291 input: { domains: ['stripe.com'], page: 2, limit: '20', department: 'finance', country: 'US' },292 endpoint: server.url,293 });294 assert.deepEqual(server.requests[0].query, {295 domain: 'stripe.com',296 page: '2',297 limit: '20',298 department: 'finance',299 country: 'US',300 });301 });302
303 it('sends enrich_mobile and webhook_url as query parameters', async () => {304 const server = await mock();305 await run({306 input: { domains: ['stripe.com'], enrichMobile: true, webhookUrl: 'https://example.com/hook' },307 endpoint: server.url,308 });309 assert.deepEqual(server.requests[0].query, {310 domain: 'stripe.com',311 page: '1',312 limit: '10',313 enrich_mobile: 'true',314 webhook_url: 'https://example.com/hook',315 });316 });317
318 it('does not send enrich_mobile when enrichMobile is off', async () => {319 const server = await mock();320 await run({ input: { domains: ['stripe.com'], enrichMobile: false }, endpoint: server.url });321 assert.equal('enrich_mobile' in server.requests[0].query, false);322 assert.equal('webhook_url' in server.requests[0].query, false);323 });324
325 it('fails on a webhook URL that is not http(s) without calling Tomba', async () => {326 const server = await mock();327 const result = await run({328 input: { domains: ['stripe.com'], webhookUrl: 'ftp://example.com' },329 endpoint: server.url,330 });331 assert.notEqual(result.code, 0);332 assert.equal(server.requests.length, 0);333 });334
335 it('reports the phone numbers returned with enrichMobile', async () => {336 const server = await mock();337 const result = await run({338 input: { domains: ['phones.com'], maxEmailsPerDomain: 10, enrichMobile: true },339 endpoint: server.url,340 });341 const emails = result.items[0].emails as { email: string; phone_data: unknown[] }[];342 assert.deepEqual(emails[2].phone_data, [{ number: '+14155550123', type: 'mobile' }]);343 assert.deepEqual(emails[0].phone_data, []);344 assert.equal(result.items[0].phoneNumbers, 3);345 });346
347 it('searches company names with company instead of domain and deduplicates them', async () => {348 const server = await mock();349 const result = await run({350 input: { companies: ['Stripe', ' stripe ', 'Acme Corp', ''] },351 endpoint: server.url,352 });353 assert.equal(result.code, 0, result.output);354 assert.deepEqual(355 server.requests.map((r) => r.query),356 [357 { company: 'Stripe', page: '1', limit: '10' },358 { company: 'Acme Corp', page: '1', limit: '10' },359 ],360 );361 const byCompany = Object.fromEntries(result.items.map((i) => [i.company, i]));362 assert.equal(byCompany.Stripe.domain, 'stripe.com');363 assert.equal((byCompany.Stripe.emails as unknown[]).length, 10);364 assert.equal(byCompany.Stripe.chargedCredits, 1);365 assert.equal(byCompany['Acme Corp'].domain, 'acmecorp.com');366 assert.deepEqual(result.chargeCounts, { 'tomba-request': 2 });367 });368
369 it('omits domain for a company search when Tomba does not report one', async () => {370 const server = await mock(() => ({ body: { data: { organization: { organization: 'X' }, emails: [] } } }));371 const result = await run({ input: { companies: ['Nowhere'] }, endpoint: server.url });372 assert.equal(result.items[0].company, 'Nowhere');373 assert.equal('domain' in result.items[0], false);374 });375
376 it('saves a company search without results with the company name', async () => {377 const server = await mock(() => ({ body: { data: null } }));378 const result = await run({ input: { companies: ['Unknown Co'] }, endpoint: server.url });379 assert.deepEqual(result.items, [380 { company: 'Unknown Co', charged: false, cached: false, error: 'No results found' },381 ]);382 assert.equal(totalCharges(result), 0);383 });384
385 it('searches domains and companies together and resumes them separately', async () => {386 const server = await mock();387 const input = { domains: ['stripe.com'], companies: ['Stripe'], useCache: false };388 const first = await run({ input, endpoint: server.url });389 assert.equal(first.code, 0, first.output);390 assert.deepEqual(391 server.requests.map((r) => r.query.domain ?? `company:${r.query.company}`),392 ['stripe.com', 'company:Stripe'],393 );394 assert.equal(first.items.length, 2);395 assert.deepEqual(first.chargeCounts, { 'tomba-request': 2 });396
397 // A resumed run (same storage) has nothing left to do.398 await run({ input, endpoint: server.url, storageDir: first.storageDir, keepStorage: true });399 assert.equal(server.requests.length, 2);400 });401
402 it('caches domain and company searches separately', async () => {403 const server = await mock();404 const first = await run({ input: { domains: ['stripe.com'] }, endpoint: server.url });405 const second = await run({406 input: { companies: ['Stripe'] },407 endpoint: server.url,408 storageDir: first.storageDir,409 });410 assert.equal(server.requests.length, 2);411 assert.equal(second.items[0].cached, false);412 assert.equal(second.items[0].charged, true);413 });414
415 it('fails without domains or companies and never calls the API', async () => {416 const server = await mock();417 const result = await run({ input: { domains: [], companies: [' '] }, endpoint: server.url });418 assert.notEqual(result.code, 0);419 assert.match(result.output, /domains.*companies/);420 assert.equal(server.requests.length, 0);421 });422
423 it('pages until maxEmailsPerDomain is reached, one request and one charge per page', async () => {424 const server = await mock();425 const result = await run({426 input: { domains: ['stripe.com'], maxEmailsPerDomain: 15, limit: '10' },427 endpoint: server.url,428 });429 assert.deepEqual(pagesRequested(server), ['stripe.com#1', 'stripe.com#2']);430 const item = result.items[0];431 assert.equal((item.emails as unknown[]).length, 15);432 assert.equal((item.emails as { email: string }[])[14].email, 'person15@stripe.com');433 assert.equal(item.pages, 2);434 assert.equal(item.chargedRequests, 2);435 assert.equal(item.charged, true);436 assert.deepEqual(result.chargeCounts, { 'tomba-request': 2 });437 });438
439 it('stops paging at the last page reported by Tomba', async () => {440 const server = await mock();441 const result = await run({442 input: { domains: ['exact.com'], maxEmailsPerDomain: 100, limit: '10' },443 endpoint: server.url,444 });445 // 20 emails, 2 full pages: total_pages stops paging without requesting an empty page 3.446 assert.deepEqual(pagesRequested(server), ['exact.com#1', 'exact.com#2']);447 assert.equal((result.items[0].emails as unknown[]).length, 20);448 assert.equal(totalCharges(result), 2);449 });450
451 it('stops paging at a short page', async () => {452 const server = await mock();453 const result = await run({454 input: { domains: ['nometa.com'], maxEmailsPerDomain: 100, limit: '10' },455 endpoint: server.url,456 });457 assert.deepEqual(pagesRequested(server), ['nometa.com#1', 'nometa.com#2']);458 assert.equal((result.items[0].emails as unknown[]).length, 15);459 assert.equal(result.items[0].pages, 2);460 assert.equal(totalCharges(result), 2);461 });462
463 it('keeps the pages it already paid for when a later page fails', async () => {464 const server = await mock();465 const result = await run({466 input: { domains: ['brokenpage.com'], maxEmailsPerDomain: 30, limit: '10' },467 endpoint: server.url,468 });469 assert.deepEqual(pagesRequested(server), ['brokenpage.com#1', 'brokenpage.com#2']);470 assert.equal((result.items[0].emails as unknown[]).length, 10);471 assert.equal(result.items[0].pages, 1);472 assert.equal(result.items[0].charged, true);473 assert.equal(totalCharges(result), 1);474 });475
476 it('charges an organization with zero emails as a negative answer', async () => {477 const server = await mock();478 const result = await run({ input: { domains: ['nomail.com'], maxEmailsPerDomain: 50 }, endpoint: server.url });479 assert.equal(server.requests.length, 1);480 const item = result.items[0];481 assert.deepEqual(item.emails, []);482 assert.equal((item.organization as { organization: string }).organization, 'Stripe');483 assert.equal(item.charged, true);484 assert.equal(item.error, undefined);485 assert.deepEqual(result.chargeCounts, { 'tomba-request': 1 });486 });487
488 it('omits the organization when includeCompanyInfo is false', async () => {489 const server = await mock();490 const result = await run({491 input: { domains: ['stripe.com'], includeCompanyInfo: false },492 endpoint: server.url,493 });494 assert.equal('organization' in result.items[0], false);495 assert.equal((result.items[0].emails as unknown[]).length, 10);496 });497
498 it('does not charge empty data', async () => {499 const server = await mock();500 const result = await run({ input: { domains: ['empty.com'] }, endpoint: server.url });501 assert.equal(result.code, 0, result.output);502 assert.deepEqual(result.items, [503 { domain: 'empty.com', charged: false, cached: false, error: 'No results found' },504 ]);505 assert.equal(totalCharges(result), 0);506 });507
508 it('does not charge Tomba error statuses and does not retry them', async () => {509 const server = await mock();510 const result = await run({ input: { domains: ['invalid.com'] }, endpoint: server.url });511 assert.equal(result.code, 0, result.output);512 assert.equal(server.requests.length, 1);513 assert.equal(result.items[0].charged, false);514 assert.match(String(result.items[0].error), /422: Invalid domain/);515 assert.equal(totalCharges(result), 0);516 });517
518 it('does not charge a non-JSON body', async () => {519 const server = await mock();520 const result = await run({ input: { domains: ['html.com'] }, endpoint: server.url });521 assert.equal(result.items[0].charged, false);522 assert.match(String(result.items[0].error), /Invalid response/);523 assert.equal(totalCharges(result), 0);524 });525
526 it('retries 429 and 5xx responses, then charges the success once', async () => {527 let calls = 0;528 const server = await mock(async (req) => {529 calls++;530 if (calls === 1)531 return {532 status: 429,533 body: { errors: { message: 'Too many requests' } },534 headers: { 'retry-after': '1' },535 };536 if (calls === 2) return { status: 503, body: {} };537 return tomba(req);538 });539 const result = await run({ input: { domains: ['stripe.com'], maxRetries: 3 }, endpoint: server.url });540 assert.equal(server.requests.length, 3);541 assert.equal(result.items[0].charged, true);542 assert.deepEqual(result.chargeCounts, { 'tomba-request': 1 });543 });544
545 it('serves repeated runs from the cache for free', async () => {546 const server = await mock();547 const input = { domains: ['stripe.com'], maxEmailsPerDomain: 20 };548 const first = await run({ input, endpoint: server.url });549 assert.equal(totalCharges(first), 2);550 const second = await run({ input, endpoint: server.url, storageDir: first.storageDir });551
552 assert.equal(server.requests.length, 2);553 assert.equal(second.items.length, 1);554 assert.equal((second.items[0].emails as unknown[]).length, 20);555 assert.equal(second.items[0].cached, true);556 assert.equal(second.items[0].charged, false);557 assert.equal(second.items[0].chargedRequests, 0);558 assert.equal(totalCharges(second), 0);559 });560
561 it('calls Tomba again when the cache is disabled', async () => {562 const server = await mock();563 const first = await run({ input: { domains: ['stripe.com'], useCache: false }, endpoint: server.url });564 await run({565 input: { domains: ['stripe.com'], useCache: false },566 endpoint: server.url,567 storageDir: first.storageDir,568 });569 assert.equal(server.requests.length, 2);570 });571
572 it('stops at the max charge limit and resumes without reprocessing', async () => {573 const server = await mock();574 const domains = ['a.com', 'b.com', 'c.com', 'd.com', 'e.com'];575 const input = { domains, maxConcurrency: 1, useCache: false };576
577 // Locally every event costs $1, so a $2 budget allows two billable page requests.578 const first = await run({ input, endpoint: server.url, maxTotalChargeUsd: 2 });579 assert.equal(first.code, 0, first.output);580 assert.equal(totalCharges(first), 2);581 assert.equal(server.requests.length, 2);582 assert.deepEqual(583 first.items.map((i) => i.domain),584 ['a.com', 'b.com'],585 );586
587 const second = await run({ input, endpoint: server.url, storageDir: first.storageDir, keepStorage: true });588 assert.equal(second.code, 0, second.output);589 // The charging log is kept with the storage: 2 events from the first run + 3 new ones.590 assert.equal(totalCharges(second), 5);591 assert.deepEqual(592 server.requests.map((r) => r.query.domain),593 domains,594 );595 });596
597 it('stops paging a domain when the charge limit is reached mid-domain', async () => {598 const server = await mock();599 const result = await run({600 input: { domains: ['stripe.com'], maxEmailsPerDomain: 30, useCache: false },601 endpoint: server.url,602 maxTotalChargeUsd: 2,603 });604 assert.equal(result.code, 0, result.output);605 assert.deepEqual(pagesRequested(server), ['stripe.com#1', 'stripe.com#2']);606 assert.equal((result.items[0].emails as unknown[]).length, 20);607 assert.equal(totalCharges(result), 2);608 });609
610 it('runs domains in parallel', async () => {611 let active = 0;612 let peak = 0;613 const server = await mock(async (req) => {614 active++;615 peak = Math.max(peak, active);616 await new Promise((r) => {617 setTimeout(r, 100);618 });619 active--;620 return tomba(req);621 });622 const domains = Array.from({ length: 8 }, (_, i) => `site${i}.com`);623 await run({ input: { domains, maxConcurrency: 4 }, endpoint: server.url });624 assert.equal(server.requests.length, 8);625 assert.ok(peak > 1 && peak <= 4, `peak concurrency ${peak}`);626 });627
628 it('fails without Tomba credentials and never calls the API', async () => {629 const server = await mock();630 const result = await run({ input: { domains: ['stripe.com'] }, endpoint: server.url, withCredentials: false });631 assert.notEqual(result.code, 0);632 assert.match(result.output, /misconfigured/);633 assert.doesNotMatch(result.output, /ta_test_key|ts_test_secret/);634 assert.equal(server.requests.length, 0);635 });636
637 it('fails on empty input', async () => {638 const server = await mock();639 const result = await run({ input: { domains: [] }, endpoint: server.url });640 assert.notEqual(result.code, 0);641 assert.equal(server.requests.length, 0);642 });643});644
645describe('domain-search standby (real-time API)', () => {646 it('answers the readiness probe and a bare GET with usage info', async () => {647 const server = await mock();648 const actor = await startStandbyActor({ endpoint: server.url });649 try {650 const probe = await actor.call('/', { headers: { 'x-apify-container-server-readiness-probe': '1' } });651 assert.equal(probe.status, 200);652 const usage = await actor.call('/');653 assert.equal(usage.status, 200);654 assert.match(String(usage.body.usage), /GET/);655 assert.equal(server.requests.length, 0);656 } finally {657 await actor.stop();658 }659 });660
661 it('searches domains and companies from GET query parameters and charges the credits per page', async () => {662 const server = await mock();663 const actor = await startStandbyActor({ endpoint: server.url });664 let stopped;665 try {666 const res = await actor.call(667 '/?domain=https://www.Stripe.com/pricing&domain=empty.com&company=Acme&limit=20&maxEmailsPerDomain=30&department=finance&country=US&includeCompanyInfo=false',668 );669 assert.equal(res.status, 200);670 const items = res.body.items as Record<string, unknown>[];671 const stripe = items.find((i) => i.domain === 'stripe.com');672 // 25 emails: page 1 has 20, page 2 the last 5; each page of limit 20 costs 2 credits.673 assert.equal((stripe?.emails as unknown[]).length, 25);674 assert.equal(stripe?.pages, 2);675 assert.equal(stripe?.chargedCredits, 4);676 assert.equal(stripe?.organization, undefined);677 const acme = items.find((i) => i.company === 'Acme');678 assert.equal(acme?.domain, 'acme.com');679 assert.equal(acme?.chargedCredits, 4);680 assert.equal(items.find((i) => i.domain === 'empty.com')?.error, 'No results found');681 const first = server.requests.find((r) => r.query.domain === 'stripe.com');682 assert.equal(first?.query.department, 'finance');683 assert.equal(first?.query.country, 'US');684 assert.equal(first?.query.limit, '20');685 } finally {686 stopped = await actor.stop();687 }688 assert.deepEqual(stopped.chargeCounts, { 'tomba-request': 8 });689 });690
691 it('accepts a POST with the same JSON input as a normal run', async () => {692 const server = await mock();693 const actor = await startStandbyActor({ endpoint: server.url });694 let stopped;695 try {696 const res = await actor.call('/', { body: { domains: ['phones.com'], enrichMobile: true } });697 assert.equal(res.status, 200);698 const [item] = res.body.items as Record<string, unknown>[];699 assert.equal((item.emails as unknown[]).length, 10);700 // 1 base credit + 5 per address with phone data (3 of 10).701 assert.equal(item.phoneNumbers, 3);702 assert.equal(item.chargedCredits, 16);703 assert.equal(server.requests[0].query.enrich_mobile, 'true');704 } finally {705 stopped = await actor.stop();706 }707 assert.deepEqual(stopped.chargeCounts, { 'tomba-request': 16 });708 });709
710 it('serves repeated requests from the cache for free', async () => {711 const server = await mock();712 const actor = await startStandbyActor({ endpoint: server.url });713 let stopped;714 try {715 await actor.call('/?domain=stripe.com');716 const second = await actor.call('/?domain=stripe.com');717 assert.ok((second.body.items as Record<string, unknown>[]).every((i) => i.cached === true));718 assert.equal(server.requests.length, 1);719 } finally {720 stopped = await actor.stop();721 }722 assert.deepEqual(stopped.chargeCounts, { 'tomba-request': 1 });723 });724
725 it('keeps serving after a request stops at maxEmailsPerDomain', async () => {726 const server = await mock();727 const actor = await startStandbyActor({ endpoint: server.url });728 try {729 const first = await actor.call('/?domain=a.com&maxEmailsPerDomain=5');730 const [a] = first.body.items as Record<string, unknown>[];731 assert.equal((a.emails as unknown[]).length, 5);732 const second = await actor.call('/?domains=b.com,c.com&maxEmailsPerDomain=15');733 const items = second.body.items as Record<string, unknown>[];734 assert.deepEqual(735 items.map((i) => (i.emails as unknown[]).length),736 [15, 15],737 );738 } finally {739 await actor.stop();740 }741 });742
743 it('rejects invalid input with 400 and unknown paths with 404', async () => {744 const server = await mock();745 const actor = await startStandbyActor({ endpoint: server.url });746 try {747 assert.equal((await actor.call('/', { body: {} })).status, 400);748 assert.equal((await actor.call('/', { body: 'not json' })).status, 400);749 assert.equal((await actor.call('/?page=abc&domain=a.com')).status, 400);750 assert.equal((await actor.call('/?enrichMobile=maybe&domain=a.com')).status, 400);751 assert.equal((await actor.call('/?webhookUrl=ftp://x&domain=a.com')).status, 400);752 assert.equal((await actor.call('/nope')).status, 404);753 assert.equal((await actor.call('/', { method: 'DELETE' })).status, 405);754 assert.equal(server.requests.length, 0);755 } finally {756 await actor.stop();757 }758 });759
760 it('returns 402 once the max charge limit is reached', async () => {761 const server = await mock();762 const actor = await startStandbyActor({ endpoint: server.url, maxTotalChargeUsd: 1 });763 try {764 const first = await actor.call('/?domain=a.com');765 assert.equal(first.status, 200);766 const second = await actor.call('/?domain=b.com');767 assert.equal(second.status, 402);768 assert.equal(server.requests.length, 1);769 } finally {770 await actor.stop();771 }772 });773});774
775describe('input schema', () => {776 it('accepts multi-part domains and lowercase country codes', () => {777 assert.deepEqual(inputSchemaErrors({ domains: ['bbc.co.uk', 'https://blog.stripe.com/'], country: 'gb' }), []);778 });779});name: Test and push latest version
on: push: branches: - main - master pull_request: workflow_dispatch:
concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true
jobs: test: name: Format, lint, build and test runs-on: ubuntu-latest steps: - uses: actions/checkout@v4
- name: Set up Node.js uses: actions/setup-node@v4 with: node-version: 22 cache: npm
- name: Install dependencies run: npm ci
- name: Check formatting run: npm run format:check
- name: Lint run: npm run lint
- name: Build run: npm run build
- name: Test run: npm test
push: name: Push Actor to Apify needs: test # Only publish from the default branch, never from pull requests. if: github.event_name != 'pull_request' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master') runs-on: ubuntu-latest steps: - uses: actions/checkout@v4
# .actor/actor.json maps TOMBA_API_KEY / TOMBA_API_SECRET to the Apify CLI secrets # @tombaApiKey / @tombaApiSecret, which `apify push` reads from ~/.apify/secrets.json. - name: Provide Tomba secrets to the Apify CLI env: TOMBA_API_KEY: ${{ secrets.TOMBA_API_KEY }} TOMBA_API_SECRET: ${{ secrets.TOMBA_API_SECRET }} run: | if [ -z "$TOMBA_API_KEY" ] || [ -z "$TOMBA_API_SECRET" ]; then echo "::error::Add the TOMBA_API_KEY and TOMBA_API_SECRET repository secrets." exit 1 fi mkdir -p ~/.apify node -e 'require("fs").writeFileSync(require("os").homedir() + "/.apify/secrets.json", JSON.stringify({ tombaApiKey: process.env.TOMBA_API_KEY, tombaApiSecret: process.env.TOMBA_API_SECRET }))'
- name: Push Actor to Apify id: push uses: apify/push-actor-action@v1 with: token: ${{ secrets.APIFY_TOKEN }}
- name: Print build details run: | echo "Build ${{ steps.push.outputs.build-id }} finished with status ${{ steps.push.outputs.build-status }}"