Pricing

Pay per usage

Superclean Job Titles

Clean messy job titles instantly. AI fixes abbreviations (Sr. → Senior), removes junk (Ninja, Rockstar), strips department tags, and standardizes formats. 3 output styles for CRM, spreadsheets, or analytics. Better data means better lead scoring and personalization.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Superlative

Actor stats

Bookmarked

Total users

Monthly active users

24 days ago

Last modified

Superclean Places

superlativetech/superclean-places

Clean messy location data from lead exports. Normalizes city, state, and country fields. Fixes abbreviations (NYC → New York, Calif → CA), standardizes formats, and parses combined fields. Works with US states, Canadian provinces, and international locations.

Superlative

Superclean Company Names

superlativetech/superclean-company-names

Transform messy company names into clean formats for outreach. Removes legal suffixes (Inc, LLC, Ltd), fixes ALL CAPS, preserves brand identity. Perfect for cold email personalization, CRM cleanup, and lead enrichment. Supports casual, formal, and legal output styles. LLM-powered normalizer.

Superlative

Superclean Person Names

superlativetech/superclean-person-names

Clean messy person names for cold email and CRM. Extracts first names for personalization, fixes ALL CAPS, handles McDonald/O'Brien/van der Berg correctly, strips titles (Dr.) and credentials (PhD). Three styles: first name only, casual, or formal.

Superlative

Superclean Product Names

superlativetech/superclean-product-names

Clean messy product names from e-commerce exports and web scraping. AI removes promo text, prices, SKUs, and formatting noise. Three styles: Clean (core name only), Full (with attributes), Searchable (lowercase for dedup). Works with Shopify, Amazon, and any catalog data.

Superlative

Superclean URLs

superlativetech/superclean-urls

Clean messy URLs from lead exports. Remove 60+ tracking parameters (utm_*, fbclid, gclid), normalize format, extract domains, and optionally verify URLs are reachable. Perfect for cold email personalization and CRM data hygiene.

Superlative

TableTidy — Messy CSV to Clean Typed Data (MCP)

beacon_labs/tabletidy-mcp

Agent-callable MCP server that turns messy CSV/spreadsheets into clean, typed, structured data: header detection, type inference, coercion, junk-row removal.

Charles Doherty

Bbb Better Business Bureau Scraper

jackwenyoung/bbb-better-business-bureau-scraper

Scraping bbb.org (Better Business Bureau) data as you wish.

Jack Wenyoung

Bbb Scraper with Contacts | Better Business Bureau

fatihtahta/bbb-scraper

Extract structured Better Business Bureau business profiles including emails, social medias, ratings, locations, reviews, complaints and more. Built for due diligence, reputation monitoring, lead enrichment, and production-grade analytics pipelines.

Fatih Tahta

Scraper for Facebook Marketplace

bkktor/scraper-for-facebook-marketplace

Better scraper for FB Marketplace

Tor

The Better Traders Blog Scraper

yourapiservice/thebettertraders-blog-scraper

The Better Traders Blog Scraper (thebettertraders.com) lets you extract blog content in HTML, JSON, and plaintext. Get authors, create/update date, images, read time, RSS, titles, SEO titles, featured images & videos, and keywords easily for content analysis and aggregation.

Your API Service

{ "name": "superclean-job-titles", "version": "0.1.0", "type": "module", "description": "Transform messy job titles into clean, consistent formats for CRM systems, HR databases, and analytics.", "engines": { "node": ">=18.0.0" }, "dependencies": { "apify": "^3.5.2", "openai": "^4.0.0" }, "scripts": { "start": "node ./src/main.js", "test": "node --test tests/*.test.js" }, "author": "Superlative", "license": "ISC" }

1# Superclean Job Titles — Methodology 2 3A two-stage pipeline that normalizes free-text job titles into a consistent form. Stage 1 applies deterministic, rule-based transformations. Stage 2 applies a large language model (LLM) to resolve semantic ambiguity (abbreviation expansion, typo correction, level interpretation). 4 5This document describes version 0.2.16 of the Superclean Job Titles tool (Apify Actor `superlativetech/superclean-job-titles`, accessible at https://apify.com/superlativetech/superclean-job-titles). 6 7## Pipeline Overview 8 9For each input title, the system performs: 10 111. **Deterministic preprocessing** — fully replicable, rule-based string transformations. 122. **Language detection** — non-Latin-script titles are flagged and not sent to the LLM. 133. **LLM normalization** — semantic cleanup according to a fixed system prompt and selected output style. 144. **Post-processing** — acronym case enforcement and whitespace cleanup. 15 16Each output record carries a confidence score (0–1) for downstream filtering. 17 18## Stage 1: Deterministic Preprocessing 19 20Applied to every title before the LLM stage. Identical input always produces identical preprocessed output. 21 22- **Symbol/emoji removal** — strips Unicode emoji ranges (U+1F300–U+1F9FF, U+2600–U+27BF, U+1F600–U+1F6FF) and decorative glyphs (★, ✓, →, etc.). 23- **Character normalization** — maps smart quotes, en/em dashes, non-breaking spaces, ellipses, and middle dots to ASCII equivalents. 24- **Whitespace normalization** — collapses runs of whitespace and trims edges. 25- **Vanity/junk word removal** — case-insensitive removal of a fixed list: *ninja, rockstar, guru, wizard, unicorn, hero, hacker, evangelist, champion, sherpa, czar, maven, virtuoso, jedi, samurai, superstar, black belt, passionate, dynamic, innovative, visionary, thought leader, disruptor, change agent*. 26- **Department/team tag stripping** — removes trailing department descriptors matching three regex patterns (e.g., "Engineer - Platform Team", "Manager (Sales Division)", "Analyst, Finance Department"). 27- **Title-case normalization** — strings that are entirely upper- or lower-case are converted to title case, with known acronyms (see below) preserved in upper case. 28- **Seniority prefix canonicalization** — leading abbreviations are mapped to canonical forms (e.g., `Sr.`, `Snr` → `Senior`; `Jr.`, `Jnr` → `Junior`; `Asst.` → `Assistant`). 29- **Acronym preservation** — a fixed set of role acronyms is kept in upper case throughout: CEO, CFO, CTO, COO, CMO, CIO, CISO, CPO, CRO, CLO, CHRO, CDO, CSO, CCO, VP, SVP, EVP, AVP, GVP, PM, TPM, EM, IC, SWE, SDE, MTS, QA, QE, SDET, DBA, DevOps, SRE, MLOps, UX, UI, UXR, HR, PR, IR, BD, BDR, SDR, AE, AM, CSM, TAM, IT, IS, BA, SA, TA, RA, PA, EA, VA, MD, DO, RN, LPN, CPA, CFA, JD, PhD. 30 31## Stage 2: LLM Normalization 32 33Preprocessed titles are sent to an LLM in batches of 10. The model is instructed via a fixed system prompt that defines: 34 35- A test of fitness ("does this look right as a LinkedIn headline / spreadsheet cell / role-category filter?"). 36- Rules for preserving role acronyms, fixing obvious typos (e.g., "Sofware" → "Software"), normalizing international spelling ("Programme" → "Program"), and handling compound titles ("Developer / Designer"). 37- Style-specific behavior — one of three styles selected by the user: 38 - **Standardized** — full, professional titles. Expands generic abbreviations ("Eng." → "Engineer") but preserves recognized acronyms (VP stays VP, not "Vice President"). Strips departmental qualifiers when they are not the role's domain. 39 - **Abbreviated** — compact form for constrained displays (e.g., "Senior Software Engineer" → "Sr. Software Eng."). 40 - **Simple** — core job function only, with all seniority and level indicators removed (e.g., "Senior Software Engineer" → "Software Engineer"; "VP of Engineering" → "Engineering"). Intended for role-based categorization. 41 42Inference parameters: 43 44- **Temperature**: 0.3 (low, for consistent output across runs). 45- **Max output tokens**: 2,048. 46- **Output format**: strict JSON array; each element contains `index`, `normalized`, and `confidence`. 47- **Retry policy**: up to 3 attempts with exponential backoff for transient failures (timeouts, 429 rate limits, 5xx errors). On exhausted retries, the preprocessed value is passed through with confidence 0. 48 49By default the system uses OpenRouter's automatic model router (`openrouter/auto`). For research use, **a specific model should be pinned** (see Reproducibility below). 50 51## Stage 3: Post-Processing 52 53After the LLM returns, three deterministic fixes are applied: 54 55- Apostrophe-S capitalization is corrected (e.g., "Stone'S" → "Stone's"). 56- All known acronyms are forced to upper case. 57- Whitespace is collapsed. 58 59## Output Schema 60 61Each input record produces one output record: 62 63| Field | Type | Description | 64|---|---|---| 65| `id` | integer | 1-based row number, preserves input order | 66| `input` | string | Original title before any processing | 67| `output` | string | Cleaned title | 68| `confidence` | float | 0.0–1.0; quality estimate from the LLM | 69 70## Confidence Scoring 71 72The LLM emits a self-reported confidence score for each title: 73 74- **≥ 0.9** — high confidence; minimal review needed. 75- **0.7–0.9** — moderate; spot-check recommended. 76- **< 0.7** — low; manual review suggested. 77- **0.0** — non-English/non-Latin-script titles (skipped LLM normalization), batch-level LLM failures, or post-retry pass-through. 78 79Researchers are advised to filter or manually review records with confidence < 0.7. 80 81## Non-English Handling 82 83Titles containing non-Latin scripts (CJK, Arabic, Hebrew, Cyrillic, Thai) are detected via Unicode range matching and **skipped at the LLM stage**. If a Latin-script substring of ≥ 2 characters can be extracted, it is returned as the output; otherwise the original input is returned. All such records receive confidence 0. 84 85## Reproducibility Considerations 86 87The deterministic preprocessing stage is fully replicable. The LLM stage is approximately, but not bit-exactly, replicable. For publishable research: 88 89- **Pin the model.** Do not use `openrouter/auto`. Specify a dated model snapshot (e.g., `openai/gpt-4o-mini-2024-07-18`) via the `model` input parameter and report the exact identifier in your methods. 90- **Pin the tool version.** Cite the Apify Actor build number used (visible on each run in the Apify Console). 91- **Report inter-run agreement.** Because LLMs are not bit-exact deterministic even at low temperature, run a representative sample (e.g., 1,000 titles) twice and report agreement rate (exact-match %) or Cohen's kappa. 92- **Preserve raw outputs.** Publish the full output table — `input`, `output`, `confidence` — alongside any analyses, so reviewers can inspect cleaning decisions. 93- **Document manual-review thresholds.** State the confidence threshold used to flag records for human adjudication, and report the number of records adjudicated.

{ "actorSpecification": 1, "fields": { "type": "object", "properties": { "id": { "type": "integer", "description": "Row number (1-based)" }, "input": { "type": "string", "description": "The original input value" }, "output": { "type": "string", "description": "The cleaned output value" }, "confidence": { "type": "number", "description": "Confidence score from 0 to 1" } } }, "views": { "overview": { "title": "All Results", "transformation": { "fields": ["id", "input", "output", "confidence"] }, "display": { "component": "table", "properties": { "id": { "label": "ID", "format": "number" }, "input": { "label": "Input", "format": "text" }, "output": { "label": "Output", "format": "text" }, "confidence": { "label": "Confidence", "format": "number" } } } } } }

{ "actorOutputSchemaVersion": 1, "title": "Superclean Job Titles Output", "description": "Normalized results with original values, cleaned values, and confidence scores.", "properties": { "allResults": { "type": "string", "title": "All Results", "description": "All normalized titles with before/after values and confidence scores", "template": "{{links.apiDefaultDatasetUrl}}/items" } } }

{ "openapi": "3.0.3", "info": { "title": "Superclean Job Titles — Standby API", "description": "Instant AI-powered job title normalization via HTTP GET. Standardize messy job titles for CRM, HR systems, and analytics. Supports standardized, abbreviated, and simple styles.", "version": "0.2.0" }, "paths": { "/": { "get": { "summary": "Clean a job title", "description": "Normalize a single job title using AI. Expands abbreviations, removes team/department info, standardizes format. Supports BYOK via X-OpenRouter-Key header.", "operationId": "cleanJobTitle", "parameters": [ { "name": "input", "in": "query", "description": "Job title to clean (e.g. `Sr. Software Eng. - Platform Team`).", "required": true, "schema": { "type": "string", "example": "Sr. Software Eng. - Platform Team" } }, { "name": "style", "in": "query", "description": "Output style. `standardized` expands abbreviations, `abbreviated` keeps them, `simple` removes seniority.", "required": false, "schema": { "type": "string", "enum": ["standardized", "abbreviated", "simple"], "default": "standardized" } }, { "name": "model", "in": "query", "description": "LLM model to use.", "required": false, "schema": { "type": "string", "default": "openrouter/auto" } } ], "responses": { "200": { "description": "Cleaned job title with confidence score.", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/CleanedName" }, "examples": { "standardized": { "summary": "Standardized style", "value": { "id": 1, "input": "Sr. Software Eng. - Platform Team", "output": "Senior Software Engineer", "confidence": 0.9 } }, "simple": { "summary": "Simple style", "value": { "id": 1, "input": "Sr. Software Eng. - Platform Team", "output": "Software Engineer", "confidence": 0.9 } } } } } }, "400": { "description": "Invalid request — missing input or invalid style.", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" }, "example": { "error": "Provide an 'input' query parameter with the job title to clean." } } } }, "405": { "description": "Method not allowed — only GET is supported.", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" }, "example": { "error": "Method not allowed. Use GET." } } } } } } } }, "components": { "schemas": { "CleanedName": { "type": "object", "required": ["id", "input", "output", "confidence"], "properties": { "id": { "type": "integer", "description": "Row number (always 1 for single-item Standby requests)." }, "input": { "type": "string", "description": "Original job title before cleaning." }, "output": { "type": "string", "description": "Cleaned job title." }, "confidence": { "type": "number", "description": "Confidence score from 0 to 1." } } }, "Error": { "type": "object", "required": ["error"], "properties": { "error": { "type": "string", "description": "Human-readable error message." } } } } } }

{ "title": "Superclean Job Titles", "description": "Paste your job titles below and click Start. Results will appear in the Output tab.", "type": "object", "schemaVersion": 1, "properties": { "items": { "title": "Job titles", "type": "array", "description": "Enter job titles to clean, one per line.", "editor": "stringList", "prefill": ["Sr. Software Eng."], "minItems": 1 }, "style": { "title": "Output style", "type": "string", "description": "Choose how job titles should be formatted in the output.", "enum": ["standardized", "abbreviated", "simple"], "enumTitles": [ "Standardized - full professional titles (Senior Software Engineer)", "Abbreviated - compact form (Sr. Software Eng.)", "Simple - core role only, no seniority (Software Engineer)" ], "default": "standardized", "editor": "select" }, "model": { "sectionCaption": "Advanced", "title": "AI model", "type": "string", "description": "AI model for normalization. Default auto-routes for best price/performance. Set a specific model like google/gemini-2.0-flash-001 for higher-quality confidence scores. Browse models at openrouter.ai/models.", "default": "openrouter/auto", "editor": "textfield" }, "openRouterApiKey": { "title": "OpenRouter API key (bring your own)", "type": "string", "description": "Optional. Provide your own OpenRouter API key to use the direct OpenRouter API instead of the built-in Apify integration. Get your key at openrouter.ai.", "editor": "textfield", "isSecret": true }, "item": { "title": "Item (single)", "type": "string", "description": "Single job title to clean — shorthand for API/integration callers. If both 'item' and 'items' are provided, 'item' is prepended to the list.", "editor": "textfield", "sectionCaption": "API shorthand", "sectionDescription": "For API and integration callers (Clay, Make, n8n) who want to pass a single item without wrapping it in an array." } }, "required": [] }

{ "actorSpecification": 1, "name": "superclean-job-titles", "title": "Superclean Job Titles", "description": "Normalize messy job titles. AI transforms 'Sr. Software Eng. - Platform Team' to 'Senior Software Engineer'. Input: string array. Output: {input, output, confidence}. Batch or instant API.", "version": "0.2", "buildTag": "latest", "usesStandbyMode": true, "webServerSchema": "./web_server_openapi.json", "dockerfile": "../Dockerfile", "input": "./input_schema.json", "output": "./output_schema.json", "storages": { "dataset": "./dataset_schema.json" }, "defaultRunOptions": { "build": "latest", "memoryMbytes": 256, "timeoutSecs": 360 }, "minMemoryMbytes": 256, "maxMemoryMbytes": 1024 }

1/** 2 * Constants for job title normalization patterns. 3 */ 4 5// Seniority prefix mappings (variations -> canonical form) 6export const SENIORITY_PREFIXES = { 7 // Senior variations 8 'sr': 'Senior', 9 'sr.': 'Senior', 10 'snr': 'Senior', 11 'snr.': 'Senior', 12 'senior': 'Senior', 13 // Junior variations 14 'jr': 'Junior', 15 'jr.': 'Junior', 16 'jnr': 'Junior', 17 'jnr.': 'Junior', 18 'junior': 'Junior', 19 // Other levels 20 'lead': 'Lead', 21 'principal': 'Principal', 22 'staff': 'Staff', 23 'chief': 'Chief', 24 'head': 'Head', 25 'associate': 'Associate', 26 'asst': 'Assistant', 27 'asst.': 'Assistant', 28 'assistant': 'Assistant', 29 'intern': 'Intern', 30 'trainee': 'Trainee', 31 'entry': 'Entry Level', 32 'entry-level': 'Entry Level', 33}; 34 35// Junk/vanity words to remove during preprocessing 36export const JUNK_WORDS = [ 37 // Vanity titles 38 'ninja', 39 'rockstar', 40 'guru', 41 'wizard', 42 'unicorn', 43 'hero', 44 'hacker', 45 'evangelist', 46 'champion', 47 'sherpa', 48 'czar', 49 'maven', 50 'virtuoso', 51 'jedi', 52 'samurai', 53 'superstar', 54 'black belt', 55 // Buzzwords 56 'passionate', 57 'dynamic', 58 'innovative', 59 'visionary', 60 'thought leader', 61 'disruptor', 62 'change agent', 63]; 64 65// Title acronyms to preserve (case-sensitive after normalization) 66export const TITLE_ACRONYMS = new Set([ 67 // C-Suite 68 'CEO', 'CFO', 'CTO', 'COO', 'CMO', 'CIO', 'CISO', 'CPO', 'CRO', 'CLO', 69 'CHRO', 'CDO', 'CSO', 'CCO', 70 // VP levels 71 'VP', 'SVP', 'EVP', 'AVP', 'GVP', 72 // Technical 73 'PM', 'TPM', 'EM', 'IC', 'SWE', 'SDE', 'MTS', 74 'QA', 'QE', 'SDET', 'DBA', 'DevOps', 'SRE', 'MLOps', 75 // Design 76 'UX', 'UI', 'UXR', 77 // Business 78 'HR', 'PR', 'IR', 'BD', 'BDR', 'SDR', 'AE', 'AM', 'CSM', 'TAM', 79 // Other 80 'IT', 'IS', 'BA', 'SA', 'TA', 'RA', 'PA', 'EA', 'VA', 81 'MD', 'DO', 'RN', 'LPN', 'CPA', 'CFA', 'JD', 'PhD', 82]); 83 84// Common abbreviation expansions (for standardized style) 85export const ABBREVIATION_EXPANSIONS = { 86 'dev': 'Developer', 87 'devs': 'Developers', 88 'swe': 'Software Engineer', 89 'sde': 'Software Development Engineer', 90 'eng': 'Engineer', 91 'eng.': 'Engineer', 92 'engr': 'Engineer', 93 'engr.': 'Engineer', 94 'mgr': 'Manager', 95 'mgr.': 'Manager', 96 'mngr': 'Manager', 97 'mngr.': 'Manager', 98 'dir': 'Director', 99 'dir.': 'Director', 100 'admin': 'Administrator', 101 'exec': 'Executive', 102 'coord': 'Coordinator', 103 'coord.': 'Coordinator', 104 'rep': 'Representative', 105 'spec': 'Specialist', 106 'tech': 'Technician', 107 'acct': 'Accountant', 108 'ops': 'Operations', 109}; 110 111// Standard abbreviations (for abbreviated style) 112export const STANDARD_ABBREVIATIONS = { 113 'Senior': 'Sr.', 114 'Junior': 'Jr.', 115 'Manager': 'Mgr.', 116 'Director': 'Dir.', 117 'Engineer': 'Eng.', 118 'Developer': 'Dev.', 119 'Administrator': 'Admin.', 120 'Coordinator': 'Coord.', 121 'Representative': 'Rep.', 122 'Specialist': 'Spec.', 123 'Assistant': 'Asst.', 124 'Associate': 'Assoc.', 125 'Executive': 'Exec.', 126 'President': 'Pres.', 127 'Vice President': 'VP', 128}; 129 130// Department/team patterns to strip 131export const DEPARTMENT_PATTERNS = [ 132 /\s*[-–—@]\s*[\w\s]+(?:Team|Department|Division|Dept\.?|Group|Unit|Org)$/i, 133 /\s*$[\w\s]*(?:Team|Department|Division|Dept\.?|Group|Unit|Org)[\w\s]*$$/i, 134 /\s*,\s*[\w\s]+(?:Team|Department|Division|Dept\.?)$/i, 135]; 136 137// Special characters to normalize to ASCII equivalents 138export const NORMALIZE_CHARS = { 139 '\u2018': "'", // Left single quote 140 '\u2019': "'", // Right single quote 141 '\u201C': '"', // Left double quote 142 '\u201D': '"', // Right double quote 143 '\u2013': '-', // En dash 144 '\u2014': '-', // Em dash 145 '\u00A0': ' ', // Non-breaking space 146 '\u2026': '...', // Ellipsis 147 '\u00B7': '.', // Middle dot 148}; 149 150// Emoji and symbol patterns to remove 151export const SYMBOL_PATTERN = /[\u{1F300}-\u{1F9FF}\u{2600}-\u{26FF}\u{2700}-\u{27BF}\u{1F600}-\u{1F64F}\u{1F680}-\u{1F6FF}®™©★☆✨✓✔✗✘●○◆◇→←↑↓]/gu; 152 153// Non-Latin script detection (same as company names) 154export const NON_LATIN_PATTERN = /[\u4E00-\u9FFF\u3400-\u4DBF\u3000-\u303F\u3040-\u309F\u30A0-\u30FF\uAC00-\uD7AF\u0600-\u06FF\u0590-\u05FF\u0400-\u04FF\u0E00-\u0E7F]/;

1/** 2 * Rule-based pre-processing for job title normalization. 3 * Applies deterministic transformations before LLM processing. 4 */ 5 6import { 7 SENIORITY_PREFIXES, 8 JUNK_WORDS, 9 TITLE_ACRONYMS, 10 DEPARTMENT_PATTERNS, 11 NORMALIZE_CHARS, 12 SYMBOL_PATTERN, 13 NON_LATIN_PATTERN, 14} from './constants.js'; 15 16/** 17 * Check if a string contains non-Latin script characters (CJK, Arabic, Cyrillic, etc.) 18 * @param {string} str - String to check 19 * @returns {boolean} True if contains non-Latin scripts 20 */ 21function containsNonLatinScript(str) { 22 return NON_LATIN_PATTERN.test(str); 23} 24 25/** 26 * Extract the English (Latin script) portion from a mixed-language string. 27 * @param {string} str - Mixed-language string 28 * @returns {string|null} English portion, or null if no substantial English found 29 */ 30function extractEnglishPortion(str) { 31 if (!str) return null; 32 33 const latinPattern = /[A-Za-z0-9][A-Za-z0-9\s.,&'"-]*/g; 34 const matches = str.match(latinPattern); 35 if (!matches) return null; 36 37 let english = matches.join(' ').trim(); 38 english = english.replace(/^[.,\s-]+|[.,\s-]+$/g, '').replace(/\s+/g, ' ').trim(); 39 40 if (english.length >= 2) { 41 return english; 42 } 43 44 return null; 45} 46 47/** 48 * Pre-process a job title value. 49 * @param {string} value - The raw input value 50 * @returns {Object} Object with preprocessed value, metadata, and detected seniority 51 */ 52export function preprocess(value) { 53 if (!value || typeof value !== 'string') { 54 return { value: '', isNonEnglish: false, extractedEnglish: null, detectedSeniority: null }; 55 } 56 57 let result = value; 58 59 // Check for non-Latin scripts before processing 60 const isNonEnglish = containsNonLatinScript(result); 61 62 // For non-English strings, try to extract the English portion 63 let extractedEnglish = null; 64 if (isNonEnglish) { 65 extractedEnglish = extractEnglishPortion(result); 66 } 67 68 // Step 1: Remove emojis and symbols 69 result = result.replace(SYMBOL_PATTERN, '').trim(); 70 71 // Step 2: Normalize special characters to ASCII (smart quotes, dashes, etc.) 72 for (const [from, to] of Object.entries(NORMALIZE_CHARS)) { 73 result = result.split(from).join(to); 74 } 75 76 // Step 3: Trim and collapse whitespace 77 result = result.trim().replace(/\s+/g, ' '); 78 79 // Step 4: Job title-specific processing (only if English) 80 let detectedSeniority = null; 81 if (!isNonEnglish) { 82 const processed = preprocessJobTitle(result); 83 result = processed.value; 84 detectedSeniority = processed.detectedSeniority; 85 } 86 87 return { value: result, isNonEnglish, extractedEnglish, detectedSeniority }; 88} 89 90/** 91 * Pre-process job titles by removing junk words, department tags, and normalizing. 92 * @param {string} value - Job title 93 * @returns {Object} Object with cleaned job title and detected seniority 94 */ 95function preprocessJobTitle(value) { 96 let result = value; 97 let detectedSeniority = null; 98 99 // Check if original needs case normalization BEFORE other processing 100 // (seniority normalization changes the case, so we need to check first) 101 const needsCaseNormalization = result.length > 0 102 && (result === result.toUpperCase() || result === result.toLowerCase()); 103 104 // Step 1: Remove junk/vanity words 105 const junkPattern = new RegExp(`\\b(${JUNK_WORDS.join('|')})\\b`, 'gi'); 106 result = result.replace(junkPattern, '').trim(); 107 108 // Step 2: Remove department/team tags 109 for (const pattern of DEPARTMENT_PATTERNS) { 110 result = result.replace(pattern, '').trim(); 111 } 112 113 // Step 3: Apply title case normalization if needed (before seniority to fix all words) 114 if (needsCaseNormalization) { 115 result = toTitleCase(result); 116 } 117 118 // Step 4: Normalize seniority prefixes 119 const seniorityResult = normalizeSeniority(result); 120 result = seniorityResult.value; 121 detectedSeniority = seniorityResult.seniority; 122 123 // Step 5: Clean up multiple spaces and trim 124 result = result.replace(/\s+/g, ' ').trim(); 125 126 return { value: result, detectedSeniority }; 127} 128 129/** 130 * Normalize seniority prefixes to standard form. 131 * @param {string} str - Job title string 132 * @returns {Object} Object with normalized value and detected seniority 133 */ 134function normalizeSeniority(str) { 135 let result = str; 136 let seniority = null; 137 138 // Check each seniority prefix at the start of the string 139 for (const [abbrev, canonical] of Object.entries(SENIORITY_PREFIXES)) { 140 // Match at start of string, case-insensitive, followed by space or end 141 const pattern = new RegExp(`^${escapeRegex(abbrev)}(?:\\s+|$)`, 'i'); 142 if (pattern.test(result)) { 143 seniority = canonical; 144 // Replace with canonical form 145 result = result.replace(pattern, `${canonical} `).trim(); 146 break; 147 } 148 } 149 150 return { value: result, seniority }; 151} 152 153/** 154 * Convert a string to title case, preserving known acronyms. 155 * @param {string} str - String to convert 156 * @returns {string} Title-cased string 157 */ 158function toTitleCase(str) { 159 return str.replace(/\b\w+\b/g, (word) => { 160 const upper = word.toUpperCase(); 161 162 // Preserve known title acronyms 163 if (TITLE_ACRONYMS.has(upper)) { 164 return upper; 165 } 166 167 // Handle hyphenated words (e.g., "FULL-STACK" -> "Full-Stack") 168 if (word.includes('-')) { 169 return word 170 .split('-') 171 .map((part) => { 172 const partUpper = part.toUpperCase(); 173 if (TITLE_ACRONYMS.has(partUpper)) { 174 return partUpper; 175 } 176 return part.charAt(0).toUpperCase() + part.slice(1).toLowerCase(); 177 }) 178 .join('-'); 179 } 180 181 // Handle words with slashes (e.g., "UX/UI" -> "UX/UI") 182 if (word.includes('/')) { 183 return word 184 .split('/') 185 .map((part) => { 186 const partUpper = part.toUpperCase(); 187 if (TITLE_ACRONYMS.has(partUpper)) { 188 return partUpper; 189 } 190 return part.charAt(0).toUpperCase() + part.slice(1).toLowerCase(); 191 }) 192 .join('/'); 193 } 194 195 // Standard title case 196 return word.charAt(0).toUpperCase() + word.slice(1).toLowerCase(); 197 }); 198} 199 200/** 201 * Escape special regex characters in a string. 202 * @param {string} string - String to escape 203 * @returns {string} Escaped string safe for use in RegExp 204 */ 205function escapeRegex(string) { 206 return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); 207}

1/** 2 * LLM-based semantic normalization for job titles using Apify's OpenRouter standby actor. 3 * @see https://apify.com/apify/openrouter 4 */ 5 6import { log } from 'apify'; 7import OpenAI from 'openai'; 8 9import { TITLE_ACRONYMS } from './constants.js'; 10 11const BATCH_SIZE = 10; // Process items in batches (kept small to avoid truncated LLM responses) 12const MAX_RETRIES = 3; // Maximum retry attempts for API calls 13const INITIAL_RETRY_DELAY_MS = 1000; // Start with 1 second delay 14const DEFAULT_CONCURRENCY = 5; // Default parallel batch limit 15const LARGE_REQUEST_THRESHOLD = 100; // Items threshold for increased concurrency 16const LARGE_REQUEST_CONCURRENCY = 10; // Concurrency for large requests 17const MAX_TOKENS = 2048; // Ensure sufficient output tokens for complete JSON responses 18 19// Apify's OpenRouter standby actor endpoint 20const OPENROUTER_BASE_URL = 'https://openrouter.apify.actor/api/v1'; 21// Direct OpenRouter API endpoint (bring your own key) 22const OPENROUTER_DIRECT_URL = 'https://openrouter.ai/api/v1'; 23 24/** 25 * Style-specific prompt modifiers for standardized, abbreviated, and simple output. 26 */ 27const STYLE_MODIFIERS = { 28 standardized: `Normalize to STANDARD industry job titles: 29 - Expand common abbreviations: "Sr." -> "Senior", "Jr." -> "Junior", "Eng." -> "Engineer", "Dev" -> "Developer", "Mgr" -> "Manager" 30 - KEEP widely-recognized title acronyms as-is (do NOT expand): 31 * VP, SVP, EVP, AVP (not "Vice President") 32 * CEO, CFO, CTO, COO, CMO, CIO, CPO, CRO (not "Chief X Officer") 33 * PM, TPM, EM (not "Product Manager" etc. - unless already spelled out) 34 - Standardize seniority: Keep seniority prefixes (Senior, Junior, Lead, Principal, Staff) 35 - Preserve level indicators: "II", "III", "Level 2" stay as-is 36 - Remove junk/vanity words: "Ninja", "Rockstar", "Guru" -> remove entirely 37 - Keep department context only if essential to the role: 38 * "Marketing Manager" stays (Marketing is the domain) 39 * "Software Engineer - Marketing Team" -> "Software Engineer" (team is just location) 40 - Fix obvious typos: "Sofware" -> "Software", "Manger" -> "Manager" 41 - Handle compound titles: "Developer / Designer" -> keep both roles 42 - Examples: 43 * "Sr. Software Eng." -> "Senior Software Engineer" 44 * "Head of Eng" -> "Head of Engineering" 45 * "VP, Product" -> "VP of Product" (keep VP, don't expand) 46 * "vp of engineering" -> "VP of Engineering" (uppercase VP) 47 * "ceo" -> "CEO" 48 * "Full Stack Dev" -> "Full Stack Developer" 49 * "Jr. UX/UI Designer" -> "Junior UX/UI Designer" 50 * "Mktg Mgr" -> "Marketing Manager"`, 51 52 abbreviated: `Create SHORT, abbreviated job titles: 53 - Use standard abbreviations: "Senior" -> "Sr.", "Junior" -> "Jr.", "Manager" -> "Mgr." 54 - Shorten common words: "Engineer" -> "Eng.", "Developer" -> "Dev.", "Director" -> "Dir." 55 - Keep role recognizable but compact 56 - Preserve essential qualifiers only 57 - Use periods after abbreviations: "Sr." not "Sr" 58 - Keep acronyms as-is: CEO, CTO, VP, PM, UX, UI, QA 59 - Examples: 60 * "Senior Software Engineer" -> "Sr. Software Eng." 61 * "Vice President of Engineering" -> "VP Eng." 62 * "Head of Product Management" -> "Head of PM" 63 * "Junior User Experience Designer" -> "Jr. UX Designer" 64 * "Chief Technology Officer" -> "CTO" 65 * "Marketing Manager" -> "Mktg. Mgr." 66 - Target: ~30 characters or less when reasonable`, 67 68 simple: `Extract ONLY the core job function, removing ALL modifiers: 69 - Remove ALL seniority: Senior, Junior, Lead, Principal, Staff, Head of 70 - Remove ALL levels: I, II, III, Level 1, Level 2 71 - Remove management level words unless management IS the core role 72 - Keep only the fundamental job function 73 - Examples: 74 * "Senior Software Engineer" -> "Software Engineer" 75 * "VP of Engineering" -> "Engineering" (VP is level, Engineering is function) 76 * "Lead Product Designer" -> "Product Designer" 77 * "Junior Data Scientist" -> "Data Scientist" 78 * "Staff Software Engineer" -> "Software Engineer" 79 * "Head of Marketing" -> "Marketing" 80 * "Engineering Manager" -> "Engineering Manager" (Manager IS the role here) 81 * "CEO" -> "Executive" (C-level reduces to executive function) 82 * "SDR" -> "Sales Development" 83 - Purpose: categorization - what KIND of work, not what LEVEL`, 84}; 85 86/** 87 * Get the goal description for each style. 88 */ 89const STYLE_GOALS = { 90 standardized: 'Clean, professional, industry-standard job titles suitable for CRM, LinkedIn, and HR systems.', 91 abbreviated: 'Short, space-efficient titles for spreadsheets, reports, and constrained displays.', 92 simple: 'Core job function only, for categorization, analytics, and role-based grouping.', 93}; 94 95/** 96 * Get the test sentence for each style. 97 */ 98const STYLE_TESTS = { 99 standardized: 'LinkedIn headline: [your answer]', 100 abbreviated: 'Spreadsheet cell (limited width): [your answer]', 101 simple: 'Role category filter: [your answer]', 102}; 103 104/** 105 * Build the system prompt for job title normalization. 106 * @param {string} style - Output style ('standardized', 'abbreviated', or 'simple') 107 * @returns {string} System prompt 108 */ 109function buildSystemPrompt(style) { 110 const effectiveStyle = STYLE_MODIFIERS[style] ? style : 'standardized'; 111 112 return `You are normalizing job titles for professional data cleanup. 113 114GOAL: ${STYLE_GOALS[effectiveStyle]} 115 116RULES: 1171. PRESERVE role-specific acronyms: CEO, CTO, CFO, VP, SVP, PM, TPM, UX, UI, QA, HR, IT, etc. 1182. Handle compound titles: "Developer / Designer" -> keep both roles (format depends on style) 1193. Fix obvious typos: "Sofware" -> "Software", "Manger" -> "Manager", "Enginer" -> "Engineer" 1204. Handle international variations: "Programme Manager" (UK) -> "Program Manager" 1215. Self-employed/Freelance: extract the actual role, ignore employment status 1226. ${STYLE_MODIFIERS[effectiveStyle]} 123 124EDGE CASES: 125- "Founder & CEO" -> "Founder & CEO" (standardized), "Founder & CEO" (abbreviated), "Executive" (simple) 126- "Co-Founder" -> "Co-Founder" (keep hyphen) 127- "Individual Contributor" or "IC" -> keep role if specified, otherwise "Individual Contributor" 128- Vanity titles with no real role ("Chief Happiness Officer") -> interpret as closest real role or flag low confidence 129 130TEST: Before returning, mentally check: "${STYLE_TESTS[effectiveStyle]}" - does it look right? 131 132OUTPUT FORMAT (strict JSON, no comments): 133[{"index": 0, "normalized": "Software Engineer", "confidence": 0.95}] 134Return ONLY the JSON array with objects containing "index", "normalized", and "confidence" keys.`; 135} 136 137/** 138 * Create a normalizer instance configured with Apify's OpenRouter standby actor. 139 * Uses APIFY_TOKEN for authentication - no separate API key needed. 140 * @param {string} model - OpenRouter model to use 141 * @param {string} style - Output style ('standardized', 'abbreviated', or 'simple') 142 * @param {string|null} openRouterApiKey - Optional OpenRouter API key for direct API access (bring your own key) 143 * @returns {Object} Normalizer with normalize() method 144 */ 145export function createNormalizer(model, style = 'standardized', openRouterApiKey = null) { 146 let openai; 147 148 if (openRouterApiKey) { 149 log.info('Using direct OpenRouter API (bring your own key)'); 150 openai = new OpenAI({ 151 baseURL: OPENROUTER_DIRECT_URL, 152 apiKey: openRouterApiKey, 153 timeout: 120_000, 154 }); 155 } else { 156 const apifyToken = process.env.APIFY_TOKEN; 157 if (!apifyToken) { 158 throw new Error('APIFY_TOKEN environment variable is required for OpenRouter access'); 159 } 160 openai = new OpenAI({ 161 baseURL: OPENROUTER_BASE_URL, 162 apiKey: 'apify-openrouter', 163 timeout: 120_000, 164 defaultHeaders: { 165 Authorization: `Bearer ${apifyToken}`, 166 }, 167 }); 168 } 169 170 const systemPrompt = buildSystemPrompt(style); 171 172 return { 173 /** 174 * Normalize an array of items using the LLM. 175 * Automatically scales concurrency based on request size. 176 * @param {Array} items - Items with preProcessed values 177 * @returns {Promise<Array>} Items with normalized values, confidence, and reasoning 178 */ 179 async normalize(items) { 180 // Split items into batches 181 const batches = []; 182 for (let i = 0; i < items.length; i += BATCH_SIZE) { 183 batches.push({ 184 items: items.slice(i, i + BATCH_SIZE), 185 startIndex: i, 186 }); 187 } 188 189 const totalBatches = batches.length; 190 191 // Scale concurrency based on total request size 192 const concurrency = items.length >= LARGE_REQUEST_THRESHOLD 193 ? LARGE_REQUEST_CONCURRENCY 194 : DEFAULT_CONCURRENCY; 195 196 log.info(`Processing ${items.length} items in ${totalBatches} batches (concurrency: ${concurrency})`); 197 198 // Process batches in parallel with concurrency limit 199 const results = new Array(items.length); 200 for (let i = 0; i < batches.length; i += concurrency) { 201 const chunk = batches.slice(i, i + concurrency); 202 const chunkStart = i + 1; 203 const chunkEnd = Math.min(i + concurrency, totalBatches); 204 205 log.info(`Processing batches ${chunkStart}-${chunkEnd}/${totalBatches}`); 206 207 const chunkResults = await Promise.allSettled( 208 chunk.map((batch) => normalizeBatch(openai, model, systemPrompt, batch.items)), 209 ); 210 211 // Place results in correct positions 212 chunk.forEach((batch, idx) => { 213 const settled = chunkResults[idx]; 214 if (settled.status === 'fulfilled') { 215 settled.value.forEach((result, resultIdx) => { 216 results[batch.startIndex + resultIdx] = result; 217 }); 218 } else { 219 // Batch failed entirely — fill with pass-through results 220 log.error(`Batch ${i + idx + 1} failed: ${settled.reason?.message}`); 221 batch.items.forEach((item, resultIdx) => { 222 results[batch.startIndex + resultIdx] = { 223 output: item.preProcessed || item.original, 224 confidence: 0, 225 error: settled.reason?.message || 'Batch failed', 226 }; 227 }); 228 } 229 }); 230 } 231 232 return results; 233 }, 234 }; 235} 236 237/** 238 * Sleep for a given number of milliseconds. 239 * @param {number} ms - Milliseconds to sleep 240 * @returns {Promise<void>} 241 */ 242function sleep(ms) { 243 return new Promise((resolve) => { 244 setTimeout(resolve, ms); 245 }); 246} 247 248/** 249 * Check if an error is retryable (network issues, rate limits, server errors). 250 * @param {Error} error - The error to check 251 * @returns {boolean} True if the error is retryable 252 */ 253function isRetryableError(error) { 254 // Network errors 255 if (error.code === 'ECONNRESET' || error.code === 'ETIMEDOUT' || error.code === 'ENOTFOUND') { 256 return true; 257 } 258 // Rate limiting (429) or server errors (5xx) 259 if (error.status === 429 || error.status >= 500) { 260 return true; 261 } 262 // OpenAI/OpenRouter specific errors 263 if (error.message?.includes('timeout') || error.message?.includes('network')) { 264 return true; 265 } 266 return false; 267} 268 269/** 270 * Normalize a batch of items with a single LLM call. 271 * Includes retry logic with exponential backoff. 272 * @param {OpenAI} openai - OpenAI client instance 273 * @param {string} model - Model identifier 274 * @param {string} systemPrompt - Type-specific system prompt 275 * @param {Array} items - Batch of items to normalize 276 * @returns {Promise<Array>} Normalized items 277 */ 278async function normalizeBatch(openai, model, systemPrompt, items) { 279 const prompt = buildPrompt(items); 280 let lastError; 281 282 for (let attempt = 1; attempt <= MAX_RETRIES; attempt++) { 283 try { 284 const completion = await openai.chat.completions.create({ 285 model, 286 messages: [ 287 { role: 'system', content: systemPrompt }, 288 { role: 'user', content: prompt }, 289 ], 290 temperature: 0.3, // Low temperature for consistent results 291 max_tokens: MAX_TOKENS, // Ensure complete JSON responses 292 timeout: 60000, // 60 second timeout 293 }); 294 295 const responseContent = completion.choices[0]?.message?.content; 296 if (!responseContent) { 297 throw new Error('Empty response from LLM'); 298 } 299 300 return parseResponse(responseContent, items); 301 } catch (error) { 302 lastError = error; 303 304 if (attempt < MAX_RETRIES && isRetryableError(error)) { 305 const delayMs = INITIAL_RETRY_DELAY_MS * (2 ** (attempt - 1)); // Exponential backoff 306 log.warning(`LLM request failed (attempt ${attempt}/${MAX_RETRIES}), retrying in ${delayMs}ms...`, { 307 error: error.message, 308 }); 309 await sleep(delayMs); 310 } else if (attempt < MAX_RETRIES) { 311 // Non-retryable error, don't retry 312 log.error('LLM request failed with non-retryable error', { error: error.message }); 313 break; 314 } 315 } 316 } 317 318 // All retries exhausted or non-retryable error - return pass-through results 319 // instead of crashing the entire run. Failed items get low confidence so users can identify them. 320 const errorMessage = lastError?.message || 'Unknown error'; 321 log.error(`LLM request failed after all retries, returning pass-through for ${items.length} items`, { error: errorMessage }); 322 return items.map((item) => ({ 323 output: item.preProcessed || item.original, 324 confidence: 0, 325 error: errorMessage, 326 })); 327} 328 329/** 330 * Build the user prompt for a batch of items. 331 * @param {Array} items - Items to include in prompt 332 * @returns {string} Formatted prompt 333 */ 334function buildPrompt(items) { 335 const itemList = items.map((item, i) => `${i}. "${item.preProcessed}"`).join('\n'); 336 337 return `Normalize the following ${items.length} job titles:\n\n${itemList}`; 338} 339 340/** 341 * Attempt to repair truncated JSON arrays by finding the last complete object. 342 * This handles cases where LLM output is cut off mid-response. 343 * @param {string} jsonStr - Potentially truncated JSON string 344 * @returns {string} Repaired JSON string (or original if repair not needed/possible) 345 */ 346function repairTruncatedJson(jsonStr) { 347 // Quick check: if it parses, no repair needed 348 try { 349 JSON.parse(jsonStr); 350 return jsonStr; 351 } catch { 352 // Continue with repair attempt 353 } 354 355 // Only attempt repair on arrays 356 if (!jsonStr.trimStart().startsWith('[')) { 357 return jsonStr; 358 } 359 360 // Find the last complete object by looking for the last "}," or "}" before truncation 361 // Work backwards to find valid JSON 362 let lastValidEnd = -1; 363 364 // Look for last complete object ending with }, 365 const lastCommaObj = jsonStr.lastIndexOf('},'); 366 if (lastCommaObj > 0) { 367 lastValidEnd = lastCommaObj + 1; // Include the } 368 } 369 370 // Also check for a final object ending with just } 371 const lastBrace = jsonStr.lastIndexOf('}'); 372 if (lastBrace > lastValidEnd) { 373 // Check if this could be a complete object (not truncated mid-value) 374 const afterBrace = jsonStr.slice(lastBrace + 1).trim(); 375 if (afterBrace === '' || afterBrace === ']' || afterBrace.startsWith(',') || afterBrace.startsWith(']')) { 376 lastValidEnd = lastBrace; 377 } 378 } 379 380 if (lastValidEnd > 0) { 381 const repaired = `${jsonStr.slice(0, lastValidEnd + 1)}]`; 382 try { 383 JSON.parse(repaired); 384 log.debug('Repaired truncated JSON response', { 385 originalLength: jsonStr.length, 386 repairedLength: repaired.length, 387 }); 388 return repaired; 389 } catch { 390 // Repair failed, return original 391 } 392 } 393 394 return jsonStr; 395} 396 397/** 398 * Parse the LLM response and map results back to original items. 399 * @param {string} content - Raw LLM response content 400 * @param {Array} originalItems - Original items for fallback 401 * @returns {Array} Items with normalized values 402 */ 403function parseResponse(content, originalItems) { 404 try { 405 // Handle potential markdown code blocks 406 let jsonStr = content.trim(); 407 if (jsonStr.includes('```')) { 408 jsonStr = jsonStr.replace(/```json?\n?/g, '').replace(/```/g, '').trim(); 409 } 410 411 // Strip JavaScript-style comments that LLMs sometimes add 412 jsonStr = jsonStr.replace(/\/\/[^\n]*/g, ''); // Remove // comments 413 jsonStr = jsonStr.replace(/\/\*[\s\S]*?\*\//g, ''); // Remove /* */ comments 414 415 // Fix malformed tuple format: {"0", "name", 0.95} -> {"index": 0, "normalized": "name", "confidence": 0.95} 416 jsonStr = jsonStr.replace( 417 /\{"(\d+)",\s*"([^"]*)",\s*([\d.]+)\}/g, 418 '{"index": $1, "normalized": "$2", "confidence": $3}', 419 ); 420 421 // Handle truncated JSON by attempting to repair incomplete arrays 422 jsonStr = repairTruncatedJson(jsonStr); 423 424 const parsed = JSON.parse(jsonStr); 425 426 if (!Array.isArray(parsed)) { 427 throw new Error('Response is not an array'); 428 } 429 430 // Map response back to original items 431 return originalItems.map((item, i) => { 432 const result = parsed.find((r) => r.index === i); 433 if (result) { 434 return { 435 ...item, 436 normalized: postProcessTitle(result.normalized || item.preProcessed), 437 confidence: clampConfidence(result.confidence), 438 reasoning: result.reasoning || '', 439 }; 440 } 441 // Fallback if item not in response 442 return { 443 ...item, 444 normalized: postProcessTitle(item.preProcessed), 445 confidence: 0.5, 446 reasoning: 'Item not found in LLM response', 447 }; 448 }); 449 } catch (error) { 450 log.warning('Failed to parse LLM response', { error: error.message, content: content.slice(0, 200) }); 451 452 // Return pre-processed values with low confidence 453 return originalItems.map((item) => ({ 454 ...item, 455 normalized: postProcessTitle(item.preProcessed), 456 confidence: 0.4, 457 reasoning: `Failed to parse LLM response: ${error.message}`, 458 })); 459 } 460} 461 462/** 463 * Clamp confidence value to valid range [0, 1]. 464 * @param {number} value - Raw confidence value 465 * @returns {number} Clamped confidence 466 */ 467function clampConfidence(value) { 468 if (typeof value !== 'number' || Number.isNaN(value)) { 469 return 0.5; 470 } 471 return Math.max(0, Math.min(1, value)); 472} 473 474/** 475 * Post-process normalized job title to fix common issues. 476 * @param {string} text - Normalized text from LLM 477 * @returns {string} Text with fixes applied 478 */ 479function postProcessTitle(text) { 480 if (!text || typeof text !== 'string') { 481 return text; 482 } 483 484 let result = text.trim(); 485 486 // Fix apostrophe-S capitalization: "Stone'S" -> "Stone's" 487 result = result.replace(/(\w)'S\b/g, "$1's"); 488 489 // Ensure known acronyms are uppercase 490 for (const acronym of TITLE_ACRONYMS) { 491 const pattern = new RegExp(`\\b${escapeRegexChars(acronym)}\\b`, 'gi'); 492 result = result.replace(pattern, acronym); 493 } 494 495 // Clean up multiple spaces 496 result = result.replace(/\s+/g, ' ').trim(); 497 498 return result; 499} 500 501/** 502 * Escape special regex characters in a string. 503 * @param {string} str - String to escape 504 * @returns {string} Escaped string 505 */ 506function escapeRegexChars(str) { 507 return str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); 508}

1/** 2 * Superclean Job Titles 3 * 4 * Transforms messy job titles into clean, consistent formats 5 * for CRM systems, HR databases, and analytics. 6 */ 7 8import http from 'node:http'; 9import { Actor, log } from 'apify'; 10 11import { createNormalizer } from './normalizer.js'; 12import { preprocess } from './preprocessor.js'; 13 14/** 15 * Normalize flexible input formats for Clay/API compatibility. 16 * Accepts: ["string"] or [{input: "string"}] 17 * Returns: ["string"] 18 */ 19function normalizeItems(items) { 20 return items.map((item) => { 21 if (typeof item === 'string') { 22 return item; 23 } 24 if (item && typeof item === 'object' && 'input' in item) { 25 return String(item.input ?? ''); 26 } 27 return String(item ?? ''); 28 }); 29} 30 31await Actor.init(); 32 33if (Actor.config.get('metaOrigin') === 'STANDBY') { 34 startStandbyServer(); 35} else { 36 await runBatchMode(); 37 await Actor.exit(); 38} 39 40// --------------------------------------------------------------------------- 41// Standby mode — HTTP server for instant single-title cleaning 42// --------------------------------------------------------------------------- 43 44function startStandbyServer() { 45 const port = Actor.config.get('containerPort'); 46 47 const server = http.createServer(async (req, res) => { 48 // Readiness probe 49 if (req.headers['x-apify-container-server-readiness-probe']) { 50 res.writeHead(200, { 'Content-Type': 'application/json' }); 51 res.end(JSON.stringify({ status: 'ready' })); 52 return; 53 } 54 55 // GET only 56 if (req.method !== 'GET') { 57 res.writeHead(405, { 'Content-Type': 'application/json' }); 58 res.end(JSON.stringify({ error: 'Method not allowed. Use GET.' })); 59 return; 60 } 61 62 // No input param → serve landing page 63 const reqUrl = new URL(req.url, `http://${req.headers.host}`); 64 if (!reqUrl.searchParams.get('input')) { 65 res.writeHead(200, { 'Content-Type': 'text/html' }); 66 res.end(getLandingPageHtml()); 67 return; 68 } 69 70 try { 71 const result = await handleStandbyRequest(reqUrl, req); 72 await Actor.pushData(result); 73 74 res.writeHead(200, { 'Content-Type': 'application/json' }); 75 res.end(JSON.stringify(result)); 76 } catch (err) { 77 const statusCode = err.statusCode || 500; 78 res.writeHead(statusCode, { 'Content-Type': 'application/json' }); 79 res.end(JSON.stringify({ error: err.message })); 80 } 81 }); 82 83 server.listen(port, () => { 84 console.log(`Superclean Job Titles Standby server listening on port ${port}`); 85 }); 86} 87 88async function handleStandbyRequest(reqUrl, req) { 89 const input = reqUrl.searchParams.get('input'); 90 91 if (!input) { 92 const err = new Error("Provide an 'input' query parameter with the job title to clean."); 93 err.statusCode = 400; 94 throw err; 95 } 96 97 const style = reqUrl.searchParams.get('style') || 'standardized'; 98 const model = reqUrl.searchParams.get('model') || 'openrouter/auto'; 99 const openRouterApiKey = req.headers['x-openrouter-key'] || null; 100 101 const validStyles = ['standardized', 'abbreviated', 'simple']; 102 if (!validStyles.includes(style)) { 103 const err = new Error(`Invalid style "${style}". Must be one of: ${validStyles.join(', ')}`); 104 err.statusCode = 400; 105 throw err; 106 } 107 108 const trimmed = input.trim(); 109 if (!trimmed) { 110 return { id: 1, input, output: '', confidence: 0 }; 111 } 112 113 // Preprocess 114 const preprocessResult = preprocess(trimmed); 115 116 // Non-English → return extracted English or original 117 if (preprocessResult.isNonEnglish) { 118 return { 119 id: 1, 120 input: trimmed, 121 output: preprocessResult.extractedEnglish || trimmed, 122 confidence: 0, 123 }; 124 } 125 126 // LLM normalization 127 const item = { 128 id: 1, 129 input: trimmed, 130 preProcessed: preprocessResult.value, 131 isNonEnglish: false, 132 extractedEnglish: preprocessResult.extractedEnglish, 133 detectedSeniority: preprocessResult.detectedSeniority, 134 }; 135 136 const normalizer = createNormalizer(model, style, openRouterApiKey); 137 const normalizedItems = await normalizer.normalize([item]); 138 139 if (normalizedItems.length === 0) { 140 return { id: 1, input: trimmed, output: trimmed, confidence: 0 }; 141 } 142 143 return { 144 id: 1, 145 input: trimmed, 146 output: normalizedItems[0].normalized, 147 confidence: Math.round(normalizedItems[0].confidence * 100) / 100, 148 }; 149} 150 151// --------------------------------------------------------------------------- 152// Batch mode — existing pipeline for standard Actor runs 153// --------------------------------------------------------------------------- 154 155async function runBatchMode() { 156 // Get input 157 const input = await Actor.getInput(); 158 let { items: rawItems, item } = input || {}; 159 160 // Support single "item" string field for API/integration callers 161 if (typeof item === 'string' && item.trim()) { 162 if (!Array.isArray(rawItems)) rawItems = []; 163 rawItems = [item.trim(), ...rawItems]; 164 } 165 166 // Coerce bare string to array 167 if (typeof rawItems === 'string') rawItems = [rawItems]; 168 169 // Validate input 170 if (!rawItems || !Array.isArray(rawItems) || rawItems.length === 0) { 171 await Actor.pushData({ error: 'missing_items', message: 'Input required: provide "items" (array) or "item" (single string). Example: {"items": ["Senior Software Engineer"]}' }); 172 await Actor.exit('Skipped: Input required: provide "items" (array) or "item" (single string). Example: {"items": ["Senior Software Engineer"]}'); 173 return; 174 } 175 176 // Extract configuration with defaults 177 const style = input.style ?? 'standardized'; 178 const model = input.model ?? 'openrouter/auto'; 179 const openRouterApiKey = input.openRouterApiKey ?? null; 180 181 // Normalize input format (supports both ["string"] and [{input: "string"}]) 182 const items = normalizeItems(rawItems); 183 184 log.info(`Starting normalization of ${items.length} job titles`, { 185 style, 186 model, 187 }); 188 189 // Step 1: Run pre-processing on normalized items 190 const processedItems = items.map((item, index) => { 191 // Handle string input 192 const value = typeof item === 'string' ? item : String(item || ''); 193 const id = index + 1; 194 195 // Validate value 196 if (!value.trim()) { 197 log.warning(`Skipping invalid item at index ${index}`, { item }); 198 return null; 199 } 200 201 // Run pre-processing 202 const preprocessResult = preprocess(value); 203 204 return { 205 id, 206 input: value, 207 preProcessed: preprocessResult.value, 208 isNonEnglish: preprocessResult.isNonEnglish, 209 extractedEnglish: preprocessResult.extractedEnglish, 210 detectedSeniority: preprocessResult.detectedSeniority, 211 }; 212 }); 213 214 // Filter out invalid items 215 const validItems = processedItems.filter((item) => item !== null); 216 217 if (validItems.length === 0) { 218 await Actor.pushData({ error: 'no_valid_items', message: 'No valid items to process after input validation' }); 219 await Actor.exit('Skipped: No valid items to process after input validation'); 220 return; 221 } 222 223 // Separate English and non-English items 224 const englishItems = validItems.filter((item) => !item.isNonEnglish); 225 const nonEnglishItems = validItems.filter((item) => item.isNonEnglish); 226 227 if (nonEnglishItems.length > 0) { 228 log.info(`Flagging ${nonEnglishItems.length} non-English items for review`); 229 } 230 231 log.info(`Pre-processed ${validItems.length} items (${englishItems.length} English, ${nonEnglishItems.length} non-English)`); 232 233 // Step 2: LLM normalization for English items only 234 let normalizedItems = []; 235 if (englishItems.length > 0) { 236 const normalizer = createNormalizer(model, style, openRouterApiKey); 237 normalizedItems = await normalizer.normalize(englishItems); 238 } 239 240 // Step 3: Build output - combine LLM results with non-English items 241 const englishResults = normalizedItems.map((item) => ({ 242 id: item.id, 243 input: item.input, 244 output: item.normalized, 245 confidence: Math.round(item.confidence * 100) / 100, 246 })); 247 248 const nonEnglishResults = nonEnglishItems.map((item) => ({ 249 id: item.id, 250 input: item.input, 251 output: item.extractedEnglish || item.input, 252 confidence: 0, 253 })); 254 255 const results = [...englishResults, ...nonEnglishResults]; 256 257 // Push results to dataset 258 259 await Actor.pushData(results); 260 261 // Log summary 262 const avgConfidence = results.reduce((sum, r) => sum + r.confidence, 0) / results.length; 263 264 log.info(`Normalization complete`, { 265 totalItems: results.length, 266 averageConfidence: Math.round(avgConfidence * 100) / 100, 267 }); 268} 269 270// --------------------------------------------------------------------------- 271// Landing page HTML for Standby mode 272// --------------------------------------------------------------------------- 273 274function getLandingPageHtml() { 275 return `<!DOCTYPE html> 276<html lang="en"> 277<head> 278<meta charset="utf-8"> 279<meta name="viewport" content="width=device-width, initial-scale=1"> 280<title>Superclean Job Titles — Superlative</title> 281<link rel="preconnect" href="https://fonts.googleapis.com"> 282<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> 283<link href="https://fonts.googleapis.com/css2?family=DM+Sans:wght@400;500;700&family=Space+Mono:wght@400;700&display=swap" rel="stylesheet"> 284<style> 285:root{--color-bg:#0a0e14;--color-surface:#12171f;--color-border:#1e2632;--color-text:#e8eaed;--color-text-muted:#8b95a5;--color-accent:#4ade80;--color-accent-dim:rgba(74,222,128,0.15);--font-display:'Space Mono',monospace;--font-body:'DM Sans',sans-serif} 286*{margin:0;padding:0;box-sizing:border-box} 287body{font-family:var(--font-body);background:var(--color-bg);color:var(--color-text);line-height:1.6;min-height:100vh;overflow-x:hidden} 288a{color:var(--color-accent);text-decoration:none;transition:color .2s ease} 289a:hover{color:#6ee7a0} 290.grid-bg{position:fixed;top:0;left:0;width:100%;height:100%;background-image:linear-gradient(var(--color-border) 1px,transparent 1px),linear-gradient(90deg,var(--color-border) 1px,transparent 1px);background-size:60px 60px;opacity:.3;pointer-events:none;z-index:0} 291.accent-line{position:fixed;top:0;left:0;width:100%;height:2px;background:linear-gradient(90deg,transparent 0%,var(--color-accent) 50%,transparent 100%);animation:shimmer 3s ease-in-out infinite;z-index:10} 292@keyframes shimmer{0%,100%{transform:translateX(-100%)}50%{transform:translateX(100%)}} 293@keyframes fadeUp{from{opacity:0;transform:translateY(20px)}to{opacity:1;transform:translateY(0)}} 294.container{position:relative;z-index:1;max-width:900px;margin:0 auto;padding:0 24px} 295header{padding:32px 0} 296.logo{font-family:var(--font-display);font-size:14px;font-weight:700;letter-spacing:.1em;text-transform:uppercase;color:var(--color-text);display:flex;align-items:center;gap:10px;text-decoration:none} 297.logo:hover{color:var(--color-text)} 298.logo-mark{width:28px;height:28px;background:var(--color-accent);display:flex;align-items:center;justify-content:center} 299.logo-mark svg{width:16px;height:16px} 300.hero{padding:80px 0 60px;animation:fadeUp .8s ease-out} 301.hero-label{font-family:var(--font-display);font-size:12px;font-weight:400;letter-spacing:.15em;text-transform:uppercase;color:var(--color-accent);margin-bottom:24px;display:flex;align-items:center;gap:12px} 302.hero-label::before{content:'';width:24px;height:1px;background:var(--color-accent)} 303h1{font-family:var(--font-display);font-size:clamp(28px,5vw,44px);font-weight:700;line-height:1.15;margin-bottom:20px;letter-spacing:-.02em} 304h1 .highlight{color:var(--color-accent)} 305.hero-desc{font-size:18px;color:var(--color-text-muted);max-width:540px;margin-bottom:28px;line-height:1.7} 306.hero-link{font-family:var(--font-display);font-size:13px;letter-spacing:.05em;color:var(--color-accent);display:inline-flex;align-items:center;gap:6px} 307.section{padding:60px 0;border-top:1px solid var(--color-border);animation:fadeUp .8s ease-out .2s backwards} 308.section-label{font-family:var(--font-display);font-size:11px;font-weight:400;letter-spacing:.2em;text-transform:uppercase;color:var(--color-text-muted);margin-bottom:24px} 309h2{font-family:var(--font-display);font-size:20px;font-weight:700;margin-bottom:16px;letter-spacing:-.01em} 310p,li{font-size:15px;color:var(--color-text-muted);line-height:1.7} 311li{margin-bottom:6px} 312ul{margin:.5rem 0 1rem 1.25rem} 313strong{color:var(--color-text);font-weight:600} 314pre{background:var(--color-surface);color:var(--color-text);padding:16px 20px;overflow-x:auto;margin:12px 0 16px;font-family:var(--font-display);font-size:13px;line-height:1.6;border:1px solid var(--color-border)} 315code{font-family:var(--font-display);font-size:13px} 316p code,li code,td code{background:var(--color-accent-dim);color:var(--color-accent);padding:2px 6px} 317table{width:100%;border-collapse:collapse;margin:12px 0 16px;font-size:14px} 318th,td{border:1px solid var(--color-border);padding:10px 14px;text-align:left} 319th{background:var(--color-surface);color:var(--color-text-muted);font-family:var(--font-display);font-size:12px;font-weight:400;letter-spacing:.05em;text-transform:uppercase} 320td{color:var(--color-text-muted)} 321.badge{display:inline-block;background:var(--color-accent-dim);color:var(--color-accent);padding:3px 8px;font-family:var(--font-display);font-size:11px;font-weight:400;letter-spacing:.05em} 322.note{font-size:13px;color:var(--color-text-muted);margin-top:8px} 323.product-grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(240px,1fr));gap:16px;margin-top:24px} 324.product-card{background:var(--color-surface);border:1px solid var(--color-border);padding:24px;transition:all .3s ease;text-decoration:none;color:inherit;display:block} 325.product-card:hover{border-color:var(--color-accent);transform:translateY(-2px)} 326.product-name{font-family:var(--font-display);font-size:14px;font-weight:700;margin-bottom:8px;color:var(--color-text);display:flex;align-items:center;gap:8px} 327.product-name .status{font-size:9px;font-weight:400;padding:3px 6px;background:var(--color-accent-dim);color:var(--color-accent);letter-spacing:.1em} 328.product-desc{font-size:14px;color:var(--color-text-muted);line-height:1.5} 329footer{padding:48px 0;border-top:1px solid var(--color-border)} 330.footer-content{display:flex;justify-content:space-between;align-items:center;flex-wrap:wrap;gap:24px} 331.footer-text{font-size:13px;color:var(--color-text-muted)} 332.footer-links{display:flex;gap:24px} 333.footer-links a{font-family:var(--font-display);font-size:12px;color:var(--color-text-muted);letter-spacing:.05em} 334.footer-links a:hover{color:var(--color-accent)} 335@media(max-width:600px){.hero{padding:60px 0 40px}.section{padding:40px 0}.footer-content{flex-direction:column;align-items:flex-start}pre{font-size:12px;padding:12px 14px}} 336</style> 337</head> 338<body> 339<div class="grid-bg"></div> 340<div class="accent-line"></div> 341 342<div class="container"> 343 <header> 344 <a href="https://apify.com/superlativetech?fpr=8e9l1" class="logo"> 345 <div class="logo-mark"> 346 <svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5"> 347 <polyline points="4 17 10 11 4 5"></polyline> 348 <line x1="12" y1="19" x2="20" y2="19"></line> 349 </svg> 350 </div> 351 Superlative 352 </a> 353 </header> 354 355 <section class="hero"> 356 <div class="hero-label">Superclean</div> 357 <h1>Job <span class="highlight">Titles</span></h1> 358 <p class="hero-desc">Instant job title normalization. AI-powered cleaning for CRM, HR systems, and lead scoring. Sub-second responses via Standby API.</p> 359 <a href="https://apify.com/superlativetech/superclean-job-titles?fpr=8e9l1" class="hero-link">View on Apify Store →</a> 360 </section> 361 362 <section class="section"> 363 <div class="section-label">Features</div> 364 <h2>What this does</h2> 365 <ul> 366 <li><strong>AI-powered cleaning</strong> — LLM normalizes job titles with context awareness</li> 367 <li><strong>Three styles</strong> — Standardized, Abbreviated, Simple</li> 368 <li><strong>Expands abbreviations</strong> — Sr. → Senior, VP → Vice President</li> 369 <li><strong>BYOK support</strong> — Use your own OpenRouter API key via header</li> 370 <li><strong>Batch mode</strong> — Process hundreds of titles via the standard Actor run</li> 371 </ul> 372 </section> 373 374 <section class="section"> 375 <div class="section-label">Getting Started</div> 376 <h2>Quick start</h2> 377 <p>Replace <code>YOUR_TOKEN</code> with your Apify API token.</p> 378 379 <p style="margin-top:16px"><strong>Standardized style:</strong></p> 380 <pre>curl "https://superlativetech--superclean-job-titles.apify.actor?token=YOUR_TOKEN&input=Sr.+Software+Eng.+-+Platform+Team"</pre> 381 382 <p><strong>Simple style:</strong></p> 383 <pre>curl "https://superlativetech--superclean-job-titles.apify.actor?token=YOUR_TOKEN&input=Sr.+Software+Eng.+-+Platform+Team&style=simple"</pre> 384 385 <p><strong>With your own OpenRouter key:</strong></p> 386 <pre>curl -H "X-OpenRouter-Key: sk-or-..." "https://superlativetech--superclean-job-titles.apify.actor?token=YOUR_TOKEN&input=VP+of+Mktg"</pre> 387 </section> 388 389 <section class="section"> 390 <div class="section-label">Reference</div> 391 <h2>Query parameters</h2> 392 <table> 393 <tr><th>Parameter</th><th>Required</th><th>Description</th></tr> 394 <tr><td><code>input</code></td><td>Yes</td><td>Job title to clean</td></tr> 395 <tr><td><code>style</code></td><td>No</td><td>Output style: <code>standardized</code> (default), <code>abbreviated</code>, or <code>simple</code></td></tr> 396 <tr><td><code>model</code></td><td>No</td><td>LLM model (default: <code>openrouter/auto</code>)</td></tr> 397 <tr><td><code>token</code></td><td>Yes</td><td>Your Apify API token</td></tr> 398 </table> 399 <h2 style="margin-top:24px">Headers</h2> 400 <table> 401 <tr><th>Header</th><th>Required</th><th>Description</th></tr> 402 <tr><td><code>X-OpenRouter-Key</code></td><td>No</td><td>Your OpenRouter API key for BYOK</td></tr> 403 </table> 404 </section> 405 406 <section class="section"> 407 <div class="section-label">Response</div> 408 <h2>Response format</h2> 409 <pre>{ 410 "id": 1, 411 "input": "Sr. Software Eng. - Platform Team", 412 "output": "Senior Software Engineer", 413 "confidence": 0.9 414}</pre> 415 </section> 416 417 <section class="section"> 418 <div class="section-label">Cost</div> 419 <h2>Pricing</h2> 420 <p>Free to use — you only pay standard Apify platform usage.</p> 421 </section> 422 423 <section class="section"> 424 <div class="section-label">Auth</div> 425 <h2>Authentication</h2> 426 <p>Authenticate using either method:</p> 427 <ul> 428 <li>Query parameter: <code>?token=YOUR_APIFY_TOKEN</code></li> 429 <li>Header: <code>Authorization: Bearer YOUR_APIFY_TOKEN</code></li> 430 </ul> 431 <p style="margin-top:8px">Get your token from <a href="https://console.apify.com/settings/integrations">Apify Console → Settings → Integrations</a>.</p> 432 </section> 433 434 <section class="section"> 435 <div class="section-label">Superlative</div> 436 <h2>More from Superlative</h2> 437 <div class="product-grid"> 438 <a href="https://apify.com/superlativetech/superclean-company-names?fpr=8e9l1" class="product-card"> 439 <div class="product-name">Company Names <span class="status">Live</span></div> 440 <p class="product-desc">Normalize company names for CRM and cold email outreach.</p> 441 </a> 442 <a href="https://apify.com/superlativetech/superclean-person-names?fpr=8e9l1" class="product-card"> 443 <div class="product-name">Person Names <span class="status">Live</span></div> 444 <p class="product-desc">Clean and format person names for personalization.</p> 445 </a> 446 <a href="https://apify.com/superlativetech/superclean-product-names?fpr=8e9l1" class="product-card"> 447 <div class="product-name">Product Names <span class="status">Live</span></div> 448 <p class="product-desc">Normalize product and brand names from exports.</p> 449 </a> 450 <a href="https://apify.com/superlativetech/superclean-places?fpr=8e9l1" class="product-card"> 451 <div class="product-name">Places <span class="status">Live</span></div> 452 <p class="product-desc">Parse and standardize location strings.</p> 453 </a> 454 <a href="https://apify.com/superlativetech/superclean-urls?fpr=8e9l1" class="product-card"> 455 <div class="product-name">URLs <span class="status">Live</span></div> 456 <p class="product-desc">Clean and normalize URLs from lead data.</p> 457 </a> 458 <a href="https://apify.com/superlativetech/superclean-phone-numbers?fpr=8e9l1" class="product-card"> 459 <div class="product-name">Phone Numbers <span class="status">Live</span></div> 460 <p class="product-desc">Format and validate phone numbers.</p> 461 </a> 462 <a href="https://apify.com/superlativetech/dns-lookup?fpr=8e9l1" class="product-card"> 463 <div class="product-name">Supernet DNS Lookup <span class="status">Live</span></div> 464 <p class="product-desc">Look up DNS records for any domain.</p> 465 </a> 466 <a href="https://apify.com/superlativetech/http-api?fpr=8e9l1" class="product-card"> 467 <div class="product-name">HTTP API <span class="status">Live</span></div> 468 <p class="product-desc">General-purpose HTTP request utility.</p> 469 </a> 470 </div> 471 </section> 472 473 <footer> 474 <div class="footer-content"> 475 <p class="footer-text">© 2026 Superlative</p> 476 <div class="footer-links"> 477 <a href="https://apify.com/superlativetech?fpr=8e9l1">Apify Store</a> 478 </div> 479 </div> 480 </footer> 481</div> 482 483</body> 484</html>`; 485}