Email Subject Line A/B Test Evaluator

Compare email subject lines before you send. This Actor ranks 2–50 variants, diagnoses the patterns behind each score, and turns the strongest option into a controlled A/B test plan. Every dataset row includes comparativeScore, six score components, risks, evidence signals, and a suggested rewrite.

It uses a versioned evidence model and deterministic analysis, so no AI key, email platform account, or live scraping is required.

Why use this Actor?

A subject line can look strong while testing several ideas at once: personalization, urgency, emoji use, length, and offer framing. If two variants differ in all five, the result does not explain what caused the difference.

Use it to:

• rank a batch of candidate subject lines consistently;
• separate clarity, specificity, attention, mobile fit, trust, and context fit;
• identify patterns and flag high-pressure or misleading framing;
• create a controlled challenger that changes one variable while preserving the message;
• process subject lines in bulk through the Apify Console, API, schedules, or automation tools.

Scores compare the variants within the submitted context. They are not predicted open rates and do not guarantee campaign performance.

Who is it for?

CRM, lifecycle, e-commerce, agency, newsletter, nonprofit, and B2B teams can use the Actor to review recurring campaign copy. It supports commercial, editorial, triggered, event, fundraising, transactional, and cold-outreach workflows.

What the Actor evaluates

Each subject line receives a contextual analysis across six dimensions:

Dimension	What it represents
clarityScore	Whether the meaning is understandable at inbox-scanning speed
specificityScore	Concrete value, deadlines, numbers, offer detail, or personalization
attentionScore	Defensible attention cues such as questions, curiosity, or urgency
mobileFitScore	Likely readability in narrower inbox previews
trustScore	Pressure, deception, punctuation noise, and other trust-risk patterns
contextFitScore	Fit with the selected campaign, industry, audience, and supplied topic

The evidence model contains more than 50 contextual rules derived from large aggregate studies, controlled experiments, campaign test series, and measurement limitations. Evidence is normalized into buyer-facing signals; the Actor does not republish third-party articles, reports, or raw subject-line databases.

How to use it

1. Enter at least two subject lines written for the same email.
2. Select the campaign type, industry, and recipient relationship.
3. Optionally describe the campaign topic or offer. This improves context checks and lets the Actor rewrite vague copy without inventing a claim.
4. Run the Actor.
5. Review the ranked dataset, then open RUN_SUMMARY for the recommended pair and isolated test variable.

Keep the audience, offer, send time, sender, and email content consistent when running the resulting test. The recommended pair is designed to isolate one copy variable where the available context allows it.

Input

{
    "subjectLines": [
        "Summer sale: 20% off until Sunday",
        "Something special just landed ☀️",
        "{{first_name}}, your summer offer is here"
    ],
    "campaignType": "promotion",
    "industry": "ecommerce",
    "audienceContext": "existing_customers",
    "campaignTopic": "20% off the summer collection until Sunday"
}

Input fields

Field	Required	Description
subjectLines	Yes	2–50 unique variants for the same email, each up to 200 characters
campaignType	Yes	The workflow the email belongs to
industry	Yes	Used only for supported contextual differences
audienceContext	Yes	Relationship between sender and recipients
campaignTopic	No	The actual email promise, offer, or topic, up to 240 characters
datasetId	No	Existing Apify dataset that should also receive the evaluation rows
runId	No	Your workflow identifier, copied to every public row when supplied

Personalization tokens such as {{first_name}}, *|FNAME|*, [first_name], and %%first_name%% are recognized. The Actor treats them as copy patterns; it does not resolve them or access recipient data.

Output

The default dataset contains one row per unique subject line, ordered by rank. The primary dataset view exposes every public output field.

{
    "subjectLine": "Summer sale: 20% off until Sunday",
    "rank": 1,
    "comparativeScore": 95,
    "recommendation": "lead_variant",
    "patternFamilies": ["offer_led", "urgency_scarcity", "numeric_specificity"],
    "characterCount": 33,
    "clarityScore": 87,
    "specificityScore": 97,
    "trustScore": 89,
    "contextFitScore": 95,
    "deliverabilityRisk": "low",
    "strengths": ["Clear at inbox-scanning speed", "Provides concrete detail"],
    "risks": [],
    "evidenceSignals": ["Readable inbox length", "Concrete deadline hypothesis"]
}

Rows also include word and emoji counts, personalization type, attention and mobile scores, a test hypothesis, a suggested variant, and the submitted context.

RUN_SUMMARY contains the campaign-level test plan:

{
    "recommendedPair": [
        "Summer sale: 20% off until Sunday",
        "{{first_name}}, Summer sale: 20% off until Sunday"
    ],
    "isolatedVariable": "personalization",
    "testReadiness": "controlled_variant_created",
    "testHypothesis": "Test whether personalization changes downstream engagement for this audience while the underlying message stays constant."
}

Run with the Apify API

Replace YOUR_APIFY_TOKEN with an Apify API token. The endpoint waits for the run and returns dataset items.

curl -X POST \
  "https://api.apify.com/v2/acts/trovevault~email-subject-line-evaluator/run-sync-get-dataset-items" \
  -H "Authorization: Bearer YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "subjectLines": [
      "Summer sale: 20% off until Sunday",
      "Something special just landed ☀️"
    ],
    "campaignType": "promotion",
    "industry": "ecommerce",
    "audienceContext": "existing_customers",
    "campaignTopic": "20% off the summer collection until Sunday"
  }'

To retrieve the campaign-level plan separately, read the RUN_SUMMARY record from the run's default key-value store.

Automation and MCP

Call it from schedules, webhooks, Make, Zapier, n8n, or Apify MCP. Use datasetId to append rows and runId to identify campaigns. It does not send email.

How to interpret the result

comparativeScore ranks this batch; it is not a forecast. Read evidence signals and risks with the score, prefer the controlled test, and verify rewrites against the real campaign promise.

Limitations

• The Actor does not predict an open rate, revenue lift, inbox placement, or statistical significance.
• Performance also depends on list, sender, deliverability, segmentation, timing, preview text, offer, and message.
• Evidence combines different periods and campaign contexts, so every recommendation still needs an audience-specific test.
• Keyword evidence is optimized for English; structural checks work across languages.
• deliverabilityRisk is a copy-pattern warning, not a spam-filter or inbox-placement test.
• Rewrites use only submitted copy and do not invent offers or recipient facts.
• The Actor does not send tests, connect to an ESP, or import campaign results.

Troubleshooting

“At least two unique subject lines are required”

Comparison is case-insensitive and trims surrounding whitespace. Add a genuinely different second variant.

Why was a subject rewritten or replaced in the test pair?

With campaignTopic, a vague line can be replaced by the stated promise. The top runner-up can also be replaced by a controlled challenger when it changes several variables at once. Remove campaignTopic if you only want structural cleanup.

The output is empty

Check for 2–50 non-empty strings. Input failures are written to RUN_SUMMARY. For reproducible problems, open the Issues tab with private subject matter removed.

Related Actor

Pair it with Email List Hygiene for format, MX, disposable-domain, and role-address checks.

Changelog

0.1 — Initial MVP

Contextual bulk comparison, six score components, 50+ evidence rules, controlled challenger generation, and pipeline fields.

Feedback and support

Use the Actor's Issues tab for bug reports, unsupported personalization formats, or examples where the context scoring produces a misleading recommendation. Include the campaign type, industry, and audience relationship so the behavior can be reproduced.