# Email Subject Line A/B Test Evaluator (`trovevault/email-subject-line-evaluator`) Actor

Compare and rank email subject lines by clarity, specificity, attention, mobile fit, trust, and context, then generate a controlled A/B test pair.

- **URL**: https://apify.com/trovevault/email-subject-line-evaluator.md
- **Developed by:** [Trove Vault](https://apify.com/trovevault) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.85 / 1,000 subjects

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Email Subject Line A/B Test Evaluator

Compare email subject lines before you send. This Actor ranks 2–50 variants, diagnoses the patterns behind each score, and turns the strongest option into a controlled A/B test plan. Every dataset row includes `comparativeScore`, six score components, risks, evidence signals, and a suggested rewrite.

It uses a versioned evidence model and deterministic analysis, so no AI key, email platform account, or live scraping is required.

### Why use this Actor?

A subject line can look strong while testing several ideas at once: personalization, urgency, emoji use, length, and offer framing. If two variants differ in all five, the result does not explain what caused the difference.

Use it to:

- rank a batch of candidate subject lines consistently;
- separate clarity, specificity, attention, mobile fit, trust, and context fit;
- identify patterns and flag high-pressure or misleading framing;
- create a controlled challenger that changes one variable while preserving the message;
- process subject lines in bulk through the Apify Console, API, schedules, or automation tools.

Scores compare the variants within the submitted context. They are not predicted open rates and do not guarantee campaign performance.

### Who is it for?

CRM, lifecycle, e-commerce, agency, newsletter, nonprofit, and B2B teams can use the Actor to review recurring campaign copy. It supports commercial, editorial, triggered, event, fundraising, transactional, and cold-outreach workflows.

### What the Actor evaluates

Each subject line receives a contextual analysis across six dimensions:

| Dimension          | What it represents                                                     |
| ------------------ | ---------------------------------------------------------------------- |
| `clarityScore`     | Whether the meaning is understandable at inbox-scanning speed          |
| `specificityScore` | Concrete value, deadlines, numbers, offer detail, or personalization   |
| `attentionScore`   | Defensible attention cues such as questions, curiosity, or urgency     |
| `mobileFitScore`   | Likely readability in narrower inbox previews                          |
| `trustScore`       | Pressure, deception, punctuation noise, and other trust-risk patterns  |
| `contextFitScore`  | Fit with the selected campaign, industry, audience, and supplied topic |

The evidence model contains more than 50 contextual rules derived from large aggregate studies, controlled experiments, campaign test series, and measurement limitations. Evidence is normalized into buyer-facing signals; the Actor does not republish third-party articles, reports, or raw subject-line databases.

### How to use it

1. Enter at least two subject lines written for the same email.
2. Select the campaign type, industry, and recipient relationship.
3. Optionally describe the campaign topic or offer. This improves context checks and lets the Actor rewrite vague copy without inventing a claim.
4. Run the Actor.
5. Review the ranked dataset, then open `RUN_SUMMARY` for the recommended pair and isolated test variable.

Keep the audience, offer, send time, sender, and email content consistent when running the resulting test. The recommended pair is designed to isolate one copy variable where the available context allows it.

### Input

```json
{
    "subjectLines": [
        "Summer sale: 20% off until Sunday",
        "Something special just landed ☀️",
        "{{first_name}}, your summer offer is here"
    ],
    "campaignType": "promotion",
    "industry": "ecommerce",
    "audienceContext": "existing_customers",
    "campaignTopic": "20% off the summer collection until Sunday"
}
```

#### Input fields

| Field             | Required | Description                                                         |
| ----------------- | -------- | ------------------------------------------------------------------- |
| `subjectLines`    | Yes      | 2–50 unique variants for the same email, each up to 200 characters  |
| `campaignType`    | Yes      | The workflow the email belongs to                                   |
| `industry`        | Yes      | Used only for supported contextual differences                      |
| `audienceContext` | Yes      | Relationship between sender and recipients                          |
| `campaignTopic`   | No       | The actual email promise, offer, or topic, up to 240 characters     |
| `datasetId`       | No       | Existing Apify dataset that should also receive the evaluation rows |
| `runId`           | No       | Your workflow identifier, copied to every public row when supplied  |

Personalization tokens such as `{{first_name}}`, `*|FNAME|*`, `[first_name]`, and `%%first_name%%` are recognized. The Actor treats them as copy patterns; it does not resolve them or access recipient data.

### Output

The default dataset contains one row per unique subject line, ordered by rank. The primary dataset view exposes every public output field.

```json
{
    "subjectLine": "Summer sale: 20% off until Sunday",
    "rank": 1,
    "comparativeScore": 95,
    "recommendation": "lead_variant",
    "patternFamilies": ["offer_led", "urgency_scarcity", "numeric_specificity"],
    "characterCount": 33,
    "clarityScore": 87,
    "specificityScore": 97,
    "trustScore": 89,
    "contextFitScore": 95,
    "deliverabilityRisk": "low",
    "strengths": ["Clear at inbox-scanning speed", "Provides concrete detail"],
    "risks": [],
    "evidenceSignals": ["Readable inbox length", "Concrete deadline hypothesis"]
}
```

Rows also include word and emoji counts, personalization type, attention and mobile scores, a test hypothesis, a suggested variant, and the submitted context.

`RUN_SUMMARY` contains the campaign-level test plan:

```json
{
    "recommendedPair": [
        "Summer sale: 20% off until Sunday",
        "{{first_name}}, Summer sale: 20% off until Sunday"
    ],
    "isolatedVariable": "personalization",
    "testReadiness": "controlled_variant_created",
    "testHypothesis": "Test whether personalization changes downstream engagement for this audience while the underlying message stays constant."
}
```

### Run with the Apify API

Replace `YOUR_APIFY_TOKEN` with an Apify API token. The endpoint waits for the run and returns dataset items.

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/trovevault~email-subject-line-evaluator/run-sync-get-dataset-items" \
  -H "Authorization: Bearer YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "subjectLines": [
      "Summer sale: 20% off until Sunday",
      "Something special just landed ☀️"
    ],
    "campaignType": "promotion",
    "industry": "ecommerce",
    "audienceContext": "existing_customers",
    "campaignTopic": "20% off the summer collection until Sunday"
  }'
```

To retrieve the campaign-level plan separately, read the `RUN_SUMMARY` record from the run's default key-value store.

### Automation and MCP

Call it from schedules, webhooks, Make, Zapier, n8n, or Apify MCP. Use `datasetId` to append rows and `runId` to identify campaigns. It does not send email.

### How to interpret the result

`comparativeScore` ranks this batch; it is not a forecast. Read evidence signals and risks with the score, prefer the controlled test, and verify rewrites against the real campaign promise.

### Limitations

- The Actor does not predict an open rate, revenue lift, inbox placement, or statistical significance.
- Performance also depends on list, sender, deliverability, segmentation, timing, preview text, offer, and message.
- Evidence combines different periods and campaign contexts, so every recommendation still needs an audience-specific test.
- Keyword evidence is optimized for English; structural checks work across languages.
- `deliverabilityRisk` is a copy-pattern warning, not a spam-filter or inbox-placement test.
- Rewrites use only submitted copy and do not invent offers or recipient facts.
- The Actor does not send tests, connect to an ESP, or import campaign results.

### Troubleshooting

#### “At least two unique subject lines are required”

Comparison is case-insensitive and trims surrounding whitespace. Add a genuinely different second variant.

#### Why was a subject rewritten or replaced in the test pair?

With `campaignTopic`, a vague line can be replaced by the stated promise. The top runner-up can also be replaced by a controlled challenger when it changes several variables at once. Remove `campaignTopic` if you only want structural cleanup.

#### The output is empty

Check for 2–50 non-empty strings. Input failures are written to `RUN_SUMMARY`. For reproducible problems, open the Issues tab with private subject matter removed.

### Related Actor

Pair it with [Email List Hygiene](https://apify.com/trovevault/email-list-hygiene) for format, MX, disposable-domain, and role-address checks.

### Changelog

#### 0.1 — Initial MVP

Contextual bulk comparison, six score components, 50+ evidence rules, controlled challenger generation, and pipeline fields.

### Feedback and support

Use the Actor's Issues tab for bug reports, unsupported personalization formats, or examples where the context scoring produces a misleading recommendation. Include the campaign type, industry, and audience relationship so the behavior can be reproduced.

# Actor input Schema

## `subjectLines` (type: `array`):

Enter 2–50 subject line variants, one per line. Use the exact personalization tokens that your email platform will receive, such as {{first\_name}} or *|FNAME|*.

## `campaignType` (type: `string`):

Choose the email workflow these subject lines belong to. This changes context scoring and test recommendations.

## `industry` (type: `string`):

Select the closest industry. Industry is used only where the evidence supports a contextual difference; it is not treated as a universal benchmark.

## `audienceContext` (type: `string`):

Describe the recipient relationship. Change this when the same campaign could be sent to prospects, active subscribers, customers, or inactive contacts.

## `campaignTopic` (type: `string`):

Briefly state what the email contains, for example “20% off the summer collection until Sunday”. This helps the Actor judge clarity and produce a safer rewrite without inventing an offer.

## `datasetId` (type: `string`):

Push the same evaluation rows into an existing Apify dataset in addition to this run's default dataset.

## `runId` (type: `string`):

Attach your own workflow or upstream run identifier to every output row.

## Actor input object example

```json
{
  "subjectLines": [
    "Summer sale: 20% off until Sunday",
    "Something special just landed ☀️",
    "{{first_name}}, your summer offer is here"
  ],
  "campaignType": "promotion",
  "industry": "ecommerce",
  "audienceContext": "existing_customers",
  "campaignTopic": "20% off the summer collection until Sunday"
}
```

# Actor output Schema

## `evaluations` (type: `string`):

No description

## `runSummary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "subjectLines": [
        "Summer sale: 20% off until Sunday",
        "Something special just landed ☀️",
        "{{first_name}}, your summer offer is here"
    ],
    "campaignTopic": "20% off the summer collection until Sunday"
};

// Run the Actor and wait for it to finish
const run = await client.actor("trovevault/email-subject-line-evaluator").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "subjectLines": [
        "Summer sale: 20% off until Sunday",
        "Something special just landed ☀️",
        "{{first_name}}, your summer offer is here",
    ],
    "campaignTopic": "20% off the summer collection until Sunday",
}

# Run the Actor and wait for it to finish
run = client.actor("trovevault/email-subject-line-evaluator").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "subjectLines": [
    "Summer sale: 20% off until Sunday",
    "Something special just landed ☀️",
    "{{first_name}}, your summer offer is here"
  ],
  "campaignTopic": "20% off the summer collection until Sunday"
}' |
apify call trovevault/email-subject-line-evaluator --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,trovevault/email-subject-line-evaluator"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/cN8YvylWKV2tsO32O/builds/xc3OmzRRK1teRWCzO/openapi.json