FreightWaves News Scraper

Scrape public FreightWaves logistics and transportation news pages into structured records for freight market intelligence, monitoring, research, and news-feed pipelines.

What it extracts

Field	Description
articleUrl	Canonical public FreightWaves article URL
title	Article headline
dek	Public excerpt or description
author	Byline when available
publishedAt, updatedAt	Publication and update timestamps
section	FreightWaves section or category
tags	Public topic labels
imageUrl	Open Graph or article image
body	Visible public article text, when requested and available
sourceUrl	Listing or input URL where the article was discovered
warnings	Non-fatal extraction notes

The dataset also contains one final recordType: "summary" record with page counts, article counts, warnings, and the effective run configuration. The same summary is written to the RUN_SUMMARY key-value store key.

When to use it

Use this actor when you need a bounded batch of public FreightWaves stories from a news, category, tag, pagination, or article URL. It is suitable for:

• Freight broker and carrier market monitoring
• 3PL and shipper competitive intelligence
• Transportation newsletter and RSS-style feeds
• Trucking, rail, maritime, air-cargo, and supply-chain topic research
• Search-indexed article archives and historical snapshots
• AI-agent workflows that need short, stable article records

Do not use it for FreightWaves SONAR data, private dashboards, subscriber-only material, account data, or actions that modify FreightWaves. The actor does not log in, bypass a paywall, or attempt to defeat a challenge page. If a public page exposes metadata but not the full body, the actor returns the metadata and adds a warning.

Input

The default input scrapes the public FreightWaves News listing:

{
  "startUrls": [
    { "url": "https://www.freightwaves.com/news/category/news" }
  ],
  "maxResults": 20,
  "maxPages": 3,
  "includeArticleBody": true,
  "requestDelayMs": 750
}

startUrls accepts up to 10 same-site FreightWaves URLs. Use a direct article URL for one known story, a category URL for a topic, a tag URL for a public tag archive, or a paginated listing URL for a known page. maxResults is the total article cap across all inputs and is bounded to 1–100. maxPages controls listing pagination and is bounded to 1–10. includeArticleBody defaults to true; set it to false for metadata-only output. requestDelayMs defaults to 750 milliseconds and is bounded to 250–5000 milliseconds for polite pacing.

Output

An article record is flat and stable so it can be consumed by APIs, tables, or AI agents:

{
  "recordType": "article",
  "articleUrl": "https://www.freightwaves.com/news/example-story",
  "canonicalUrl": "https://www.freightwaves.com/news/example-story",
  "articleId": "a4f1c3d89bb2",
  "title": "Freight market update",
  "dek": "A short public excerpt.",
  "author": "FreightWaves Staff",
  "publishedAt": "2026-08-01T12:00:00.000Z",
  "updatedAt": null,
  "section": "News",
  "imageUrl": "https://www.freightwaves.com/wp-content/uploads/example.jpg",
  "tags": ["trucking", "logistics"],
  "body": "Visible public article text...",
  "sourceUrl": "https://www.freightwaves.com/news/category/news",
  "scrapedAt": "2026-08-02T10:00:00.000Z",
  "warnings": [],
  "summary": null
}

When the actor reaches the article cap, the first records are kept and the run stops. The final summary record includes articlesScraped, listingPagesVisited, articlePagesVisited, requestedMaxResults, warnings, and buildNumber. If the target returns a valid empty page, the run completes with a truthful summary instead of manufacturing article records.

Pricing

Event	Price
Actor start	$0.00005 per run
Article returned	$0.005 per unique article

For example, a run capped at 20 articles has a maximum article-event cost of $0.10 plus the start event. Apify compute and proxy usage may also apply when the pay-per-usage billing option is selected. The actor prints its maximum article-event cost before the first article charge and reports the actual article charges at completion.

Reliability and limits

The crawler uses a single concurrent request worker, a bounded request queue, session persistence, retry handling for transient upstream errors, and same-host URL validation. It connects directly to the public site by default so the actor can run with Apify Limited permissions and without requiring account-level proxy APIs. The public News listing is used as the default canary path. HTTP 403/429 responses retire the session and retry; repeated request failures are recorded in the run summary. A completely blocked run fails honestly with an actionable status message rather than returning a misleading empty dataset.

The source site can change its HTML, date labels, navigation, or article-body layout. Extraction uses metadata, JSON-LD, semantic selectors, and generic fallbacks. Optional fields are explicit null values when not exposed. Public-page availability is not a guarantee that every story has full text, and this actor should be operated in compliance with FreightWaves terms, applicable law, and the target site's published access preferences.

API example

JavaScript:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const { defaultDatasetId } = await client.actor('umqHl4LhqmPHcBh0L/freightwaves-news-scraper').call({
  maxResults: 10,
  startUrls: [{ url: 'https://www.freightwaves.com/news/category/news' }]
});
const { items } = await client.dataset(defaultDatasetId).listItems();
console.log(items);

Python:

from apify_client import ApifyClient
import os

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("umqHl4LhqmPHcBh0L/freightwaves-news-scraper").call(
    run_input={"maxResults": 10}
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["title"])

FAQ

Can I scrape a single article?

Yes. Pass the article URL in startUrls and set maxResults to 1.

Does it scrape FreightWaves SONAR?

No. SONAR, private dashboards, and subscriber-only content are outside the actor's public-page scope.

Why is body null?

The public page may expose metadata without full text, or you may have set includeArticleBody to false. Check the record's warnings array.

Does the actor publish or modify content?

No. It is a read-only scraper for public pages.

How should I report a markup change?

Include the input URL, run ID, missing field, and a small example of the changed page in the Apify issue thread. Do not share account credentials or private content.

Legal note

This actor is provided for lawful access to public web content. You are responsible for complying with FreightWaves' terms, robots/access preferences, copyright rules, privacy law, and any other applicable requirements. Do not use it to bypass authentication, paywalls, rate limits, or technical access controls.