# Shiply Freight Marketplace Scraper (`muhammadafzal/shiply-com-freight-marketplace-scraper`) Actor

Extract public Shiply freight listings with routes, distance, quote counts, urgency, and listing URLs for carrier and logistics research.

- **URL**: https://apify.com/muhammadafzal/shiply-com-freight-marketplace-scraper.md
- **Developed by:** [Muhammad Afzal](https://apify.com/muhammadafzal) (community)
- **Categories:** Automation, Other
- **Stats:** 2 total users, 1 monthly users, 62.5% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $11.00 / 1,000 shiply listings

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Shiply Freight Marketplace Scraper

> **Free-plan limit:** Apify free-plan runs can return at most **5 dataset records per run** from this Actor, even if you request more.

Collect public transport-job listings from Shiply country marketplaces into a stable dataset for carrier prospecting, lane research, demand monitoring, and logistics analysis. The Actor reads public search-result pages; it does not log in, place quotes, message users, or access private delivery details.

### What it extracts

Each dataset item represents one unique public Shiply listing.

| Field | Meaning |
|---|---|
| `listingId` | Stable ID from the public listing URL |
| `title` | Item or shipment title |
| `listingUrl` | Public Shiply listing link |
| `country` | Selected Shiply marketplace |
| `origin`, `destination` | Public route endpoints shown in search |
| `distance`, `distanceUnit` | Numeric route distance and `mi`/`km` unit |
| `postedAtText` | Source date or relative posting time, when shown |
| `quotesCount` | Number of providers quoting, when shown |
| `isUrgent`, `isFeatured` | Marketplace status flags |
| `thumbnailUrl` | Search-result thumbnail, when present |
| `sourcePageUrl`, `scrapedAt` | Provenance and UTC collection time |

Nullable fields remain `null` when Shiply does not show them. The Actor never invents missing locations, dates, quotes, or images.

### When to use it

- Carriers and dispatch teams: watch public jobs on preferred Shiply marketplaces.
- Freight brokers: identify public lane demand and quoting activity.
- Market analysts: compare origins, destinations, mileage, and listing volume over time.
- Data teams: schedule bounded snapshots and export JSON, CSV, Excel, XML, or RSS from Apify.
- AI agents: call the Actor through Apify MCP and consume the declared dataset schema.

Do not use this Actor to access private Shiply account data, accepted-job addresses, messages, customer contact details, or login-only listing details. It does not submit quotes or bypass CAPTCHAs.

### Input

| Field | Default | Purpose |
|---|---:|---|
| `countries` | `["uk"]` | One or more Shiply country presets: UK, US, Canada, Australia, India, Germany, France, Spain, Italy, Brazil, or Mexico |
| `startUrls` | `[]` | Optional public Shiply `/search` or `/listings/search` URLs |
| `maxResults` | `50` | Hard cap on unique dataset rows and result-event cost |
| `maxPagesPerSearch` | `5` | Pagination cap for every selected search |
| `maxConcurrency` | `2` | Simultaneous browser pages, capped at 5 |
| `requestDelayMillis` | `1000` | Polite per-request delay in milliseconds |
| `proxyConfiguration` | Apify Proxy | Network route configuration |

Small Canada run:

```json
{
  "countries": ["ca"],
  "maxResults": 10,
  "maxPagesPerSearch": 1,
  "maxConcurrency": 1,
  "requestDelayMillis": 1000
}
```

Multiple marketplaces:

```json
{
  "countries": ["uk", "us", "ca"],
  "maxResults": 100,
  "maxPagesPerSearch": 3
}
```

Custom public search page:

```json
{
  "countries": [],
  "startUrls": [{ "url": "https://www.shiply.com/ca/listings/search" }],
  "maxResults": 25
}
```

### Example output

```json
{
  "listingId": "9NKNEB3D9",
  "title": "Dodge Charger",
  "listingUrl": "https://www.shiply.com/ca/transport/Dodge-Charger/9NKNEB3D9",
  "country": "Canada",
  "origin": "Calgary, Alberta, T2G",
  "destination": "Ottawa, Ontario, K1P",
  "distance": 3487,
  "distanceUnit": "km",
  "postedAtText": null,
  "quotesCount": 3,
  "isUrgent": true,
  "isFeatured": false,
  "thumbnailUrl": null,
  "sourcePageUrl": "https://www.shiply.com/ca/search",
  "scrapedAt": "2026-08-28T10:00:00.000Z"
}
```

The `OUTPUT` key-value record classifies the run as `DATA`, `EMPTY`, `BLOCKED`, `REJECTED`, or `FAILED` and reports delivered rows, event counters, challenge pages, failed requests, warnings, and completion time. Diagnostics stay out of the listing dataset so exports remain homogeneous.

### Pricing

This Actor uses **pay-per-event** pricing. These are the current Apify Store event prices:

| Event | Price (USD) | When it is charged |
|---|---:|---|
| `apify-default-dataset-item` | FREE: $0.015; BRONZE: $0.014; SILVER: $0.013; GOLD: $0.011; PLATINUM: $0.011; DIAMOND: $0.011 | Shiply listing — One unique, schema-valid public Shiply marketplace listing written to the default dataset. |
| `apify-actor-start` | $0.005 | Actor Start — Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event). — Charged once per run. |

Apify platform usage for this Actor run is included alongside the event prices above; no separate per-run platform-usage fee is passed to users.

This covers Apify platform usage for this Actor run. Other Apify products or usage outside this Actor run may still follow your account plan.

### Reliability and failure behavior

Shiply currently protects public search pages with Cloudflare. The Actor uses a real browser, coherent cookies and sessions, bounded retries, a configurable delay, and Apify Proxy support. It detects challenge pages instead of parsing them as empty data. A blocked run writes no fabricated records and no result events, then points to `OUTPUT` for diagnostics.

Pagination follows Shiply's 25-result offsets (`/25`, `/50`, and so on). Records are deduplicated by marketplace and listing ID before writing. A page-count signal is used when available; otherwise pagination continues only after a full 25-row page. Concurrency, retry count, pages, and results are bounded to avoid loops and unexpected spend.

Public markup, availability, and bot controls can change. If a country preset changes its public route, provide the current public search page through `startUrls` and report the preset issue with the run ID. Lowering concurrency and using a proxy geography aligned with the marketplace can improve reliability, but this Actor will not solve CAPTCHAs or defeat an access control.

### API and MCP

Run through the Apify API with `run-sync-get-dataset-items`, schedule it in Console, connect webhooks, or add the Actor to Apify MCP. Inputs and outputs use stable field names and explicit nullability so agents can route calls and interpret empty versus blocked outcomes.

Example JavaScript client call:

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('YOUR_USERNAME/shiply-com-freight-marketplace-scraper').call({
  countries: ['ca'],
  maxResults: 25,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
```

### Legal and privacy

This unofficial Actor is not affiliated with or endorsed by Shiply Ltd. Collect only public data you are authorized to use. Review Shiply's current terms, robots directives, and applicable laws before running at scale. Respect privacy, database rights, GDPR/UK GDPR, CCPA, contractual restrictions, and deletion obligations. Do not use results for spam, harassment, unlawful profiling, or attempts to identify or contact private individuals. Locations are reproduced only at the public precision Shiply displays.

For support, include the Apify run ID, redacted input, expected marketplace, and whether `OUTPUT.outcome` was `EMPTY`, `BLOCKED`, or `FAILED`.

# Changelog

This Actor's version history is a separate document: https://apify.com/muhammadafzal/shiply-com-freight-marketplace-scraper/changelog.md

# Actor input Schema

## `countries` (type: `array`):

Use this to select Shiply country marketplaces. Choose one or more country codes; the default is the United Kingdom. This does not filter arbitrary origin or destination countries inside a listing.

## `startUrls` (type: `array`):

Use this when you already have a public Shiply search page. Accepts HTTPS Shiply URLs ending in /search or /listings/search, for example https://www.shiply.com/ca/search. Detail, account, and quote URLs are rejected.

## `maxResults` (type: `integer`):

Use this to cap dataset rows and predictable pay-per-result cost. The Actor stops after this many unique public listings. Verified Apify free-plan runs are capped at 5 listings; paid and local/unknown runs retain the requested limit. Default 50; allowed range 1–10,000.

## `maxPagesPerSearch` (type: `integer`):

Use this to bound pagination for every selected country or custom URL. Each page normally contains up to 25 listings. Default 5; allowed range 1–100.

## `maxConcurrency` (type: `integer`):

Use this to control simultaneous browser pages. Lower values are gentler when Shiply rate-limits requests. Default 2; allowed range 1–5.

## `requestDelayMillis` (type: `integer`):

Use this to add a polite delay before parsing each page. Default 1000 ms; allowed range 0–30,000 ms. This is not a CAPTCHA bypass.

## `proxyConfiguration` (type: `object`):

Use this when Shiply challenges your network route. Apify Proxy is enabled by default; select only proxy groups and countries available to your account.

## Actor input object example

```json
{
  "countries": [
    "ca"
  ],
  "startUrls": [],
  "maxResults": 10,
  "maxPagesPerSearch": 5,
  "maxConcurrency": 2,
  "requestDelayMillis": 1000,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Public Shiply freight marketplace listings written to the default dataset.

## `summary` (type: `string`):

Outcome, delivered count, billing counters, warnings, and completion time.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "countries": [
        "ca"
    ],
    "maxResults": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("muhammadafzal/shiply-com-freight-marketplace-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "countries": ["ca"],
    "maxResults": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("muhammadafzal/shiply-com-freight-marketplace-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "countries": [
    "ca"
  ],
  "maxResults": 10
}' |
apify call muhammadafzal/shiply-com-freight-marketplace-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,muhammadafzal/shiply-com-freight-marketplace-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/n3eLALzXrqTSqqqjL/builds/nNSUMThB8Ki0x6CkA/openapi.json