Export Datasets to SFTP, FTP, Dropbox & Google Drive avatar

Export Datasets to SFTP, FTP, Dropbox & Google Drive

Pricing

from $4.50 / 1,000 file delivereds

Go to Apify Store
Export Datasets to SFTP, FTP, Dropbox & Google Drive

Export Datasets to SFTP, FTP, Dropbox & Google Drive

Deliver any scraper's dataset as a CSV, JSON, JSONL or Excel file (optionally gzipped) to your own SFTP or FTPS server, Dropbox or Google Drive after every run. Atomic uploads, SFTP host-key pinning, retries, and one destination failing never blocks the others. $0.0045 per file delivered.

Pricing

from $4.50 / 1,000 file delivereds

Rating

0.0

(0)

Developer

Charles Markette

Charles Markette

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

a day ago

Last modified

Categories

Share

Dataset Delivery exports any Apify dataset as a CSV, JSON, JSONL or Excel (XLSX) file, optionally gzipped, and uploads it to your own storage: an SFTP server, an FTP/FTPS server, Dropbox or Google Drive. Connect it as the last step after any scraper, and every run drops a fresh file where your client, your BI tool (Power BI, Tableau, Looker), your data warehouse loader or your colleagues pick it up. It's for agencies and freelancers delivering scraped data to clients, data teams with an SFTP drop folder, and anyone who wants scraper results as files in Dropbox or Google Drive without writing upload code.

  • SFTP, with password or private-key login and host-key pinning: the way most enterprise data handoffs work.
  • FTP and FTPS (explicit TLS), in passive mode.
  • Dropbox, through your own Dropbox app (a long-lived refresh token), including files far above Dropbox's 150 MiB single-upload limit.
  • Google Drive, through a service account, into a folder you share with it (best in a shared drive).

Why use Dataset Delivery?

  • Atomic uploads. On SFTP and FTP the file is written as <name>.part and renamed into place only when it's complete, so a job watching the folder never picks up half a file. Overwriting uses an atomic POSIX rename where the server supports it, and never deletes the old file before the new one is in place. For drop folders that react to any new file (or forbid renames), choose Upload mode: direct.
  • Safe with SFTP. Pin the server's host-key fingerprint and the Actor refuses any other server before it sends the password or key. Password login without a pin is refused by default (the error tells you the server's fingerprint to pin), because without it a man-in-the-middle could capture the password and the file.
  • Independent destinations. Fill in any combination; each one is delivered and retried on its own, so a broken FTP login never stops the SFTP and Dropbox deliveries. The run fails only when every destination failed.
  • Retries the right things. SFTP and FTP connections get 3 attempts with backoff; Dropbox and Google Drive requests are retried on rate limits (HTTP 429) and server errors (5xx), honouring Retry-After. Big Dropbox uploads use upload sessions, big Drive uploads are resumable and continue from where the connection dropped. Wrong passwords, a host-key mismatch, a folder not shared with the service account or a Dropbox path outside an App-folder app fail at once, with the fix in the message.
  • Streams big datasets. The dataset is read in pages into a temp file on disk, never held in memory, and Excel files are written in openpyxl's write-only mode.
  • Spreadsheet-safe. CSV values starting with =, +, - or @ get a leading ' so Excel and Google Sheets don't run scraped text as a formula; Excel cells are always stored as text, never formulas. Nested fields are flattened into columns like seller/name, as in Apify's own CSV export.
  • Pay only for what arrived. Empty datasets, dry runs, skipped and failed destinations aren't charged as deliveries. A restarted run skips destinations it already recorded as delivered and isn't charged for them again (see Limitations for the small gap).
  • Your secrets stay secret. Hosts, user names, passwords, keys and tokens are stored encrypted. Hosts, passwords, keys and tokens are masked in every log line, error message and output (including passwords in user:password@ URLs, Dropbox and Google access tokens, and signed JWTs); user names are masked wherever they appear as a separate word.

Use cases

  • Client data delivery: a daily CSV of scraped prices, leads or listings on the client's SFTP server, named by date.
  • BI and data warehouse loads: drop gzipped CSV or JSONL files into a folder that Power BI, Tableau, Looker or a warehouse loader picks up.
  • Shared reports: an Excel file in a Google Drive shared drive or a Dropbox folder that colleagues open directly; with overwrite, the Drive link stays the same.
  • Archiving: keep a dated JSONL copy of every crawl in Dropbox or on your own server.
  • Several destinations at once: the same file to SFTP and Google Drive in one run, each delivered and retried on its own.

How to use Dataset Delivery

Setup per destination

Fill in only the destinations you want. To check a setup, run the Actor once without a dataset: it delivers a small test file with 3 sample items (charged like a normal delivery). Or turn on Dry run (free) to build the file and see its name, size and row count without uploading.

SFTP (about 3 minutes)

  1. Ask for (or create) an SFTP user with write access to a folder. Fill in SFTP host, port (usually 22), username, and the password or a private key.

  2. Key login (recommended): make a key pair with ssh-keygen -t ed25519 -f apify_delivery (ssh-keygen manual). Add the public key (apify_delivery.pub) to the user's ~/.ssh/authorized_keys on the server (sshd: AUTHORIZED_KEYS FILE FORMAT), and paste the whole private key file (apify_delivery, starting with -----BEGIN OPENSSH PRIVATE KEY-----) into SFTP private key. If the key has a passphrase, put it in Key passphrase. Ed25519, ECDSA and RSA keys work, in OpenSSH or PEM format. PuTTY .ppk keys need converting first (PuTTYgen → Conversions → Export OpenSSH key).

  3. Pin the host key: run once without a fingerprint. With a password, the run stops before logging in and its error shows This server's host key is SHA256:… (ssh-ed25519); with a key, it delivers and logs a warning with the same fingerprint. Paste that SHA256:… value into Host key fingerprint. Better still, compare it with what the server's administrator gets from ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub on the server (the format is the one ssh-keygen -l prints, SHA-256 by default). From then on, any other server is refused before logging in. A server has one key per type (Ed25519, ECDSA, RSA); you can paste several fingerprints separated by commas.

    Without a pin, the server's identity isn't checked: a man-in-the-middle can capture the password and the file. So password login without a pin is refused unless you turn on Allow an unpinned host key with a password (at your own risk). Key login without a pin is allowed (the key itself can't be captured, but the file can), with a warning in every run.

  4. Set SFTP remote folder (absolute like /data/outbound, or relative to the login folder). It's created if missing.

FTP / FTPS

Fill in FTP host, port (usually 21), username and password. FTP security defaults to explicit TLS (FTPS, AUTH TLS); choose none only for a trusted network, because plain FTP sends the password and the file unencrypted. For a server with a self-signed certificate you trust, turn off Verify the TLS certificate; the run then logs a warning, because without verification a man-in-the-middle could capture the password and the file. Implicit FTPS (port 990) isn't supported. Passive mode is on by default; data connections reuse the TLS session, which servers like vsftpd (require_ssl_reuse) insist on.

Dropbox (about 3 minutes, once)

  1. In the Dropbox App Console, click Create app, choose Scoped access and either App folder (the app can only use its own folder, /Apps/<app name>/; safest) or Full Dropbox. (Dropbox: OAuth guide)
  2. On the app's Permissions tab, tick files.content.write and files.metadata.read, and click Submit.
  3. On the Settings tab, copy the App key and App secret into Dropbox app key and Dropbox app secret.
  4. Get a refresh token (long-lived, it doesn't expire on its own): open this URL in your browser with your app key, click Allow and copy the code it shows: https://www.dropbox.com/oauth2/authorize?client_id=<APP KEY>&response_type=code&token_access_type=offline Then exchange the code for tokens in a terminal:
    curl https://api.dropboxapi.com/oauth2/token -d code=<CODE> -d grant_type=authorization_code -d client_id=<APP KEY> -d client_secret=<APP SECRET>
    Copy refresh_token from the answer into Dropbox refresh token. (Dropbox: authorization, Using OAuth 2.0 with offline access)
  5. Set Dropbox folder, like /reports. With an App folder app this is inside /Apps/<app name>/, so write /reports, not /Apps/<app name>/reports.

For a quick test you can instead paste a short-lived token from the App Console's Generate button into Dropbox access token; it expires after a short time, so scheduled runs need the refresh token.

Google Drive (about 5 minutes)

  1. In the Google Cloud console, pick or create a project and enable the Google Drive API.
  2. Under IAM & Admin → Service accounts, create a service account (no roles needed), open it, and under Keys → Add key → Create new key → JSON download its key (Google: create service account keys). Paste the whole file into Service account key (JSON).
  3. Use a shared drive. Service accounts have no storage quota of their own and can't own files, so they can only create files in a shared drive (or upload on behalf of a person with OAuth, which this Actor doesn't do). In Google Drive, open the shared drive (or a folder in it) → Manage members, add the service account's client_email (…@….iam.gserviceaccount.com) with the Contributor or Content manager role. Google's own words: "Service accounts don't have storage quota and can't own any files", so they must "upload files and folders into shared drives" (Drive API: resolve errors, storageQuotaExceeded). Shared drives are part of Google Workspace.
  4. Open the folder and copy its link (https://drive.google.com/drive/folders/…) into Folder.

A folder in someone's My Drive shared with the service account as Editor is accepted, and the run warns you: creating a new file there fails with storageQuotaExceeded (the Actor then explains this). Every Drive call sets supportsAllDrives=true, which Google requires for apps that work with shared drives (Drive API: shared drives support).

Deliver a file after every scraper run (integration)

  1. Open your scraper's Actor or saved task, go to the Integrations tab and click Add integration.
  2. Search for Dataset Delivery, choose to start it when the run succeeded, and set the input. Apify fills in {{resource.defaultDatasetId}} with that run's dataset:
{
"datasetId": "{{resource.defaultDatasetId}}",
"format": "csv",
"gzip": true,
"fields": ["title", "price", "url", "seller.name"],
"fileNameTemplate": "exports/{date}/{actorName}-{datetime}.{ext}",
"ifExists": "overwrite",
"onlyIfItems": true,
"sftpHost": "sftp.client.example.com",
"sftpPort": 22,
"sftpUsername": "apify",
"sftpPrivateKey": "-----BEGIN OPENSSH PRIVATE KEY-----\n…\n-----END OPENSSH PRIVATE KEY-----",
"sftpHostKeyFingerprint": "SHA256:…",
"sftpRemoteDir": "/inbound/prices",
"dropboxAppKey": "…",
"dropboxAppSecret": "…",
"dropboxRefreshToken": "…",
"dropboxFolder": "/Prices"
}
  1. Save, and test the integration on a past run.

The file name uses single braces ({date}), which this Actor fills in; Apify itself fills in double-brace variables in an integration's input, so keep {{…}} for Apify's own (like {{resource.defaultDatasetId}}). Each run then delivers, for example, /inbound/prices/exports/2026-10-13/amazon-scraper-2026-10-13T08-30-00Z.csv.gz to the SFTP server and the same file to Dropbox.

Only new items: put Merge & Deduplicate Datasets in "only new" mode between the scraper and this Actor (connect this Actor to the dedupe Actor's runs) to deliver only items that weren't in earlier files.

Input example

A weekly Excel report into a Google Drive shared drive, replacing last week's file under the same link:

{
"datasetId": "aBcDeFgHiJkLmNoPq",
"format": "xlsx",
"fileNameTemplate": "weekly-leads.{ext}",
"ifExists": "overwrite",
"xlsxSheetName": "Leads",
"googleServiceAccountKey": "{ \"type\": \"service_account\", … }",
"googleDriveFolder": "https://drive.google.com/drive/folders/…"
}
FieldNotes
DatasetThe dataset to deliver. In integrations use {{resource.defaultDatasetId}}. Empty = a test file with 3 sample items (delivered and charged like a normal file).
File formatcsv (default), json (one array), jsonl (one item per line) or xlsx. CSV and XLSX flatten nested fields into columns like address/city and images/0.
Compress with gzipCSV, JSON and JSONL: delivers .csv.gz, .json.gz, .jsonl.gz. Ignored for XLSX, which is already compressed.
Fields / Remove these fieldsPick fields (in that order, dots for nested ones, named as you wrote them; missing ones are empty) and/or drop fields (dots for nested).
File nameDefault {actorName}-{date}.{ext}. Placeholders: {date} (2026-10-13, UTC), {datetime} (2026-10-13T08-30-00Z, UTC), {runId}, {datasetId}, {actorName}, {ext} (csv, csv.gz, xlsx …). {{date}} works too in manual runs, but in integrations use single braces. / makes sub-folders (created as needed). Placeholder values are reduced to letters, digits, ., _ and -; a name with .., a leading /, a backslash, a control character, an empty folder name or a stray {/} is refused before the file is built.
Name for {actorName}Default: the name of the Actor whose run made the dataset (looked up on the platform), else the dataset's name, else dataset.
If a file existsoverwrite (default; atomic on SFTP; on Google Drive the same file gets a new version, so its link stays the same), skip (that destination gets nothing and isn't charged), or addSuffix: report-1.csv, report-2.csv … (Dropbox names it report (1).csv itself).
Upload modeSFTP and FTP: atomic (default: <name>.part, then rename) or direct (writes the final name, no .part, no rename; a watcher may see the file while it's still being written).
Only if the dataset has itemsDefault on: empty datasets deliver nothing and are free. Off: an empty dataset delivers a header-only CSV (from Fields), [], an empty JSONL or an Excel sheet with just the header.
Dry runBuilds the file and reports its name, size and row count per destination, without uploading (free). Works with no destination configured too.
CSV delimiter / UTF-8 BOM / formula protectionComma (default), semicolon, tab or pipe. BOM off by default (turn it on for Excel with accented characters). Formula protection on by default.
Excel sheet name / When rows don't fit one sheetDefault Data. A sheet holds 1,048,576 rows including the header: split (default) continues on Data (2), Data (3) …; fail delivers nothing and says so.
SFTP host, port, username, password, private key, key passphrasePassword or key (or both: the key is tried first).
Host key fingerprint / Allow an unpinned host keySHA256:… as ssh-keygen -l prints it (with or without SHA256: and the trailing =); several separated by commas. Empty: password login is refused (the error shows the fingerprint) unless Allow an unpinned host key with a password is on; key login works with a warning.
SFTP / FTP remote folderAbsolute or relative to the login folder; created if missing. Empty = the login folder.
FTP host, port, username, password, security, verify certificate, passive modeEmpty username = anonymous. Security tls (default) or none.
Dropbox app key, app secret, refresh token / access token, folderRefresh token + key + secret, or a short-lived access token.
Service account key, FolderThe JSON key, and the folder's link or ID.

Hosts, user names, passwords, private keys, passphrases, Dropbox keys and tokens, and the service account key are secret inputs, stored encrypted. Ports are plain numbers (Apify's secret inputs are text, objects and arrays only).

Output example

The run's dataset has one row per configured destination:

[
{
"destination": "sftp",
"status": "delivered",
"remotePath": "/inbound/prices/exports/2026-10-13/amazon-scraper-2026-10-13T08-30-00Z.csv.gz",
"bytes": 184213,
"rows": 2500,
"format": "csv.gz",
"datasetId": "aBcDeFgHiJkLmNoPq",
"error": null,
"message": "Uploaded 184,213 bytes (posix-rename).",
"url": null,
"target": "SFTP port 22, key login, host key pinned",
"attempts": 1,
"deliveredAt": "2026-10-13T08:30:12+00:00"
}
]
  • destination is sftp, ftp, dropbox or googleDrive; status is delivered, skipped, failed or dry run; remotePath is where the file landed (the full path on SFTP/FTP/Dropbox, <folder name>/<file> on Drive); url is the Drive link; error says why a destination failed, with secrets masked.
  • The key-value store OUTPUT holds the file name, format, size, rows, columns, sheets, notes (for example cells cut to Excel's 32,767 characters), status (done, partial, failed, skipped or dry run) and the same per-destination rows.
  • With nothing configured at all (no dataset, no destination: the Store's daily test with the default input), the run returns a single row with setup instructions and exits successfully. A run with a dataset but no destination fails with a clear message.
  • The run fails (non-zero exit) only when every configured destination failed, or when the input itself is wrong (an unknown placeholder, a dataset that can't be read, XLSX over the row limit in fail mode).

How much does it cost to deliver files?

Dataset Delivery uses pay-per-event pricing: $0.0045 per file delivered, plus Apify's $0.00005 run start fee. One charge per destination that got the file: delivering one dataset to SFTP and Dropbox is two files delivered. Test files (a run without a dataset) are real deliveries and are charged. Empty datasets that are skipped, dry runs, destinations skipped because the file exists, failed destinations, and the result rows in the dataset aren't charged as deliveries; only the start fee applies.

Examples use the Free-plan rate. Where an example doesn't add it, each run also has its run-start charge, counted per GB of Actor memory (minimum one; one at the default memory); Apify Proxy (where used) is billed separately.

Worked example: a scraper that runs every day and delivers its CSV to an SFTP server and to Google Drive delivers 2 files per run: 2 × $0.0045 + $0.00005 = $0.00905 per run, about $0.27 per month. The file's size or row count doesn't change the price.

If you set a maximum cost per run, the Actor checks it before building and uploading anything and delivers to only as many destinations as it allows (in the order SFTP, FTP, Dropbox, Google Drive), marking the rest as skipped.

FAQ

SFTP refuses to log in with my password and shows a fingerprint.

Password login without a pinned host key is refused by default, because without the pin a man-in-the-middle could capture the password. Copy the SHA256:… value from the error into Host key fingerprint (ideally after checking it with the server's administrator) and run again.

Google Drive fails with "storageQuotaExceeded".

Service accounts have no storage of their own, so they can create files only in a shared drive. Add the service account to a shared drive (or a folder in one) as Contributor or Content manager, and use that folder.

My Dropbox token stopped working.

Short-lived access tokens from the App Console's Generate button expire. For scheduled runs use the app key, app secret and a refresh token (see Dropbox setup).

My FTPS server uses port 990.

That's implicit FTPS, which isn't supported. Use the server's explicit TLS port (usually 21).

The run ran out of disk space.

A run's disk is twice its memory. Give the run more memory, or use gzip, JSONL or fewer fields.

Can a job watching the folder pick up a half-written file?

Not in the default atomic mode on SFTP and FTP: the file is uploaded as <name>.part and renamed when complete. Use direct only if your drop folder forbids renames.

Can I deliver to Amazon S3, OneDrive or email?

Not with this Actor: it delivers to SFTP, FTP/FTPS, Dropbox and Google Drive. To email a CSV of the results, use Send Email Report & Slack, Discord, Teams, Telegram Alerts with an attachment.

Limitations

  • Restarts: after each successful delivery, the destination is recorded in the run's key-value store together with the file name, and a restarted run (for example after a platform migration) skips recorded destinations and reuses the same file name. The gap: if the run is restarted in the moment after a file arrived but before that record was saved, the restarted run uploads that destination again (with overwrite it replaces the first copy, with addSuffix a second copy appears) and may charge it again, because the charge's idempotency key protects only against a repeat within a few minutes.
  • SFTP: 3 connection attempts with backoff (on refused or dropped connections, timeouts and failed handshakes; a wrong password, a refused key or a host-key mismatch fail at once). Timeouts: 20 seconds to connect, read the banner, negotiate keys and log in; 120 seconds for any single read or write. Overwrite renames with the posix-rename@openssh.com extension (atomic replace); if the server doesn't support it, with a plain rename; if that refuses because the file exists, the old file is renamed to <name>.<random>.bak, the new one renamed into place, and only then the .bak deleted (for a moment the name doesn't exist). If moving the old file aside fails, nothing is deleted and the destination fails; if the new file can't be renamed into place, the old one is put back. Permission denied on the rename is final. Once the file is in place, the delivery counts as done even if reading back its absolute path fails. skip and addSuffix never replace a file. DSA keys aren't supported.
  • FTP: the same 3 attempts and timeouts. After the upload the size is checked with SIZE where the server supports it. Some servers refuse to rename over an existing file; then the old file is moved aside to a .bak name first, exactly as on SFTP, so it's never deleted before the new file is in place. A server that won't encrypt the data connection (PROT P) fails with a clear message. Existence checks use SIZE (or a folder listing on servers without it).
  • Dropbox: files up to 150 MiB go up in one request ("Do not use this to upload a file larger than 150 MiB", files/upload); bigger ones through an upload session in 8 MiB chunks (upload_session/start, append_v2, finish), up to Dropbox's per-file maximum of 2^41 − 2^22 bytes (about 2 TB). If an answer gets lost and Dropbox reports a different offset, the upload continues from Dropbox's correct_offset. Paths and names may use any characters: they're sent in the Dropbox-API-Arg header as JSON with 0x7F and every non-ASCII character escaped as \uXXXX, as Dropbox requires. An expired access token is refreshed once and the request repeated.
  • Google Drive: files up to 5 MB go up in one multipart request (new files) or media request (new version of an existing file); bigger ones as a resumable upload in 8 MiB chunks (multiples of 256 KiB, as Google requires), resumed from the server's Range after an interruption, and restarted once if the session expired. Google's per-file maximum is 5,120 GB (files.create). Overwrite uses files.update on the existing file (same ID and link, a new revision); if several files have the same name, the most recently modified one is updated. The stored file's MD5 checksum is compared with the local one. A shared drive folder holds at most 500,000 items.
  • Retries over HTTP: up to 5 attempts per request on HTTP 408, 429, 5xx (and Drive's 403 rate-limit errors) and network errors, waiting at most 120 seconds at a time and 5 minutes in total. A service asking for a longer wait counts as failed for this run.
  • Excel: 1,048,576 rows and 16,384 columns per sheet, 32,767 characters per cell (longer values are cut, and the run says how many). Integers beyond 2^53 are written as text so no digits are lost. Control characters Excel can't store are removed.
  • Columns: CSV and XLSX get one column per flattened field, in the order fields first appear (with Fields, in your order). More than 50,000 columns (usually long arrays) fails with a hint to pick fields or use JSON/JSONL.
  • Memory and disk: the dataset is read in pages of about 16 MB of JSON (the first page is 100 items; then 10 to 10,000 items, depending on item size), so memory use depends on the page, not the dataset. The file is built once in a temp folder that's deleted at the end. A run's disk on Apify is twice its memory (2 GB at the default 1 GB). JSON and JSONL need disk for the output file only; CSV and XLSX first stage the flattened rows (about the size of the JSON), then write the output next to them, and delete the staged copy right after, so they need roughly the JSON size plus the output size. Running out of disk stops the run with a message to raise the run's memory (or use gzip, JSONL or fewer fields).
  • Not affiliated with Dropbox or Google.

Dataset Delivery is part of a small suite of Actors that work together after any scraper:

ActorUse it to
CSV & Excel to DatasetThe other direction: turn CSV, Excel or Google Sheets files into a dataset.
Merge & Deduplicate DatasetsDeliver only items that weren't in earlier runs.
Google Sheets SyncKeep the same data in a live Google Sheet instead of a file.
Send Email Report & Slack, Discord, Teams, Telegram AlertsTell your team in Slack or by email when a new file is ready.
SEC Funding Rounds TrackerDeliver a daily file of newly funded companies from SEC filings.
Vector DB SyncLoad the same crawl into Pinecone or Qdrant for RAG.
Phone Number Validator & FormatterClean and format phone numbers before delivering a lead list.
Dataset Filter & TransformDrop, rename or flatten columns so the file matches what the receiver expects.