Survey Data Cleaner (CSV, SPSS .sav, Qualtrics)

Clean a survey export with fixed, documented rules and get back files you can check: the cleaned data, a reconstructed codebook, and a row-level change report listing every value that was changed (row, respondent ID, column, old value, new value, rule). Quality problems such as speeders, straight-lining and out-of-range answers are flagged, never deleted, so the decision stays with you.

Works with:

• CSV / TSV (comma, semicolon, tab or pipe; UTF-8, UTF-16 or Windows-1252),
• Qualtrics CSV/TSV exports (the question-text and ImportId rows under the header are detected, removed and kept as variable labels),
• SPSS .sav files (variable labels, value labels and user-missing values are read; a cleaned .sav is written back with its labels).

What it changes

Rule	What it does
qualtrics-header-row	Removes the Qualtrics question-text and ImportId rows under the header (the question text becomes the variable label)
whitespace	Trims leading/trailing spaces, collapses repeated spaces and non-breaking spaces, removes invisible characters (line breaks inside answers are kept)
missing-code	Replaces missing-value codes (NA, N/A, NULL, ., -99, -999 ... configurable) with an empty cell
spss-user-missing	Replaces values declared user-missing in the SPSS file with an empty cell (the value label, e.g. "Refused", is kept in the report)
duplicate-id	Removes later rows with a respondent ID already seen (the first row is kept)
number-format	In numeric columns, writes numbers plainly: 1,234 → 1234, 3.0 → 3, +5 → 5
date-format	Rewrites unambiguous YYYY/MM/DD dates as YYYY-MM-DD
category-case	In columns with 30 or fewer distinct answers, unifies spelling variants that differ only by case or spacing to the most common one

Every change except the Qualtrics header-row removal (qualtrics-header-row) can be switched off in the input (the Qualtrics extra header rows are always removed, because they are not answers). Flags (below) only mark rows; they never change your data.

What it flags (nothing is deleted)

Flag	Meaning
speeder	Completion time below a fraction (default 0.33) of the median duration
straight-lining	The same answer to every item of a question grid (columns named like Q5_1, Q5_2, ...; at least 5 items by default)
out-of-range	Outside a range you give (e.g. {"Q4": {"min": 1, "max": 5}}, patterns like Q3_* allowed), or not one of the labelled codes of an SPSS variable
not-a-number	Text in an otherwise numeric column
incomplete	Qualtrics response with Finished = 0 or Progress below 100
duplicate-row	Row identical to an earlier one when there is no respondent ID column

Flags go to FLAGS.csv, the dataset, and a qc_flags column in the cleaned file.

Output

In the run's key-value store:

• CLEANED.csv (UTF-8 with BOM so Excel opens it correctly) and, for SPSS input, CLEANED.sav;
• CODEBOOK.csv and CODEBOOK.json: per variable, the label, detected type, non-missing and missing counts, distinct values, range, value labels or observed answers with counts, missing codes found, SPSS format;
• CHANGES.csv: every change; FLAGS.csv: every flag;
• REPORT.html: a readable report of all of the above;
• OUTPUT: a JSON summary (rows in/out, counts by rule).

The dataset holds one item per change and per flag (up to 5,000 by default; the CSV files always hold all of them). Row numbers refer to the original file: for CSV, the spreadsheet row (header = row 1); for SPSS, the case number.

Input

Give a File URL (a direct download link that works without a login), or upload the file to a key-value store in your Apify account (Storage tab) and pick that store; the first .csv, .tsv, .txt or .sav record is used unless you name its key. Everything else is optional: the respondent ID column, missing-value codes, allowed ranges, duration column, speeder threshold and grid size.

With no file at all, the actor cleans a small built-in demo file (a made-up Qualtrics export with 30 responses) so you can see the output; demo runs are not charged.

Limits

• The rules are mechanical. They don't know your study design: check the change report before analysis, and remove codes such as -99 from the missing list if they are real answers in your data.
• Standardizing missing codes merges different kinds of missing (e.g. "refused" and "don't know") into empty cells; the change report and the codebook keep the original codes. Switch the rule off if you need them distinct.
• Qualtrics value labels can't be recovered from a single CSV export; the codebook lists the observed answers instead.
• Dates in day/month or month/day order are left as they are. Excel (.xlsx) and Stata files are not read yet.
• One file per run; files up to 100 MB by default (raise memory for large files).

Your data

The file is read only during the run. Madrasco does not receive, keep or see a copy; the outputs are stored in your own Apify run storage and follow your account's retention settings. Survey data often contains personal information: only process data you are allowed to process, and consider removing names and contact details before upload.

Support

Open an issue in the Issues tab. This actor is built and maintained by Madrasco with AI assistance; a human owner can be reached on request.