Off by default. Keeps one row when a board posts the same title at the same stated location more than once, and records how many copies were collapsed in the row's duplicates_merged field. Same title at a DIFFERENT location is never touched — a retailer posting "Sales Associate" in seventy-six towns is filling seventy-six real openings, and the store is in the location field, not the title. Postings with no stated location are never collapsed either: unstated is not the same as same-place. Corpus-wide this removes 3.03% of open postings (95% CI 2.86-3.22%), measured on 2026-09-03 by reading 1,317 boards live — 166,320 postings, 57% of the corpus — drawn by board size, never by how much a board repeats itself. Read the shape before you decide: the rate falls off a cliff with board size (500+ postings 7.60%, 100-499 2.87%, 30-99 1.64%, 10-29 0.90%, 3-9 zero), and it is concentrated — only 34% of boards carry a single duplicate, and the worst 10 account for 47% of them. If you pull small or mid-size boards it has nothing to find. It is also a lower bound: we strip noise words but resolve nothing, so "Leeds" and "Leeds, UK" stay two places. Why this is off while the other two Quality filters are on: they remove rows that are not paid openings at all, whereas three requisitions for three headcount at one site look exactly like three copies of one job and the ATS does not publish headcount. Dropping a real opening is worse than carrying a duplicate, so the choice is yours. RUN_STATS.duplicates_merged reports the count whether this is on or off, so you can see what it would take before turning it on.
