CSV

A grid, not a tree and not a line diff. Which record changed, and which cell in it — with rows paired on a key column so the order two exports came out in stops mattering.

Click to enlarge

A grid, because that is the question#

The reader's question about two CSVs is which cell changed, and neither a structural tree nor a line diff can answer it. So this engine has its own row model and its own view: a sticky header, a row-number gutter, and a changed cell showing the old value beside the new one.

The gutter shows both record numbers — 12 → 9 for a row that moved — because "row 412 of the export" is how anyone talks about a CSV. Cell states are their own thing: a cell is unchanged, changed, added, removed, or ignored — the last of those when a normalisation rule matched it, or when its column is excluded from the comparison. Either way its difference is still counted as suppressed rather than quietly dropped.

Columns are compared as well as rows. The union of both headers is kept in the before file's order, with columns only the after file has appended, and a header cell is marked when only one side has it. Reading a table in a different column order is unreadable even when every value is present, which is why the order is not normalised away.

Pairing rows on a key column#

Two exports of the same table usually differ in row order for no reason at all — whatever the database felt like that afternoon. Pick a key column from the toolbar and order stops mattering: rows pair on that value, so an edited row is an edited row and a moved row is not a change at all.

Pairing rows by position#

Without a key, rows are aligned first and then paired. Each row becomes a signature; the two lists of signatures are aligned; and where a block of removals is followed by a block of additions, the two blocks are zipped and each zipped pair kept as one modification when the rows resemble each other — at least half their comparable cells the same.

Without the alignment step, inserting one row at the top would report every row below it as modified. Without the zipping, every edit would be a removal plus an addition.

Two things that make position pairing work#

Similarity needs enough columns to judge by. A one-column table can never clear any threshold — a changed cell means zero cells match — so a narrow table would report every edit as a deletion plus an addition. Below three comparable columns the engine trusts the alignment instead and keeps the pair. Text lines are the opposite case, which is why the text diff needs no such exception.

Row signatures are masked too, not just cell comparisons. When a normalisation rule is on — a regenerated id, a build timestamp — the mask is applied to the signature as well. Otherwise two rows differing only in a regenerated id fail to align, and every row below them reads as changed while the rule that should have hidden the difference sits there switched on.

Delimiters, and TSV#

The delimiter is sniffed by counting candidates — comma, tab, semicolon, pipe — outside quoted fields, across the first 64 KB. Counting raw occurrences would pick the comma for a semicolon-delimited file whose fields contain commas, which is most European exports.

A .tsv is tab-delimited by its name, settled before the sniffer gets a chance: the extension says so definitively and the content may contain either character.

The parser is ours and follows RFC 4180 in one pass over the characters, which covers the three cases that catch every hand-rolled implementation:

a,"b,c",d          → three fields: a | b,c | d
"one
two",three        → two fields; a record is not a line
"say ""hi"""      → one field: say "hi"

CRLF and a lone CR both end a record, a trailing newline does not start an empty one, and rows whose field count differs from the first row's are counted and named in the notes rather than silently padded.

Controls in the toolbar#

All three re-run the engine rather than filtering the view, so the counts in the summary strip always describe the grid on screen.

Leading and trailing spaces inside a cell are ignored by default, which is what makes two exports from different tools comparable at all. A column excluded from the comparison keeps its place in the grid — the header is marked, its cells are marked ignored, and any difference it hides is still counted as suppressed rather than dropped.

Whatever ran is named in the notes — "Read as comma-delimited with a header row", "Rows paired on order_id, so row order is ignored", "Leading and trailing spaces in cells were ignored" — and those notes travel into any exported report.

Filtering and finding#

The toolbar has a status filter — All, Added, Removed, Changed, Identical — and the toolbar box is a filter here, as it is for JSON and folders: rows that do not match disappear. It matches the values, including the old value of a changed cell, so a record stays findable by the text it used to have. When nothing survives, the pane says "No rows match this filter" rather than going blank.

Change navigation walks the rows that are not identical, in the filtered order, so ‹ n/m › counts what is actually on screen.

The sample pair#

06-csv/ is a six-row order export. Four rows are edited — two of them in two cells, one in three and one in four — while one row is inserted, one removed, and one only moved.

Compared with the default position pairing it comes back as 2 added, 2 removed, 3 modified with 1 identical row and 7 changed cells — and that is the honest reading of two files whose rows are in different places: the last edited row could not be zipped with anything similar enough, so it reads as a removal plus an addition.

Pick Pair on order_id and the same two files read the way you meant them: every edited row becomes an edited row, and the row that only moved stops counting at all.

twinscope 06-csv/before.csv 06-csv/after.csv

Limits#

The strip also carries this engine's own vocabulary beside the three counts: the before and after row totals, identical rows, changed cells, and columns added or removed. The Diff Radar scores changed cells against the cell count rather than rows against rows, so a one-cell edit in a wide table does not read as a whole changed row.