Two implementations, one dataset, on purpose

The arrangement

I keep a small internal platform for my own money and work records. It's a desktop app in TypeScript and a phone app in Kotlin, with one dataset underneath. The rules that turn stored rows into figures I act on (what a month's salary was worth in another currency, which projects roll up under which parents, what a reporting period totals to) exist twice, once in each language.

Having two implementations of the same logic is normally something you apologise for on the way to merging them. I've stopped seeing it that way, and I think it's worth having two when a wrong answer is expensive, the rules are small enough to state as data, and you're willing to write those answers down independently of both implementations.

What a second version catches

Before the port existed, there was only one implementation. A single implementation can't disagree with anything, so whatever it computes is the answer by definition. If its rounding drifts a penny downward on every payslip, then that's just what a payslip is worth. There's nothing to compare it against and nothing to flag it.

Two ports that have to agree behave differently. If one language decides something by default and the other decides it differently, that doesn't vanish into the result. It shows up as the two disagreeing, and then I go and find out why. It means changing things twice, but I trust the numbers a lot more.

The obvious trap is to stop there and let the two implementations check each other. That only catches the places where they differ. Both can be wrong in the same direction, especially when the second was written by reading the first, which mine was. So I don't hold the ports against each other at all. They're both held against a third thing that's neither of them.

The shared test data

packages/replay-vectors is a directory of committed data, written before any Kotlin existed. It holds event logs, each paired with the SHA-256 of the projection it has to produce, plus inputs and expected outputs for the date functions, the currency rules, the report windows and one month-by-month series.

Both suites read those files off disk at run time. The Kotlin test walks up to the repository root and doesn't keep a copy inside the Android app, because a copy would go out of date.

One rule does most of the work. Neither implementation is allowed to generate these files at test time, because a corpus regenerated by the thing under test only proves that the code agrees with itself.

It's worth being clear about what a failure means. If the vectors go red, the corpus is doing its job. Somebody changed the projector, the replay order or the date arithmetic, and every device now derives something different. Dealing with that is a decision, and regenerating isn't the default. Either the change was wrong, or it was right and the corpus gets re-run and committed on purpose. That's also the moment the other suite needs updating, because it's about to go red too.

Where I can state an expectation by hand, I do. Projector hashes come from the reference implementation, because there's no other way to write down what a nine-hundred-line replay produces. Date arithmetic is small enough to state independently.

{
  "function": "addMonths",
  "cases": [
    { "args": ["2026-01-31", 1], "expected": "2026-02-28", "why": "the decision: clamp, not roll" },
    { "args": ["2026-02-30", 1], "throws": true }
  ]
}

What addMonths does to the 31st of a month with thirty days is a choice someone made, and so is what rounding does to a .5 tie. Both are written out as literals, and the generator stops dead if the code disagrees with them.

Why I used rows from my own data

Every name, amount and date in that directory is invented, with one exception. The month-by-month series is generated from a copy of my live database. It carries 755 input rows and the expected output for all five reporting periods against a fixed today.

The reason is the data's shape. A fixture of invented numbers proves that two implementations agree about invented numbers, and the cases that really matter aren't ones anybody would think to invent. In my data, 44 of 67 projects roll up under parents, a soft-deleted retainer is still holding real payments, there are 27 payslips against 559 published rates, and there's a Boxing Day payday for a date the central bank will never publish a rate for.

Held against that, the two implementations agree to the minor unit across all 28 months. That's what the arrangement pays back.

What belongs in a fixture

The generator names the columns it takes one by one, instead of taking the whole table. The rule behind that is the thing from this note I'd most expect to reuse elsewhere.

I don't ask "is it in the table". A column goes in the fixture if the answer to "does the arithmetic read it" is yes.

The columns that pass are the ones a rule really reads, so amounts, dates, currencies, parent links and lifecycle. A few survive for a structural reason instead of an arithmetic one. created_at, updated_at, platform and method are only carried because the schemas declare them NOT NULL and the Kotlin test builds full entities from the rows. Nothing computes with them.

Everything else is left behind, and leaving it behind changed no expectation in either suite, which proves nothing was reading it. The column list also doubles as documentation of what the reports layer depends on, and a new column has to argue its way in. I think that's a much better default than a SELECT * that widens every time the schema does without anyone noticing.

Making the phone tests re-run

The last piece is boring, but a lot rests on it. The Kotlin suite reads the shared directory off disk at run time, and nothing in that path was a declared input of the Android test task. The series file is regenerated whenever the desktop implementation changes, and that's the normal way the file moves. But a commit that only changed the corpus left the task up to date, and the vectors moved without the phone ever being re-checked. Declaring the directory as an input means a content change re-runs the suite on its own.

I checked it in three runs. It runs, a second run is up to date, and a one-penny edit to a committed figure re-runs it and fails. So I know it really does fail when it should.