Designing around a source that publishes on a schedule

When the ECB publishes

The European Central Bank publishes its daily reference rates at around 16:00 CET on working days, and not at all at weekends or on holidays.

That sentence has two completely different kinds of absence in it, and almost every design mistake I made here came from treating them as one. A Saturday has no rate and never will. Today at nine in the morning has no rate yet. Code that can't tell those apart handles one correctly and keeps getting the other wrong.

So now I assume any source that publishes on a schedule gives me two kinds of gap, and I handle them separately.

Where the fallback hid a missing rate

The permanent gap is easy, and I handled it right from the start. A payday with no rate of its own can reach back up to four days for the most recent published one, never forwards. A rate published after I was paid didn't exist when I was paid, so using it would be cheating with hindsight. Four days covers a long bank-holiday weekend. Beyond that the market has really moved, so the right answer is "no rate" and a stale rate doesn't get to stand in for this day's.

That's a good rule, and it's also the one that hides the second kind of gap.

The query asking "which paydays are missing a rate?" was built on top of the lookup that already applies the fallback. So a payday whose own rate just hadn't been published yet read as satisfied, served by the day before, which is a real, valid, recently published rate. The backfill saw nothing missing and returned early, and because the range it requests is derived from what's missing, that date was never asked for again.

Here's what that does to a number. £2,000 converted at 1.351 and not 1.3600 is about eighteen dollars out, permanently, and the sync reported it as a clean success. The fallback worked fine. It just wasn't built for this case.

Asking two questions

My first instinct was to make the fallback cleverer and teach it about publication times, or how recent a substitute is allowed to be. That was the wrong approach. The lookup serves the code that displays figures as well as the code that fetches them, and those want opposite things. Display wants the best available answer now, and fetching wants to know what's still worth asking for.

So the fallback is untouched, and the backfill asks two questions instead of one.

• No usable rate at all. Nothing within reach, so the conversion shows a gap.
• Served by an earlier day, and recent enough that the real one may yet arrive. Converting fine, on a substitute, within ten days of today.

Both lists go into the requested range, but only the first is ever reported as missing. That separation is what makes it work. The second list has to influence what gets fetched without contaminating what gets reported, because a payday converting on a substitute rate is fine for now, it's just not final.

const wanted = [...new Set([
  ...await payslipsMissingRates(db),
  ...await payslipsOnProvisionalRates(db, today),
])].sort()

The ten-day window is what stops the second list becoming permanent. In my entire history there's one payday the bank will never publish a rate for (Boxing Day), correctly served by the 24th's rate, on both devices, for ever. Without a window it would rejoin the request list every two minutes until the end of time, asking for something that doesn't exist. With it, it stays provisional for ten days, ages out, and settles as the answer.

Ten days isn't a clever number. It's just longer than any plausible publication delay and shorter than "always", and it's basically me stating how long I'm willing to keep hoping.

How often to ask again

Once a date can be re-requested, the cost of re-requesting it has to be part of the design from the start.

The sync loop runs every couple of minutes. A single payday waiting on its own rate stays provisional for up to ten days, which at that interval comes to around seven thousand requests to a free, keyless public endpoint, for one payslip. And a pay_date in the future, which nothing in the app validates against, can never be satisfied at all, so one mistyped payslip is roughly seven hundred requests a day, indefinitely. That's a lot to ask of something I'm getting for free, so I don't want to hammer it.

So the range carries a thirty-minute cooldown. That's well under the roughly twenty-four hours a rate takes to appear, so nothing that was going to arrive sooner gets delayed. The attempt is recorded before the request goes out, so a range that fails is throttled the same as one that succeeds, because an endpoint that's down is the case where hammering it helps least.

The clock can go backwards

The first version of that cooldown was written the way everyone writes it.

if (now - at < COOLDOWN_MS) return  // throttled

That expression is true for every negative number, and now - at goes negative any time the clock moves backwards, whether that's an NTP correction, a manual change or a laptop resuming with a stale time. One of those and the range is wedged shut until the application restarts, with no error anywhere, and the payslip stays stuck on a substitute rate.

Only a forward-moving clock should be allowed to throttle.

const since = now - lastAttempt.at
if (since >= 0 && since < BACKFILL_COOLDOWN_MS) return

Now a backwards clock expires the cooldown and doesn't extend it, which I think is the right way round to fail. The cost is one extra request, against a worst case of never asking again. The same reasoning covers a restart clearing the throttle. One extra request on launch is a better trade than persisting a throttle into the database and synchronising it between devices.

There's one known cost, and it's written down beside the code. The range spans the oldest wanted date to the newest, so a payslip recorded a moment ago only changes it if it falls outside that span. Otherwise it waits out the cooldown like any repeat. It's bounded at half an hour and I've accepted it.