From c33eed342eaf4c2b178017dfa3d1f27a2474e9ab Mon Sep 17 00:00:00 2001 From: Matt Spinola Date: Sun, 9 Aug 2026 09:57:24 -0400 Subject: [PATCH] Two sentences the step-2 rewrite left pointing at the old gate #105 moved this doc to marketdata-update and corrected the --require-final paragraph itself, but two supporting sentences elsewhere in the file still describe the wall-clock mechanism that was replaced. The restart-on-failure paragraph told the reader each retry is a last_database_update_time check. That is the dead mechanism, named in the one paragraph whose whole job is explaining why the retry loop is cheap enough to run every ten minutes. It is now what actually happens: a short trailing-window read and a date compare. The testing section said a daytime run only proves the wrapper resolves. That was true of a clock gate, which could not be satisfied before evening no matter what the store held. Under a date comparison it is conditional, and the condition is worth stating because it is an operational property rather than a detail: a daytime run defers only if last night captured. If last night failed or never fired, Norgate is ahead of the store and the daytime run captures immediately, so a missed night self-heals at the next trigger instead of waiting another evening. The old wording told an operator the opposite of that. Docs only, no code touched. Co-Authored-By: Claude Opus 5 --- docs/WINDOWS_SCHEDULING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/WINDOWS_SCHEDULING.md b/docs/WINDOWS_SCHEDULING.md index ff83ead..1342f1a 100644 --- a/docs/WINDOWS_SCHEDULING.md +++ b/docs/WINDOWS_SCHEDULING.md @@ -109,7 +109,7 @@ Register-ScheduledTask -TaskName "cotdata COT (Fri release)" -Action $act -Trigg **Event-driven bars with `--require-final`.** The producer reads two Norgate databases: **Continuous Futures** (the `&ES` / `_CCB` series) and **Futures** (the individual `ES-2026H` contracts used to reconstruct volume). Their **Final** prices land ~8:40pm ET (Futures) and ~8:55pm ET (Continuous Futures), but your Norgate Data Updater still has to *pull* them on its next poll. Rather than guess a fixed time, `--require-final` asks a **data** question: does Norgate hold a NEWER settled bar than the store already does, for a quorum of liquid reference symbols? That needs no wall-clock cutoff and no trading calendar, so it is immune to Norgate's publish-time drift — an early publish is caught early and a late one simply defers. (It replaced a fixed `--final-cutoff`, which broke in production on 2026-07-27 when Norgate finalized one database at 8:49pm and the check demanded 8:55pm for both. See [design/finals_ready_data_driven.md](design/finals_ready_data_driven.md).) Until it is ready the run **defers with a non-zero exit** having fetched nothing, so the restart setting below turns "fire at 8:55pm" into "run the moment NDU has the Finals." -**Retry / wait via restart-on-failure.** Give each task a *restart on failure* — it does double duty: it retries transient fetch errors, and (for the price task) waits out the gap between 8:55pm and NDU actually pulling the Finals (each retry is a cheap `last_database_update_time` check that exits immediately until ready). On a genuine no-session day the retries simply exhaust, harmlessly. `schtasks` can't set this, so use PowerShell (applies to all three tasks): +**Retry / wait via restart-on-failure.** Give each task a *restart on failure* — it does double duty: it retries transient fetch errors, and (for the price task) waits out the gap between 8:55pm and NDU actually pulling the Finals (each retry is a short trailing-window read and a date compare, exiting immediately until ready). On a genuine no-session day the retries simply exhaust, harmlessly. `schtasks` can't set this, so use PowerShell (applies to all three tasks): ```powershell $s = New-ScheduledTaskSettingsSet -RestartInterval (New-TimeSpan -Minutes 10) -RestartCount 6 @@ -156,7 +156,7 @@ Confirm the relevant `newest data` date advanced (and `last write (UTC)` is rece ### Testing the timing and conditions -- **A daytime bars run only proves the wrapper resolves** — with no Finals yet it just defers. To exercise the actual write path in daylight, run `marketdata-update --bars --domain futures` by hand (no `--require-final`), or fire the task after ~8:55pm ET. +- **A daytime bars run usually only proves the wrapper resolves** — if last night captured, the store already holds the newest settled bar, so a daytime run finds nothing newer and defers. The exception is worth knowing: if last night's run *failed or never fired*, a daytime run finds Norgate ahead of the store and captures immediately, so a missed night self-heals at the next trigger rather than waiting for the evening. To exercise the write path on demand regardless, run `marketdata-update --bars --domain futures` by hand (no `--require-final`), or fire the task after ~8:55pm ET. - **Test the trigger itself** by moving it a couple of minutes out, watching it fire, then setting it back: ```bat schtasks /Change /TN "marketdata bars" /ST 14:20