The Engine Roadmap: Where We Are and What Comes Next

Three weeks ago the scheduling core got a born-clean schema and a working wizard. The engine can build a blueprint, fan it out into date-specific instances, and lock those instances as historical truth when the day is done. That is real and running. But the operational half — the half where people cancel a participant for one Tuesday, pull a bus mid-week, override a billing line, and then ask "why is Nathan not on Saturday?" — is still a design document. This post maps the territory between where the code ends today and where the design says it needs to go.

What is live right now

Component	Status	What it does
core_engine schema (19 tables)	Migrated to DB2	Programs, instances, intents, cards, billables, routes, artifacts, tags
Program Creation Wizard	Finished	Six-step blueprint builder with finalize → syncRethread → billing generation
syncRethread	Partially complete	Generates instances, attendance, shifts, vehicle dispatch. Missing: routes placeholder, full lock snapshots, manual_lock_at gaps
Instance API (/api/v1/engine/*)	Built (520 lines)	Window, instances CRUD, attendance, staff, vehicles, routes, billables, lock endpoints
Schedule page (page_schedule)	Built (Phase B)	Fortnight grid, instance detail drawer, lock button
Intentions CRUD	Built	GET/POST/PATCH/DELETE on core_engine.intents; /process endpoint bulk-marks as processed
Billing generation	Exists	util_generateBilling.js creates instance_billables rows

The engine can create a program and project it into the active window. It can lock an instance so the past doesn't change when the blueprint does. What it cannot yet do is modify a projected instance through a deliberate, traceable, reversible-by-forward-action change — and that is the entire point of the next phase.

The eight-phase plan

The master plan lives at admin/tasks/tasks_active/engine-build-out/ with a phase file for each letter and a 403-line mental-model document called HOW_IT_SHOULD_WORK.md that synthesises Brett's own words from the recovered session. The phases and their dependency chain:

A (close gaps) ──┬── B (schedule UI)      [reads what A produces]
                 ├── C (intent engine)     [extends syncRethread that A fixes]
                 └── D (addresses)         [lock snapshot reads addresses]

D ──→ E (roster / transport / workspace)   [routes resolve pickup/dropoff]

A ──→ E (roster / workspace)              [shifts/attendance need to exist]
A ──→ G (billing closure)                 [billables need full denorm at lock]

F (activities), H (governance) — no hard deps; any time after A

Phase A — Close the engine gaps (partially done)

syncRethread generates most instance children now, and the lock endpoint snapshots some values. But a few gaps remain before any downstream UI can be trusted:

• Routes placeholder — instance_routes rows aren't seeded at fan-out time
• Full lock denormalisation — staff names, participant pickup/dropoff addresses from participant_addresses, vehicle names + rego, billing code text
• manual_lock_at — the mechanism that protects a manual assignment (e.g. "Tom was dragged onto Wednesday's shift") from being wiped on re-finalise

Phase B — Active window UI (done)

The schedule page exists. It reads from the engine API and renders a fortnight grid with an instance detail drawer. Lock buttons work. This phase shipped with the schema migration.

Phase C — The intent engine and event sourcing (designed, not built)

This is the keystone. Currently intentions.js is CRUD-only: you can store rows that say "cancel participant X on date Y," but nothing reads those rows and mutates the corresponding instance. The calendar is decorative.

The Phase C design package lives in phase_c_detail/ across six documents. The architecture is an event-sourcing model with four per-domain event tables:

Table	Scope	Write rate
core_engine.program_events	Blueprint changes	Slow
core_engine.intent_events	Forward-dated operator wishes	Medium
core_engine.instance_events	Live operational mutations	Fast
core_engine.billing_events	Financial deltas	Medium, retained forever

Every mutation becomes a row in an event table. No silent UPDATEs. No undo button — only forward verbs. If you cancel Joan and then realise you cancelled the wrong day, you write an add_attendance event. The history reads "cancelled at 14:00, re-added at 14:03." Both are preserved. Both are replayable.

The verb catalog covers four domains and 28 verbs so far, from create_program and cancel_attendance to pull_vehicle and retain_billing_short_notice. Each verb has a defined payload, a target projection table, an idempotency rule, and a V-Pol (validation policy) family.

The design also wires into the Brainframe Tag Trinity — the existing brainframe schema that tracks origin (O-Tag), reason (R-Tag), and authority (V-Pol) for every decision. Every event row carries a reason_tag column. Asking "why is Nathan not on Saturday?" walks back through intent_events → reason_log → origin_registry → the inbound SMS from Nathan's dad in one query.

None of this has been migrated. The four event tables are fully designed (DDL written, naming conflicts checked, indexes specified) but the SQL file has not been cut. No backend code references program_events, intent_events, instance_events, or billing_events yet. Eight open questions (Q1–Q8) have proposed defaults and await Brett's confirmation before DDL is run.

Phase D — Participant addresses (schema exists, integration pending)

core_source.participant_addresses has primary/mailing/pickup/dropoff flags. The lock endpoint still reads legacy single columns. Transport routes can't resolve real pickups until this integration lands.

Phase E — Card-consuming pages (roster, transport, workspace)

The roster page exists but doesn't consume instance_shifts from core_engine. Transport and the day-of workspace are new pages. All three depend on full fan-out (Phase A) and address resolution (Phase D for transport).

Phase F — Activities and program list management

Independent of E. Can ship any time after A. Mostly CRUD work on program_activities and page_prog_settings.

Phase G — Billing closure (invoice gen + NDIS bulk upload)

instance_billables is the source of truth. finance.js already splits agency-managed (bulk CSV) from plan-managed (invoice PDF). billing-history.js knows the NDIS PRODA format. The pipeline just needs closing end-to-end: billable → CSV/PDF → PRODA ingest → reconciliation → status = 'paid'.

Phase H — Governance (settings rename + audit history)

Polish. Renames page_loom-settings → page_org_settings. Adds core_engine.program_audit_log so admins can scrub through time and see the blueprint as it was on a given past date. Locked instances are unaffected because they carry snapshots.

The critical path

A → C → everything else

Phase C is the bottleneck. Without applyIntents(), the intent calendar is a write-only store. Without event tables, there is no audit trail, no "Why" modal, no time-travel slider, no Trinity handshake. The design is complete — 251 lines of verb catalog, 471 lines of schema DDL, 258 lines of UX spec, 221 lines of migration plan. What remains is confirmation on eight open questions, cutting the migration SQL, and writing the three new backend modules: util_eventCommit.js, util_applyIntents.js, and the verb handler map.

The snapshot principle (why this matters)

The single most important idea in the engine is not the event sourcing or the Trinity integration. It is this:

A foreign key is not history.

If a locked billable line still references the live finance.billing_rates row, then changing a rate changes the past. The engine prevents this by denormalising every value that mattered at lock time — participant names, pickup addresses, staff names, vehicle registrations, billing codes, unit prices, total amounts — onto the instance and its children. Future template edits, rate changes, and participant address moves cannot rewrite a locked day.

This principle is enforced in code today (the lock endpoint snapshots several fields) and will be fully enforced when Phase A closes the remaining gaps and Phase C makes every mutation traceable to its origin.

Quick reference

Phase	One-line summary	Depends on	Status
A	Close syncRethread + lock gaps	—	Partially done
B	Schedule fortnight + instance detail	A	Done
C	Intent engine + event sourcing + Trinity	A	Designed, not built
D	Participant addresses integration	A	Schema exists, integration pending
E	Roster, transport, workspace pages	A + D	Not started
F	Activities + program list management	A	Not started
G	Billing closure (invoice + NDIS bulk)	A	Not started
H	Governance (settings + audit history)	A	Not started