aquacom-gateway-suite/docs/exec-plans/active/001-implementation-roadmap.md

ExecPlan 001 — Implementation Roadmap: AquaCom Integration Gateway

Document type: ExecPlan (umbrella roadmap) Status: Active Created: 2026-10-06 Scope: Phases 0–14 (implementation specification §60) through the initial production release Authoritative references: ARCHITECTURE.md · docs/product/software-specification.md · docs/product/codex-implementation-specification.md · docs/configuration/configuration-reference.md · docs/protocol/ (gap registry, operations registry, test vectors)

This is the umbrella plan: it sequences the whole implementation and records cross-phase decisions. Substantial individual features may spawn focused child ExecPlans in this directory (implementation specification §59); this roadmap remains the index. Per ARCHITECTURE.md §1, role-authoritative documents win over any restatement here.

---

1. Purpose

Provide a single, gated execution path from the current documentation-complete, evidence-prepared repository state to the initial production release defined by software-specification.md §47 / implementation specification §74, such that:

• every implemented AquaCom behavior traces to verified evidence (AGENTS.md, known-operations.md);
• each phase ends in a working, testable repository state (implementation specification §79);
• the one open blocking decision (technology selection) is resolved explicitly and recorded before code begins;
• protocol gaps are consumed at the phase that needs them, with documented workarounds where possible.

2. Current state (2026-10-06)

Exists and verified:

• Complete, consistent documentation set: short-form AGENTS.md entry point; ARCHITECTURE.md with document precedence (§1) and invariants INV-001…INV-016; software-specification.md with requirement IDs AQG-001…AQG-030 and RFC-2119 keywords; agent-neutral implementation specification; configuration-reference.md.
• Protocol evidence: authoritative source (AquaCom – RTU level, Rev 1.6, ITT Flygt AB, 2004) as PDF + markdown conversion in docs/protocol/.
• docs/protocol/protocol-gaps.md — 10 entries (1 VERIFY, 9 OPEN), each with working hypotheses where applicable.
• docs/protocol/known-operations.md — OPS-001…OPS-012 registered with evidence sections, scope status, and an evidence-complete implementation order.
• docs/protocol/test-vectors.md — 35 source checksum vectors verified by recomputation against the §2.4 algorithm (including the §3.6.4 Wait-byte pair), framing/session vectors, 4 synthetic wraparound vectors, WGS/WGH/WGD decode scenarios, simulator scenario requirements.

Does not exist yet: src/, tests/, simulator/, schemas/, config/, CI, README.md, CHANGELOG.md, any implementation.

Open: technology selection (decision D-001 below) — the only blocking item.

3. Desired state

The initial production scope (implementation specification §74): verified live and historical AquaCom acquisition for multiple stations; canonical data model; live cache; historical pipeline with deduplication and durable spool; ≥1 queryable history repository; ≥2 independently configurable storage destinations; OPC UA Data Access + Historical Access with stable NodeIds and security on by default; diagnostics and structured logging; simulator; automated tests including restart recovery; configuration validation. Explicitly excluded: process control, setpoint writes, OPC UA HistoryUpdate, HA clustering (implementation specification §75).

4. Constraints

Hard rules that bind every phase (traceability in parentheses):

1. Never invent AquaCom protocol behavior; evidence classes only (AGENTS.md; implementation specification §3.1).
2. AquaCom logic only in src/aquacom; OPC UA and storage never parse telegrams (INV-001/002/003).
3. Historical samples never update current state (INV-004); SourceTimestamp preserved (INV-005).
4. OPC UA HistoryRead queries a repository, never the RTU (INV-006).
5. Storage failure never stops live acquisition (INV-007).
6. Stable StationId/PointId/NodeIds across restart, upgrade, rename, transport change (INV-008/009).
7. Watermark advances on durable spool commit (INV-014); dedup correct across restart (INV-015).
8. Out-of-order historical arrival supported (INV-013); missing-timestamp records quarantined (INV-016).
9. Live acquisition priority over backfill (AQG-028); every queue bounded (ARCHITECTURE §51).
10. Secure defaults; no secrets in the repo (AGENTS.md Security).

5. Open decisions

ID	Decision	Blocks	Recommendation / resolution path
D-001	Implementation language + OPC UA server stack	Phase 1 (all code)	The hard constraint is server-side Historical Access (Part 11) — far less commonly supported than Data Access. Run a HistoryRead smoke test per candidate before committing. Candidates in priority order: C#/.NET (OPC Foundation .NET Standard stack — reference HA implementation), Java/Milo (HA supported), Python/asyncua (fast development; verify HA server completeness first).
D-002	Configuration format	Phase 0 (schema skeleton)	YAML — all examples in configuration-reference.md are already YAML. Low risk.
D-003	First history repository technology	Phase 8	Behind the HistoryRepository interface (ARCHITECTURE §25), so replaceable. Embedded (e.g. SQLite) minimizes external dependencies for the first release; external (PostgreSQL/TimescaleDB) remains a valid Phase 12+ sink. Decide at Phase 8 entry.
D-004	Second storage sink technology	Phase 12	NDJSON file archive is the lowest-risk second destination (config examples already use it). Decide at Phase 12 entry.
D-005	CI platform	Phase 0	Whatever the repository host provides; must run format/lint/static/unit/integration + schema validation without network access (implementation specification §57).
D-006	Configuration schema technology	Phase 0	JSON Schema if D-002 = YAML (standard YAML-schema validation path).

Pre-implementation checklist (project owner, in order):

1. Resolve D-001. I can build and run the HA smoke test for each candidate stack on request — a ~50-line HistoryRead server+client per candidate settles it empirically.
2. PDF spot-checks (~30 min, recommended before Phase 3/6; non-blocking for Phases 0–2): confirm AQ-GAP-001(a) MCC type table, (b) WGS answer mp1–mp128 byte ranges, (e) whether §3.7 truly ends mid-telegram; AQ-GAP-002 whether the PDF defines alp more explicitly; AQ-GAP-006 whether the MAT request really prints 557.
3. Confirm deployment target: OS/host, and whether field contact is TCP via terminal server (serial-over-IP) or direct serial — determines which Phase 2 transports to implement first.

What does not block starting: real RTU hardware or captures (the simulator drives Phases 0–13; every gap that would need a live capture has a documented workaround), aquacom.md consolidation, operations documentation (Phase 14).

6. Implementation sequence

6.1 Phase/gate overview

Phase	Scope (§60)	Gate — exit criteria	Key gap dependency
0	Repository skeleton, CI, README/CHANGELOG, config schema skeleton	G0: CI green on placeholder test; schema validates the §72 minimal config; tree matches implementation specification §5	—
1	Canonical core: StationId, PointId, CurrentValue, HistoricalSample, DataQuality, AcquisitionSource, config domain	G1: domain unit tests + serialization round-trips pass; identifier-stability rule enforced (rename ≠ new PointId); core has zero aquacom/opcua/storage imports	—
2	Transport abstraction + mock transport; TCP first (serial-over-IP per deployment answer)	G2: connect/send/receive/timeout/reconnect tests pass; supports shared-medium arbitration (software-specification §5.1)	—
3	Codec framework: framing, checksum, Wait-byte stripping, validation, fixture framework	G3: 100 % of approved vectors green (AQ-CK-001…035, S1–S4, framing vectors AQ-FR-001…005); invalid telegrams rejected with diagnostics	AQ-GAP-001 (VERIFY items against PDF)
4	AquaCom simulator	G4: scripted WGS success/timeout/malformed/checksum-error/configurable values; no shared codec code (self-validation ban, §49)	—
5	Station/session manager	G5: multiple simulated stations; one failing station does not affect others; state machine per §18; configurable thresholds	—
6	Live acquisition: WGS decode → point mapping → scaling → CurrentValue	G6: simulator → CurrentValue with correct value, quality, timestamps (Source vs Received distinguished); scaling formula applied; alarm/point split via Nal	AQ-GAP-002 (workaround: decode via Nal, alp diagnostic-only)
7	OPC UA live interface: DA, subscriptions, stable NodeIds, security	G7: OPC UA test client browses/reads/subscribes; NodeIds stable across restart; security None disabled	—
8	Local historical repository (first queryable store)	G8: store / query interval / bounds / restart persistence behind the §25 interface	—
9	Historical pipeline: validation → dedup → durable spool → distribution	G9: duplicate submissions produce no duplicate records (idempotent across restart, INV-015); spool survives restart; out-of-order writes accepted (INV-013)	—
10	AquaCom historical acquisition: WGH retrieval, watermark, backfill state machine, throttling	G10: simulator's missing interval recovered after reconnect; watermark resumes from persistent state; backfill throttled so live polling stays responsive	AQ-GAP-003 (hypothesis: block value count from station CFG — validate against simulator)
11	OPC UA Historical Access: HistoryRead raw values, continuation	G11: HA client reads both originally stored and recovered values; continuation works; INV-006 test: HistoryRead never triggers AquaCom traffic	—
12	Second storage destination (sink abstraction)	G12: one HistoricalSample → repository + sink B; failure of B does not stop A	—
13	Store-and-forward resilience	G13: destination offline → history accumulates → delivered on restore; restart during backlog loses no acknowledged data (§53)	—
14	Production hardening: security, resource limits, performance validation, ops documentation	G14: §77 DoD reviewed item-by-item; §69 performance validation with capacity targets recorded	—
O-1 (optional, after 10)	WGD periodical reports (AQUACOM_REPORT)	report day retrieved, scaled per point, NDA handled as missing-not-failed	AQ-GAP-009 (documented defaults)

6.2 Task mapping (implementation specification §79)

T001→P0 · T002→P1 · T003→P0/P1 (config model + schema validation) · T004→P2 · T005+T006→P3 · T007→P4 · T008→P6 (WGS, first verified live operation) · T009→P5/P6 · T010→P6 (current-state service) · T011→P7 · T012→P8 · T013→P9 · T014→P9/P13 (durable spool) · T015+T016→P10 (WGH + backfill state machine) · T017→P11 · T018→P12 · T019→P13 (full reconnect/backfill acceptance) · T020→P14.

Each task ends in a working repository state. Recommended task granularity: one §79-sized task per working session, each with the Goal / Files / Behavior / Constraints / Tests / Acceptance structure (§61).

6.3 Gap consumption schedule

Gap	Consumed at	Handling
AQ-GAP-001	before Phase 3 table implementations	VERIFY against PDF; 35 approved vectors already cover framing/checksum
AQ-GAP-002	Phase 6	workaround documented (decode via Nal); hypothesis strengthened by all four examples
AQ-GAP-003	Phase 10	hypothesis wired to station CFG; simulator validation; close on first real capture
AQ-GAP-004	text points (none in initial scope)	deferred
AQ-GAP-005	any time PDF text is available	transcribe long examples → recompute → approve as wraparound fixtures (synthetics cover the branch meanwhile)
AQ-GAP-006	Phase 3 fixture import	use 540 once PDF-confirmed; verified answer vector approvable independently
AQ-GAP-007	SCE handling (Phase 3/6)	payload treated as optional diagnostic, never parse-critical
AQ-GAP-008/009/010	dialled lines / WGD / alarms	extension scope; not in initial release

7. Files/components affected

src/{core,aquacom,history,storage,opcua,diagnostics,host}/ per ARCHITECTURE.md §6 and dependency direction §7 · tests/{unit,integration,protocol,opcua,history,acceptance}/ · simulator/ · fixtures/ (imported from docs/protocol/test-vectors.md) · schemas/ · config/examples/ · README.md, CHANGELOG.md · CI definition · documentation updates in docs/ as behavior lands (never after).

8. Data model changes

None yet — the canonical contracts are frozen in ARCHITECTURE.md §13–§14 and implementation specification §10–§13. Any contract change requires: update the role-authoritative document first, update dependents, update tests, record the decision here. The core module must compile without AquaCom/OPC UA/storage imports (enforced in CI from Phase 1).

9. Failure handling

The failure matrix of software-specification.md §35 / ARCHITECTURE.md §55 is normative: station failure isolates; storage failure isolates and buffers; OPC UA failure never stops acquisition; malformed telegrams are rejected with diagnostics; restart loses no acknowledged history. Each phase's tests must cover the failure modes it introduces (implementation specification §56 fault injection); every new failure mode gets a diagnostic (AGENTS.md Task completion).

10. Testing

• Pyramid per §50: deterministic unit/component tests dominate; integration via simulator; acceptance scenario at the top.
• Fixtures: imported only from docs/protocol/test-vectors.md; approved fixtures immutable; new vectors require evidence and registry update (never delete).
• Protocol tests run without external network (§57); simulator must not share codec code (§49).
• Restart-recovery tests at every phase that adds persistent state (§53).
• Bug fixes carry regression tests (AGENTS.md).

11. Migration/compatibility

Storage schemas versioned with explicit migrations; startup never silently destroys incompatible data; destructive migration requires administrative action (implementation specification §65). configVersion gating with explicit migration tooling (§66). OPC UA NodeId stability across releases (§68); configuration schema changes never force namespace changes (configuration-reference.md §71).

12. Progress

Date	Entry
2026-10-05	Documentation set completed and made consistent; protocol evidence acquired (Rev 1.6 source); gap registry (10 entries), operations registry (OPS-001…012), and test-vector catalogue (35 verified vectors) created. Phase 0 documentation deliverables done.
2026-10-06	This roadmap created. Awaiting D-001 (technology selection) before Phase 0 code tasks.
2026-10-06	README.md and CHANGELOG.md created — the §5 repository-structure documentation items for Phase 0 are complete. Remaining Phase 0 deliverables: technology selection (D-001), code skeleton, CI, configuration schema.

Update this log at every phase gate; spawn child ExecPlans for §59-triggering work and link them here.

13. Decisions log

ID	Date	Decision	Reference
DR-001	2026-10-05	Documentation unescaped, de-duplicated; single-source-of-truth precedence defined	ARCHITECTURE.md §1
DR-002	2026-10-05	AGENTS.md reduced to short entry-point form per its own spec §4/§6	AGENTS.md
DR-003	2026-10-05	Watermark advances on durable spool commit, not final delivery	ARCHITECTURE.md §21, INV-014
DR-004	2026-10-05	Duplicate-conflict default REJECT_CONFLICT	software-spec §13
DR-005	2026-10-05	Deduplication correct across restart (persisted state or idempotent storage)	INV-015
DR-006	2026-10-05	Pipeline accepts out-of-order historical arrival; queries order by SourceTimestamp	INV-013
DR-007	2026-10-05	Records without usable SourceTimestamp quarantined with diagnostics	INV-016
DR-008	2026-10-05	Control/setpoint writes OUT-OF-SCOPE for initial release; read-only integration first	known-operations OPS-005/006/009/010
DR-009	2026-10-05	WGS decode via Nal; alp diagnostic-only until AQ-GAP-002 closes	AQ-GAP-002 hypothesis
DR-010	2026-10-05	WGH block value count derived from station CFG	AQ-GAP-003 hypothesis
DR-011	2026-10-06	Umbrella roadmap established; D-001 recorded as the blocking decision	this document

14. Unexpected findings

• Source document contains checksum/typo defects: MAT fetch request prints 557 (recomputes 540; 557 = SLT fetch checksum — copy-paste error), narrative destination mismatches (§2.2.3.1 "plant 89" vs telegram dest 86; §3.4.10.2 fixed-line dest given as plant number) — recorded in AQ-GAP-005/006.
• The PDF→markdown conversion is lossy at specific tables and truncates §3.7; byte-exact recomputation of long examples from the conversion is impossible (AQ-GAP-001/005). The PDF is authoritative.
• The §2.4 checksum algorithm verified exactly against all 35 single-line examples — including the subtle rules that spaces around # are summed and Wait bytes are excluded.

15. Acceptance evidence

Production evaluation requires all of (implementation specification §77), evidenced by:

1. Mandatory end-to-end scenario (§54 T0–T8; software-specification.md §46): recorded as an automated acceptance test (T019); passing run attached to the Phase 14 gate.
2. Protocol traceability audit: every implemented operation maps to an known-operations.md entry with evidence; remaining gaps explicitly listed (§77 items 2, 15) — the gap registry is the evidence.
3. Vector suite green: all approved vectors passing in CI (§77 item 12).
4. Restart/data-integrity tests (§53): no acknowledged history lost across restart, outage, backlog (items 7–9).
5. Security tests (§55): untrusted certs rejected, None disabled in production config, no secrets logged.
6. DoD checklist review: the 15 items of §77 walked item-by-item at G14, each with its artifact.
7. Items 13 (validated against real equipment/traffic) and operational documentation remain post-simulator milestones dependent on field access — explicitly tracked, not silently dropped.

16. Plan maintenance

• Update §12 (Progress) at every phase gate; record new decisions in §13 immediately.
• Spawn a child ExecPlan (same structure, §59) for: any new AquaCom operation, storage/persistence architecture, OPC UA namespace redesign, security changes, cross-module refactors, data migration. Link children from §12.
• Move this plan to docs/exec-plans/completed/ when G14 and §15 are satisfied.
• This plan never overrides the role-authoritative documents (ARCHITECTURE.md §1).