From 08d44babbbd60d4772fdc70471539810cb419005 Mon Sep 17 00:00:00 2001 From: Jonas Bengtsson Date: Mon, 5 Oct 2026 21:27:47 +0200 Subject: [PATCH] =?UTF-8?q?=F0=9F=93=9D=20Update=20documents?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- AGENTS.md | 2053 +--- ARCHITECTURE.md | 3606 +++---- docs/configuration/configuration-reference.md | 3833 ++------ .../codex-implementation-specification.md | 8402 ++--------------- docs/product/software-specification.md | 2197 ++--- missing_docs.md | 76 + 6 files changed, 3718 insertions(+), 16449 deletions(-) create mode 100644 missing_docs.md diff --git a/AGENTS.md b/AGENTS.md index cb13302..5ae8cb7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,1682 +1,21 @@ -\# AquaCom Integration Gateway – Codex Instructions +# AquaCom Integration Gateway +This repository implements the **AquaCom Integration Gateway**: it communicates with AquaCom-compatible field equipment, normalizes live and historical process data, exposes data through OPC UA (Data Access and Historical Access), and delivers history to one or more configurable storage destinations. +Read the authoritative documentation before changing implementation behavior: -This repository implements the \*\*AquaCom Integration Gateway\*\*. - - - -The gateway communicates with AquaCom-compatible field equipment, normalizes live and historical process data, exposes data through OPC UA, and supports one or more historical storage destinations. - - - -Before making architectural or behavioral changes, read: - - - -\- `ARCHITECTURE.md` - -\- `docs/product/software-specification.md` - -\- `docs/architecture/` - -\- `docs/protocol/` - -\- `docs/configuration/configuration-reference.md` - - - -For substantial work, also read or create the applicable ExecPlan under: - - - -```text - -docs/exec-plans/ - -``` - - - -\--- - - - -\## 1. Primary architecture - - - -The intended architecture is: - - - -```text - -AquaCom - - │ - - ▼ - -Transport - - │ - - ▼ - -AquaCom Protocol Engine - - │ - - ▼ - -Station / Session Manager - - │ - - ▼ - -Canonical Data Model - - │ - - ├──────────────► Current-State Service - - │ │ - - │ ▼ - - │ OPC UA - - │ Data Access - - │ - - └──────────────► Historical Pipeline - - │ - - ┌──────────┼──────────┐ - - │ │ │ - - ▼ ▼ ▼ - - Repository Sink A Sink B - - │ - - ▼ - - OPC UA - - Historical Access - -``` - - - -Maintain these boundaries. - - - -\--- - - - -\## 2. Non-negotiable architecture rules - - - -The following rules are mandatory. - - - -\### 2.1 AquaCom isolation - - - -All AquaCom-specific parsing, encoding, framing, checksum handling and protocol behavior belongs in the AquaCom modules. - - - -No OPC UA, history repository or storage adapter shall parse AquaCom telegrams. - - - -\### 2.2 OPC UA isolation - - - -The OPC UA implementation shall consume canonical gateway data. - - - -The OPC UA implementation shall not: - - - -\- construct AquaCom requests; - -\- decode AquaCom responses; - -\- perform direct AquaCom history retrieval; - -\- depend upon AquaCom telegram structures. - - - -\### 2.3 Storage isolation - - - -Storage implementations shall consume canonical historical records. - - - -Storage adapters shall not depend upon: - - - -\- AquaCom telegrams; - -\- AquaCom addressing; - -\- AquaCom command structures; - -\- OPC UA node implementations. - - - -\### 2.4 Canonical model - - - -Data shall cross subsystem boundaries through the canonical gateway model. - - - -Do not bypass the canonical model for convenience. - - - -\### 2.5 Live and historical data are different - - - -Historical samples shall never be replayed as current values merely because they were recently received. - - - -For example: - - - -```text - -Gateway receives at: 18:30 - -Historical sample timestamp: 14:05 - -``` - - - -The value belongs to history at `14:05`. - - - -It is not a new live value at `18:30`. - - - -\### 2.6 Preserve timestamps - - - -Keep at least the following concepts separate: - - - -```text - -SourceTimestamp - -ReceivedTimestamp - -``` - - - -Where applicable also maintain: - - - -```text - -ServerTimestamp - -UpdatedTimestamp - -``` - - - -Never silently replace a valid AquaCom source timestamp with gateway receipt time. - - - -\### 2.7 Stable identifiers - - - -`StationId`, `PointId` and OPC UA NodeIds must remain stable across: - - - -\- process restart; - -\- gateway restart; - -\- software upgrade; - -\- display-name changes; - -\- transport-address changes. - - - -Display names are not stable identifiers. - - - -\### 2.8 Live acquisition priority - - - -Normal live acquisition shall take priority over bulk historical backfill where both compete for limited AquaCom communication capacity. - - - -\--- - - - -\## 3. Critical AquaCom rule - - - -\*\*Never invent AquaCom protocol behavior.\*\* - - - -Do not infer undocumented protocol fields merely because an implementation appears plausible. - - - -Do not invent: - - - -\- telegram fields; - -\- byte positions; - -\- ASCII fields; - -\- framing rules; - -\- checksum algorithms; - -\- addressing behavior; - -\- command codes; - -\- response codes; - -\- value scaling; - -\- timestamp encoding; - -\- alarm semantics; - -\- historical-data semantics. - - - -AquaCom behavior may only be implemented when supported by evidence. - - - -Acceptable evidence includes: - - - -1\. authoritative AquaCom documentation; - -2\. validated traffic captures; - -3\. validated behavior from an existing known-good implementation; - -4\. approved protocol test vectors. - - - -If required protocol information is unavailable: - - - -1\. do not guess; - -2\. leave the feature unimplemented; - -3\. add an entry to: - - - -```text - -docs/protocol/protocol-gaps.md - -``` - - - -4\. add interfaces, fixtures or simulation support around the missing implementation where useful. - - - -Compilation success is not evidence that an AquaCom implementation is correct. - - - -\--- - - - -\## 4. Protocol traceability - - - -Every implemented AquaCom operation shall be traceable to its evidence. - - - -Document protocol implementation under: - - - -```text - -docs/protocol/ - -``` - - - -Where practical record: - - - -```text - -Operation - -Evidence source - -Specification section - -Capture/test-vector reference - -Supported controller types - -Implementation status - -Associated tests - -Known limitations - -``` - - - -Every implemented telegram type shall have automated test vectors. - - - -\--- - - - -\## 5. Protocol fixtures - - - -Protocol fixtures belong under: - - - -```text - -fixtures/aquacom/ - -``` - - - -Fixtures derived from operational traffic shall be sanitized before being committed. - - - -Do not commit unnecessary: - - - -\- production IP addresses; - -\- phone numbers; - -\- station identifiers; - -\- credentials; - -\- private network information; - -\- other operationally sensitive information. - - - -Reference fixtures used for protocol validation should remain immutable unless there is a documented reason to change them. - - - -\--- - - - -\## 6. Core data model - - - -The core domain shall remain independent of AquaCom, OPC UA and storage technology. - - - -Core concepts include: - - - -```text - -StationId - -PointId - -CurrentValue - -HistoricalSample - -DataQuality - -AcquisitionSource - -CommunicationState - -StationState - -HistorySynchronizationState - -``` - - - -The core layer must not reference an OPC UA library or database-specific API. - - - -\--- - - - -\## 7. CurrentValue requirements - - - -A canonical current value shall conceptually contain: - - - -```text - -StationId - -PointId - -Value - -DataType - -Quality - -SourceTimestamp? - -ReceivedTimestamp - -UpdatedTimestamp - -AcquisitionSource - -``` - - - -A missing source timestamp must remain distinguishable from a gateway-generated timestamp. - - - -\--- - - - -\## 8. HistoricalSample requirements - - - -A canonical historical sample shall conceptually contain: - - - -```text - -StationId - -PointId - -Value - -DataType - -Quality - -SourceTimestamp - -ReceivedTimestamp - -AcquisitionSource - -AcquisitionId? - -Metadata? - -``` - - - -A normal historical process sample requires a meaningful `SourceTimestamp`. - - - -Do not fabricate one solely to make storage easier. - - - -\--- - - - -\## 9. Historical processing - - - -The historical pipeline shall logically follow: - - - -```text - -HistoricalSample - - │ - - ▼ - -Validation - - │ - - ▼ - -Normalization - - │ - - ▼ - -Deduplication - - │ - - ▼ - -Durable Spool - - │ - - ├────► Repository - - ├────► Storage Sink A - - └────► Storage Sink B - -``` - - - -Historical delivery shall be restart-safe. - - - -\--- - - - -\## 10. Deduplication - - - -Default historical identity is conceptually: - - - -```text - -StationId - -\+ - -PointId - -\+ - -SourceTimestamp - -``` - - - -Use a stronger device-provided identity if verified and available. - - - -Duplicate handling must be deterministic. - - - -Do not silently overwrite conflicting historical values. - - - -The preferred default for conflicting values is: - - - -```text - -REJECT\_CONFLICT - -``` - - - -and generate diagnostics. - - - -\--- - - - -\## 11. History synchronization - - - -Historical synchronization state shall survive restart. - - - -Do not advance a synchronization watermark until the implementation has sufficient evidence that the corresponding data interval has been successfully processed. - - - -Historical synchronization shall support interruption and continuation. - - - -Large backfills shall be throttleable. - - - -\--- - - - -\## 12. OPC UA rules - - - -OPC UA is a northbound interface to gateway information. - - - -\### Current data - - - -Map canonical current information approximately as: - - - -```text - -Value - - → OPC UA Value - - - -SourceTimestamp - - → OPC UA SourceTimestamp - - - -Gateway processing timestamp - - → OPC UA ServerTimestamp - - - -DataQuality - - → OPC UA StatusCode - -``` - - - -\### Historical data - - - -OPC UA Historical Access shall query a History Repository abstraction. - - - -Never implement: - - - -```text - -OPC UA HistoryRead - - ↓ - -synchronous AquaCom request - - ↓ - -wait for RTU - -``` - - - -Implement: - - - -```text - -AquaCom history synchronization - - ↓ - -Historical Repository - - - -OPC UA HistoryRead - - ↓ - -Historical Repository - -``` - - - -The client-visible OPC UA history interface must remain decoupled from field-device communication latency. - - - -\### NodeIds - - - -OPC UA NodeIds shall be deterministic and stable. - - - -Changing a point's display name shall not normally change its NodeId. - - - -\### Writes - - - -Process-control writes are outside the initial production scope unless explicitly approved. - - - -OPC UA Historical Update shall be disabled by default. - - - -\--- - - - -\## 13. Storage rules - - - -Distinguish: - - - -```text - -Repository - -``` - - - -from: - - - -```text - -Storage Sink - -``` - - - -A \*\*Repository\*\* supports historical queries. - - - -A \*\*Storage Sink\*\* accepts historical delivery but does not necessarily support queries. - - - -One component may implement both. - - - -Do not require every storage destination to become an OPC UA Historical Access source. - - - -\--- - - - -\## 14. Multiple storage destinations - - - -Historical data may be delivered to multiple destinations. - - - -Each destination shall maintain independent: - - - -\- health; - -\- acknowledgement; - -\- retry state; - -\- diagnostics. - - - -Failure of one storage destination shall not unnecessarily stop other destinations. - - - -A destination may be configured as: - - - -```text - -REQUIRED - -``` - - - -or: - - - -```text - -OPTIONAL - -``` - - - -Required delivery behavior shall be explicit. - - - -\--- - - - -\## 15. Durable spool - - - -Historical data pending storage shall use persistent buffering. - - - -The spool must not be an unbounded in-memory queue. - - - -Spool state shall survive process restart. - - - -Each destination shall acknowledge delivery independently. - - - -Do not delete a record solely because one of several required destinations accepted it. - - - -\--- - - - -\## 16. Configuration - - - -Configuration shall be external, machine-readable and schema-validated. - - - -Validate configuration before activating it. - - - -Invalid configuration shall not silently replace the last known valid configuration. - - - -Configuration shall support concepts including: - - - -```text - -gateway - -stations - -transports - -points - -live acquisition - -historical acquisition - -OPC UA - -history repository - -storage destinations - -security - -logging - -diagnostics - -``` - - - -Do not hard-code station-specific process definitions into protocol code. - - - -\--- - - - -\## 17. Secrets - - - -Never commit production: - - - -\- passwords; - -\- API tokens; - -\- private keys; - -\- OPC UA private keys; - -\- database credentials; - -\- secret connection strings. - - - -Example configuration shall use placeholders. - - - -\--- - - - -\## 18. Security - - - -Production OPC UA configuration shall use secure endpoints by default. - - - -Do not make insecure OPC UA operation the default merely to simplify development. - - - -Keep development-only insecure behavior clearly separated and documented. - - - -Administrative capabilities shall be authorized separately from ordinary data access. - - - -If uncertain between a permissive and restrictive security behavior, prefer the restrictive behavior and document the decision. - - - -\--- - - - -\## 19. Diagnostics - - - -New failure modes shall include meaningful diagnostics where practical. - - - -Diagnostics should allow operators to distinguish: - - - -```text - -transport failure - -protocol failure - -station offline - -invalid response - -history synchronization failure - -storage failure - -spool backlog - -OPC UA failure - -configuration failure - -``` - - - -Avoid generic errors where useful context is available. - - - -\--- - - - -\## 20. Logging - - - -Prefer structured logging. - - - -Include contextual information where available: - - - -```text - -component - -stationId - -pointId - -operation - -correlationId - -error - -``` - - - -Raw AquaCom telegram logging shall be separately configurable. - - - -Do not enable full telegram logging as the normal production default. - - - -\--- - - - -\## 21. Correlation - - - -AquaCom request/response transactions should have correlation identifiers. - - - -Where practical, propagate the correlation identifier into: - - - -\- transport diagnostics; - -\- protocol logs; - -\- station diagnostics; - -\- processing errors. - - - -\--- - - - -\## 22. Simulator - - - -The AquaCom simulator is a first-class project component. - - - -Development and CI shall not require operational field equipment. - - - -The simulator shall eventually support: - - - -\- normal communication; - -\- configurable live values; - -\- historical values; - -\- delayed response; - -\- timeout; - -\- disconnect; - -\- malformed telegram; - -\- invalid checksum; - -\- duplicate history; - -\- conflicting history; - -\- clock anomalies; - -\- reconnect/backfill. - - - -Do not implement the simulator so that it merely calls the same decoder being tested to construct all expected protocol behavior. Avoid self-validating tests. - - - -\--- - - - -\## 23. Testing rules - - - -Every functional change shall have applicable automated tests. - - - -Every bug fix should include a regression test where practical. - - - -Do not weaken or remove tests merely to make a change pass. - - - -Tests shall include appropriate combinations of: - - - -```text - -unit tests - -protocol fixture tests - -integration tests - -restart/recovery tests - -acceptance tests - -``` - - - -Protocol tests must run without external network access. - - - -\--- - - - -\## 24. Mandatory architecture tests - - - -Maintain tests that demonstrate at least: - - - -```text - -AquaCom simulator - - → AquaCom decoder - - - -AquaCom live acquisition - - → Current-State Service - - - -Current-State Service - - → OPC UA current value - - - -AquaCom history - - → Historical Pipeline - - - -Historical Pipeline - - → Repository - - - -Historical Pipeline - - → multiple storage destinations - - - -Repository - - → OPC UA HistoryRead - -``` - - - -\--- - - - -\## 25. Critical historical acceptance behavior - - - -The following behavior must always remain valid: - - - -```text - -1\. Station communicates normally. - - - -2\. Current value is exposed through OPC UA. - - - -3\. Communication is lost. - - - -4\. The device retains historical information. - - - -5\. Communication returns. - - - -6\. Live acquisition resumes. - - - -7\. Missing history is retrieved. - - - -8\. Historical samples retain original timestamps. - - - -9\. Recovered historical samples are persisted. - - - -10\. Recovered historical samples do NOT overwrite - - current state simply because they were just received. - - - -11\. OPC UA HistoryRead returns the recovered values - - at their original timestamps. - - - -12\. Re-retrieving the same historical interval does - - not produce unintended duplicates. - -``` - - - -Any change that breaks this scenario is an architectural regression. - - - -\--- - - - -\## 26. Bounded resources - - - -Do not introduce unbounded: - - - -\- queues; - -\- history buffers; - -\- retry lists; - -\- logs; - -\- response payloads; - -\- continuation state; - -\- caches. - - - -Resource limits and overflow behavior shall be explicit. - - - -\--- - - - -\## 27. Graceful shutdown - - - -Shutdown shall preserve consistency. - - - -Conceptual order: - - - -```text - -stop new administrative work - - ↓ - -stop scheduling new acquisition - - ↓ - -complete or cancel active transactions safely - - ↓ - -persist synchronization state - - ↓ - -flush durable state where possible - - ↓ - -stop OPC UA - - ↓ - -close storage destinations - - ↓ - -terminate - -``` - - - -\--- - - - -\## 28. Restart behavior - - - -Following restart: - - - -```text - -load configuration - - ↓ - -validate persistent state - - ↓ - -recover spool - - ↓ - -restore history watermarks - - ↓ - -resume AquaCom communication - - ↓ - -resume live acquisition - - ↓ - -resume historical synchronization - -``` - - - -Prioritize restoration of live operation before large history recovery jobs. - - - -\--- - - - -\## 29. Documentation requirements - - - -Update documentation when observable behavior changes. - - - -Relevant locations include: - - - -```text - -ARCHITECTURE.md - -docs/architecture/ - -docs/protocol/ - -docs/configuration/ - -docs/operations/ - -``` - - - -Do not leave major architectural decisions only in source-code comments. - - - -\--- - - - -\## 30. ExecPlan requirement - - - -Create or update an ExecPlan for substantial work involving: - - - -\- new AquaCom operations; - -\- historical synchronization; - -\- persistence; - -\- storage architecture; - -\- OPC UA namespace changes; - -\- security; - -\- configuration format changes; - -\- cross-module refactoring; - -\- data migration; - -\- significant recovery behavior. - - - -Store active plans under: +| Document | Role | +|---|---| +| `ARCHITECTURE.md` | System structure, boundaries, interfaces, invariants. Also defines document precedence (§1). | +| `docs/product/software-specification.md` | Functional requirements — what the gateway shall do. | +| `docs/product/codex-implementation-specification.md` | Implementation process: phases, task sizing, testing strategy, completion criteria. | +| `docs/configuration/configuration-reference.md` | Configuration model, validation rules, examples. | +| `docs/architecture/` | Detailed subsystem architecture. | +| `docs/protocol/` | AquaCom protocol knowledge, evidence, test vectors, and the protocol-gap registry. | +When documents overlap, the precedence defined in `ARCHITECTURE.md` §1 applies: software-specification for requirements, ARCHITECTURE for structure, the implementation specification for process, the configuration reference for configuration semantics. This file never overrides them. +For substantial work (new AquaCom operations, history synchronization, persistence, storage architecture, OPC UA namespace changes, security, configuration format changes, cross-module refactoring, data migration, significant recovery behavior), create or update an ExecPlan under: ```text @@ -1684,355 +23,51 @@ docs/exec-plans/active/ ``` +Move completed plans to `docs/exec-plans/completed/`. +--- -Move completed plans to: +## Non-negotiable architecture rules +1. AquaCom protocol parsing, encoding, framing, checksums and addressing belong only in the AquaCom modules. No OPC UA, history or storage code shall parse AquaCom telegrams. +2. The OPC UA implementation consumes canonical gateway data only. It shall never construct AquaCom requests, decode AquaCom responses, or depend on AquaCom telegram structures. +3. Storage implementations consume canonical historical records only. They shall not depend on AquaCom telegrams, addressing, command structures, or OPC UA node internals. +4. Data crosses subsystem boundaries only through the canonical model. +5. Historical samples shall never be replayed as current values merely because they were recently received. A historical value describes the process at its SourceTimestamp, not at gateway receipt time. +6. Preserve `SourceTimestamp` separately from `ReceivedTimestamp` (and `ServerTimestamp`/`UpdatedTimestamp` where applicable). Never silently replace a valid AquaCom source timestamp with gateway receipt time. +7. `StationId`, `PointId` and OPC UA NodeIds must remain stable across restart, upgrade, display-name changes, and transport-address changes. Display names are not stable identifiers. +8. Live acquisition takes priority over bulk historical backfill where both compete for limited AquaCom communication capacity. +## AquaCom protocol evidence -```text +**Never invent AquaCom protocol behavior.** Telegram fields, byte positions, framing rules, checksums, command/response codes, addressing, scaling, timestamp encoding, alarm semantics and historical-data semantics may only be implemented when supported by evidence: -docs/exec-plans/completed/ +1. authoritative AquaCom documentation; +2. validated traffic captures; +3. validated behavior from a known-good implementation; +4. approved protocol test vectors. -``` +If required protocol information is unavailable: do not guess, leave the feature unimplemented, record the gap in `docs/protocol/protocol-gaps.md`, and build the interfaces, fixtures and simulation support around the missing area. Compilation success is not evidence that an AquaCom implementation is correct. +Every implemented AquaCom operation shall be traceable to its evidence, and every implemented telegram type shall have automated test vectors. +## Testing -An ExecPlan should contain: +Every functional change shall have applicable automated tests; every bug fix should include a regression test where practical. Do not weaken or remove tests merely to make a change pass. Protocol tests must run without external network access, and the AquaCom simulator shall not share parsing code with the production decoder in ways that make protocol tests self-validating. +## Security +Never commit production secrets, passwords, API tokens, private keys, OPC UA private keys, database credentials, or secret connection strings. Example configuration shall use placeholders. Production OPC UA configuration shall use secure endpoints by default; insecure modes must not become the production default. If uncertain between a permissive and restrictive security behavior, prefer the restrictive behavior and document the decision. -```text +## Handling uncertainty -Purpose +- **Local implementation detail** — choose the simplest solution consistent with the architecture. +- **Architectural decision** — record the decision and reasoning in the current ExecPlan. +- **AquaCom protocol uncertainty** — do not guess; record it in `docs/protocol/protocol-gaps.md`. +- **Security or safety uncertainty** — prefer the safer behavior and document the decision. -Current state +## Task completion -Desired state - -Constraints - -Affected components - -Implementation sequence - -Data-model changes - -Failure handling - -Testing - -Compatibility/migration - -Progress - -Decisions - -Unexpected findings - -Acceptance evidence - -``` - - - -Keep the plan current as implementation progresses. - - - -\--- - - - -\## 31. Handling uncertainty - - - -Classify uncertainties as: - - - -\### A. Local implementation detail - - - -Choose the simplest solution consistent with existing architecture. - - - -\### B. Architectural decision - - - -Record the decision and reasoning in the current ExecPlan. - - - -\### C. AquaCom protocol uncertainty - - - -Do not guess. - - - -Record it in: - - - -```text - -docs/protocol/protocol-gaps.md - -``` - - - -\### D. Security or safety uncertainty - - - -Prefer the safer behavior and document the decision. - - - -\--- - - - -\## 32. Task completion - - - -Do not consider a task complete until applicable conditions are satisfied: - - - -\- implementation is complete; - -\- relevant tests exist; - -\- tests pass; - -\- lint/static checks pass; - -\- configuration validation passes; - -\- relevant documentation is updated; - -\- architecture boundaries remain intact; - -\- new AquaCom behavior has evidence; - -\- new failure modes have diagnostics; - -\- no secrets were introduced. - - - -\--- - - - -\## 33. Initial scope - - - -The initial production goal is primarily read-oriented integration. - - - -Expected scope includes: - - - -```text - -AquaCom live acquisition - -AquaCom historical acquisition - -multiple stations - -canonical data model - -current-state cache - -historical pipeline - -deduplication - -persistent synchronization state - -durable spool - -historical repository - -multiple storage destinations - -OPC UA Data Access - -OPC UA Historical Access - -security - -diagnostics - -simulation - -automated testing - -restart recovery - -``` - - - -Unless explicitly approved, do not implement: - - - -```text - -remote pump control - -setpoint changes - -generic OPC UA process writes - -OPC UA Methods for process control - -OPC UA HistoryUpdate - -``` - - - -\--- - - - -\## 34. Future control functionality - - - -If process control is later introduced, use a separate command architecture: - - - -```text - -OPC UA/API - - │ - - ▼ - -Authorization - - │ - - ▼ - -Command validation - - │ - - ▼ - -Command service - - │ - - ▼ - -AquaCom control operation - - │ - - ▼ - -Confirmation / readback - - │ - - ▼ - -Audit - -``` - - - -Never implement process control as: - - - -```text - -OPC UA write - - ↓ - -raw AquaCom telegram - -``` - - - -without an explicit command-validation and authorization layer. - - - -\--- - - - -\## 35. Guiding implementation principle - - - -Optimize for: - - - -1\. correctness; - -2\. traceability; - -3\. recoverability; - -4\. testability; - -5\. maintainability; - -6\. operational diagnostics. - - - -Do not optimize for implementation speed at the expense of protocol correctness. - - - -When AquaCom knowledge is incomplete, build the interfaces, simulation, data model and tests around the missing area first. - - - -Then implement real protocol behavior only when adequate evidence exists. - - - -The central architectural rule remains: - - - -> A historical value describes the process at its source timestamp, not at the time the gateway happened to receive it. - - - -And the central protocol rule remains: - - - -> Never invent undocumented AquaCom behavior. +Do not consider a task complete until: implementation is complete; relevant tests exist and pass; lint/static checks and configuration validation pass; relevant documentation is updated; architecture boundaries remain intact; new AquaCom behavior has evidence; new failure modes have diagnostics; no secrets were introduced. +A list of documentation referenced by this repository but not yet created is maintained in `missing_docs.md`. \ No newline at end of file diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 4fb2dc5..b678aa1 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,3602 +1,2718 @@ -# \# AquaCom Integration Gateway – Architecture +# AquaCom Integration Gateway – Architecture -# +## 1. Purpose -# \## 1. Purpose +This document defines the high-level software architecture of the **AquaCom Integration Gateway**. -# +It describes: -# This document defines the high-level software architecture of the \*\*AquaCom Integration Gateway\*\*. +- major system components; -# +- dependency direction; -# It describes: +- data flows; -# +- architectural boundaries; -# \- major system components; +- live-data handling; -# \- dependency direction; +- historical-data handling; -# \- data flows; +- OPC UA integration; -# \- architectural boundaries; +- storage integration; -# \- live-data handling; +- diagnostics; -# \- historical-data handling; +- security boundaries; -# \- OPC UA integration; +- extension points. -# \- storage integration; +Detailed functional requirements are defined in: -# \- diagnostics; +```text -# \- security boundaries; +docs/product/software-specification.md -# \- extension points. +``` -# +Implementation requirements and Codex-specific development rules are defined in: -# Detailed functional requirements are defined in: +```text -# +docs/product/codex-implementation-specification.md -# ```text +AGENTS.md -# docs/product/software-specification.md +``` -# ``` +**Document roles and precedence.** When documents overlap or conflict, the following precedence applies: -# +1. `docs/product/software-specification.md` — the authoritative statement of **what** the gateway shall do (requirements). +2. This document — the authoritative statement of **how the system is structured** to meet those requirements (architecture, boundaries, interfaces). +3. `docs/product/codex-implementation-specification.md` — the authoritative statement of **how implementation work proceeds** (phases, task sizing, test strategy, completion criteria). It restates architectural rules for implementer convenience but does not redefine them. +4. `docs/configuration/configuration-reference.md` — the authoritative reference for **configuration semantics** (field names, defaults, validation rules). +5. `AGENTS.md` — the entry-point summary for coding agents; it points to the documents above and never overrides them. -# Implementation requirements and Codex-specific development rules are defined in: +A data contract or interface is defined **once** in the authoritative document for its role; other documents link to it rather than restating it with different names. Where a contract is currently restated (for historical reasons), the definition in the role-authoritative document wins. -# +Detailed subsystem architecture is documented under: -# ```text +```text -# docs/product/codex-implementation-specification.md +docs/architecture/ -# AGENTS.md +``` -# ``` +AquaCom-specific protocol knowledge is documented under: -# +```text -# Detailed subsystem architecture is documented under: +docs/protocol/ -# +``` -# ```text +--- -# docs/architecture/ +# 2. System purpose -# ``` +The AquaCom Integration Gateway provides a modern integration boundary between AquaCom-compatible field equipment and higher-level systems. -# +The gateway shall: -# AquaCom-specific protocol knowledge is documented under: +- communicate with AquaCom field devices; -# +- acquire current process values; -# ```text +- retrieve historical process values; -# docs/protocol/ +- normalize AquaCom-specific information into a protocol-independent model; -# ``` +- expose current values using OPC UA; -# +- expose historical values using OPC UA Historical Access; -# \--- +- store historical data in one or more configurable destinations; -# +- recover historical information following communication interruptions; -# \# 2. System purpose +- provide operational and protocol diagnostics. -# +The gateway shall not make higher-level systems dependent on AquaCom-specific behavior. -# The AquaCom Integration Gateway provides a modern integration boundary between AquaCom-compatible field equipment and higher-level systems. +--- -# +# 3. Architectural principle -# The gateway shall: +The central architecture is: -# +```text -# \- communicate with AquaCom field devices; +AquaCom field device -# \- acquire current process values; + │ -# \- retrieve historical process values; + ▼ -# \- normalize AquaCom-specific information into a protocol-independent model; +AquaCom Transport -# \- expose current values using OPC UA; + │ -# \- expose historical values using OPC UA Historical Access; + ▼ -# \- store historical data in one or more configurable destinations; +AquaCom Protocol Engine -# \- recover historical information following communication interruptions; + │ -# \- provide operational and protocol diagnostics. + ▼ -# +Station / Session Manager -# The gateway shall not make higher-level systems dependent on AquaCom-specific behavior. + │ -# + ▼ -# \--- +Canonical Data Model -# + │ -# \# 3. Architectural principle + ├─────────────────────► Current-State Service -# + │ │ -# The central architecture is: + │ ▼ -# + │ OPC UA Data Access -# ```text + │ -# AquaCom field device + └─────────────────────► Historical Pipeline -# │ + │ -# ▼ + ┌───────────┼────────────┐ -# AquaCom Transport + │ │ │ -# │ + ▼ ▼ ▼ -# ▼ + Repository Storage A Storage B -# AquaCom Protocol Engine + │ -# │ + ▼ -# ▼ + OPC UA Historical Access -# Station / Session Manager +``` -# │ +The canonical data model forms the boundary between AquaCom-specific functionality and all northbound integrations. -# ▼ +--- -# Canonical Data Model +# 4. Architectural goals -# │ +The architecture shall support: -# ├─────────────────────► Current-State Service +- protocol isolation; -# │ │ +- storage independence; -# │ ▼ +- stable internal identifiers; -# │ OPC UA Data Access +- stable OPC UA NodeIds; -# │ +- correct handling of source timestamps; -# └─────────────────────► Historical Pipeline +- reliable history recovery; -# │ +- multiple historical storage destinations; -# ┌───────────┼────────────┐ +- store-and-forward; -# │ │ │ +- independent subsystem testing; -# ▼ ▼ ▼ +- restart recovery; -# Repository Storage A Storage B +- future replacement of individual technologies; -# │ +- future addition of other northbound interfaces. -# ▼ +The architecture shall avoid tight coupling between: -# OPC UA Historical Access +```text -# ``` +AquaCom -# +OPC UA -# The canonical data model forms the boundary between AquaCom-specific functionality and all northbound integrations. +Database technology -# +Historian technology -# \--- +SCADA platform -# +``` -# \# 4. Architectural goals +--- -# +# 5. Architectural boundaries -# The architecture shall support: +The following boundaries are mandatory. -# +## 5.1 AquaCom boundary -# \- protocol isolation; +Only the AquaCom subsystem may: -# \- storage independence; +- encode AquaCom telegrams; -# \- stable internal identifiers; +- decode AquaCom telegrams; -# \- stable OPC UA NodeIds; +- calculate or verify AquaCom checksums; -# \- correct handling of source timestamps; +- interpret AquaCom operation codes; -# \- reliable history recovery; +- interpret AquaCom addressing; -# \- multiple historical storage destinations; +- interpret AquaCom-specific value formats. -# \- store-and-forward; +No OPC UA or storage module shall perform AquaCom protocol processing. -# \- independent subsystem testing; +--- -# \- restart recovery; +## 5.2 Canonical model boundary -# \- future replacement of individual technologies; +AquaCom data shall be converted into canonical gateway objects before being forwarded to: -# \- future addition of other northbound interfaces. +- current-state handling; -# +- history handling; -# The architecture shall avoid tight coupling between: +- OPC UA; -# +- storage systems; -# ```text +- diagnostics where appropriate. -# AquaCom +The canonical model shall remain independent of AquaCom and OPC UA. -# OPC UA +--- -# Database technology +## 5.3 OPC UA boundary -# Historian technology +The OPC UA subsystem shall consume canonical gateway information. -# SCADA platform +It shall not: -# ``` +- communicate directly with AquaCom stations; -# +- decode telegrams; -# \--- +- synchronously request historical data from an RTU; -# +- depend upon a particular storage technology. -# \# 5. Architectural boundaries +--- -# +## 5.4 Storage boundary -# The following boundaries are mandatory. +Storage destinations shall receive canonical historical records. -# +Storage implementations shall not depend upon: -# \## 5.1 AquaCom boundary +- AquaCom telegrams; -# +- AquaCom transport; -# Only the AquaCom subsystem may: +- OPC UA server internals. -# +--- -# \- encode AquaCom telegrams; +# 6. Source tree -# \- decode AquaCom telegrams; +The logical source structure is: -# \- calculate or verify AquaCom checksums; +```text -# \- interpret AquaCom operation codes; +src/ -# \- interpret AquaCom addressing; +├── core/ -# \- interpret AquaCom-specific value formats. +├── aquacom/ -# +├── history/ -# No OPC UA or storage module shall perform AquaCom protocol processing. +├── storage/ -# +├── opcua/ -# \--- +├── diagnostics/ -# +└── host/ -# \## 5.2 Canonical model boundary +``` -# +The exact language-specific package or project structure may differ, but the architectural responsibilities shall remain equivalent. -# AquaCom data shall be converted into canonical gateway objects before being forwarded to: +--- -# +# 7. Dependency direction -# \- current-state handling; +Allowed dependencies shall broadly follow: -# \- history handling; +```text -# \- OPC UA; + host -# \- storage systems; + │ -# \- diagnostics where appropriate. + ┌───────────┼─────────────┐ -# + │ │ │ -# The canonical model shall remain independent of AquaCom and OPC UA. + ▼ ▼ ▼ -# + opcua history diagnostics -# \--- + │ │ -# + │ storage -# \## 5.3 OPC UA boundary + │ │ -# + └──────┬────┘ -# The OPC UA subsystem shall consume canonical gateway information. + ▼ -# + core -# It shall not: + ▲ -# + │ -# \- communicate directly with AquaCom stations; + aquacom -# \- decode telegrams; +``` -# \- synchronously request historical data from an RTU; +The most important rule is: -# \- depend upon a particular storage technology. +```text -# +core -# \--- +``` -# +shall not depend upon: -# \## 5.4 Storage boundary +```text -# +aquacom -# Storage destinations shall receive canonical historical records. +opcua -# +storage technology -# Storage implementations shall not depend upon: +``` -# +--- -# \- AquaCom telegrams; +# 8. Core subsystem -# \- AquaCom transport; +The `core` subsystem contains protocol-independent domain concepts. -# \- OPC UA server internals. +Examples include: -# +```text -# \--- +StationId -# +PointId -# \# 6. Source tree +CurrentValue -# +HistoricalSample -# The logical source structure is: +DataQuality -# +AcquisitionSource -# ```text +CommunicationState -# src/ +StationState -# ├── core/ +HistorySynchronizationState -# ├── aquacom/ +``` -# ├── history/ +The core subsystem defines shared contracts used between other modules. -# ├── storage/ +It shall contain no AquaCom protocol logic. -# ├── opcua/ +It shall contain no OPC UA-specific types. -# ├── diagnostics/ +It shall contain no database-specific types. -# └── host/ +--- -# ``` +# 9. AquaCom subsystem -# +The AquaCom subsystem contains: -# The exact language-specific package or project structure may differ, but the architectural responsibilities shall remain equivalent. +```text -# +Transport -# \--- +Protocol codec -# +Telegram validation -# \# 7. Dependency direction +Station/session handling -# +Live acquisition -# Allowed dependencies shall broadly follow: +Historical acquisition -# +Protocol diagnostics -# ```text +``` -# host +Logical structure: -# │ +```text -# ┌───────────┼─────────────┐ +AquaCom Transport -# │ │ │ + │ -# ▼ ▼ ▼ + ▼ -# opcua history diagnostics +Protocol Codec -# │ │ + │ -# │ storage + ▼ -# │ │ +Protocol Operations -# └──────┬────┘ + │ -# ▼ + ▼ -# core +Station Session -# ▲ + │ -# │ + ├── Live Acquisition -# aquacom + └── Historical Acquisition -# ``` +``` -# +Protocol implementation shall be evidence-based. -# The most important rule is: +Undocumented AquaCom behavior shall not be guessed. -# +--- -# ```text +# 10. AquaCom transport -# core +The transport layer is responsible for communication only. -# ``` +It may support implementations such as: -# +```text -# shall not depend upon: +TCP -# +Serial -# ```text +Serial-over-IP -# aquacom +Other supported AquaCom transport -# opcua +``` -# storage technology +The transport layer shall not interpret process values. -# ``` +Conceptual interface: -# +```text -# \--- +open -# +close -# \# 8. Core subsystem +send -# +receive -# The `core` subsystem contains protocol-independent domain concepts. +health -# +``` -# Examples include: +--- -# +# 11. AquaCom protocol engine -# ```text +The protocol engine is responsible for: -# StationId +- framing; -# PointId +- telegram validation; -# CurrentValue +- checksum handling; -# HistoricalSample +- request encoding; -# DataQuality +- response decoding; -# AcquisitionSource +- protocol error interpretation. -# CommunicationState +Conceptual transformation: -# StationState +```text -# HistorySynchronizationState +raw bytes -# ``` + │ -# + ▼ -# The core subsystem defines shared contracts used between other modules. +AquaCom message -# + │ -# It shall contain no AquaCom protocol logic. + ▼ -# +logical protocol result -# It shall contain no OPC UA-specific types. +``` -# +The codec shall be independently testable without a physical AquaCom device. -# It shall contain no database-specific types. +--- -# +# 12. Station/session manager -# \--- +Each AquaCom station has an independent logical session. -# +Conceptual station states: -# \# 9. AquaCom subsystem +```text -# +DISABLED -# The AquaCom subsystem contains: +INITIALIZING -# +CONNECTING -# ```text +ONLINE -# Transport +DEGRADED -# Protocol codec +OFFLINE -# Telegram validation +RECOVERING -# Station/session handling +``` -# Live acquisition +One failing station shall not stop communication with unrelated stations. -# Historical acquisition +The Station Manager owns: -# Protocol diagnostics +- station lifecycle; -# ``` +- communication status; -# +- retries; -# Logical structure: +- scheduling; -# +- station diagnostics; -# ```text +- history recovery coordination. -# AquaCom Transport +--- -# │ +# 13. Canonical data model -# ▼ +The gateway uses a protocol-independent canonical model. -# Protocol Codec +## 13.1 CurrentValue -# │ +Conceptually: -# ▼ +```text -# Protocol Operations +CurrentValue -# │ + StationId -# ▼ + PointId -# Station Session + Value -# │ + DataType -# ├── Live Acquisition + Quality -# └── Historical Acquisition + SourceTimestamp -# ``` + ReceivedTimestamp -# + UpdatedTimestamp -# Protocol implementation shall be evidence-based. + AcquisitionSource -# +``` -# Undocumented AquaCom behavior shall not be guessed. +--- -# +## 13.2 HistoricalSample -# \--- +Conceptually: -# +```text -# \# 10. AquaCom transport +HistoricalSample -# + StationId -# The transport layer is responsible for communication only. + PointId -# + Value -# It may support implementations such as: + DataType -# + Quality -# ```text + SourceTimestamp -# TCP + ReceivedTimestamp -# Serial + AcquisitionSource -# Serial-over-IP + AcquisitionId -# Other supported AquaCom transport +``` -# ``` +Historical samples shall retain the timestamp associated with the process state. -# +--- -# The transport layer shall not interpret process values. +# 14. Timestamp model -# +The gateway shall distinguish at least: -# Conceptual interface: +```text -# +SourceTimestamp -# ```text +ReceivedTimestamp -# open +``` -# close +Where applicable also: -# send +```text -# receive +UpdatedTimestamp -# health +ServerTimestamp -# ``` +``` -# +Their meanings are: -# \--- +| Timestamp | Meaning | -# +|---|---| -# \# 11. AquaCom protocol engine +| SourceTimestamp | Time associated with the value at the source | -# +| ReceivedTimestamp | Time the gateway received the information | -# The protocol engine is responsible for: +| UpdatedTimestamp | Time current gateway state was changed | -# +| ServerTimestamp | Timestamp used by the OPC UA server | -# \- framing; +The following must never be assumed: -# \- telegram validation; +```text -# \- checksum handling; +ReceivedTimestamp = SourceTimestamp -# \- request encoding; +``` -# \- response decoding; +--- -# \- protocol error interpretation. +# 15. Fundamental historical rule -# +The fundamental historical-data rule is: -# Conceptual transformation: +> A historical sample represents the process at its SourceTimestamp, not at the time the gateway receives the sample. -# +Example: -# ```text +```text -# raw bytes +Historical value: -# │ + SourceTimestamp = 14:05 -# ▼ + ReceivedTimestamp = 18:30 -# AquaCom message +``` -# │ +The value belongs to history at: -# ▼ +```text -# logical protocol result +14:05 -# ``` +``` -# +It shall not become a new current process value at: -# The codec shall be independently testable without a physical AquaCom device. +```text -# +18:30 -# \--- +``` -# +--- -# \# 12. Station/session manager +# 16. Live-data architecture -# +Live AquaCom values follow: -# Each AquaCom station has an independent logical session. +```text -# +AquaCom response -# Conceptual station states: + │ -# + ▼ -# ```text +Protocol decode -# DISABLED + │ -# INITIALIZING + ▼ -# CONNECTING +Point mapping -# ONLINE + │ -# DEGRADED + ▼ -# OFFLINE +Canonical CurrentValue -# RECOVERING + │ -# ``` + ▼ -# +Current-State Service -# One failing station shall not stop communication with unrelated stations. + │ -# + ├────────► OPC UA -# The Station Manager owns: + │ -# + └────────► Historical pipeline -# \- station lifecycle; + if live historization is enabled -# \- communication status; +``` -# \- retries; +The Current-State Service maintains the latest accepted value for each configured point. -# \- scheduling; +--- -# \- station diagnostics; +# 17. Current-State Service -# \- history recovery coordination. +Responsibilities: -# +- maintain the latest current value; -# \--- +- reject obsolete current-state updates where applicable; -# +- expose current values to OPC UA; -# \# 13. Canonical data model +- notify subscribers of valid changes; -# +- preserve quality and timestamps. -# The gateway uses a protocol-independent canonical model. +Conceptual interface: -# +```text -# \## 13.1 CurrentValue +update(CurrentValue) -# +get(PointId) -# Conceptually: +getAll(StationId) -# +subscribe(PointId) -# ```text +``` -# CurrentValue +Historical backfill shall not directly call `update(CurrentValue)`. -# StationId +--- -# PointId +# 18. Historical-data architecture -# Value +Historical AquaCom data follows: -# DataType +```text -# Quality +AquaCom history -# SourceTimestamp + │ -# ReceivedTimestamp + ▼ -# UpdatedTimestamp +Historical Acquisition Manager -# AcquisitionSource + │ -# ``` + ▼ -# +Canonical HistoricalSample -# \--- + │ -# + ▼ -# \## 13.2 HistoricalSample +Historical Pipeline -# + │ -# Conceptually: + ├────────► History Repository -# + ├────────► Storage Sink A -# ```text + ├────────► Storage Sink B -# HistoricalSample + └────────► Other configured destinations -# StationId +``` -# PointId +The historical pipeline is independent from OPC UA. -# Value +--- -# DataType +# 19. Historical Acquisition Manager -# Quality +The Historical Acquisition Manager is responsible for: -# SourceTimestamp +- determining missing intervals; -# ReceivedTimestamp +- requesting AquaCom history; -# AcquisitionSource +- processing historical responses; -# AcquisitionId +- maintaining synchronization state; -# ``` +- managing watermarks; -# +- performing reconnect/backfill; -# Historical samples shall retain the timestamp associated with the process state. +- throttling history retrieval; -# +- avoiding interference with live polling. -# \--- +Live acquisition shall normally have higher communication priority than bulk history recovery. -# +--- -# \# 14. Timestamp model +# 20. Historical synchronization -# +Each station may maintain persistent synchronization state. -# The gateway shall distinguish at least: +Conceptually: -# +```text -# ```text +StationId -# SourceTimestamp +HistoryStreamId -# ReceivedTimestamp +LastCompleteTimestamp -# ``` +LastAttemptTimestamp -# +LastSuccessfulTimestamp -# Where applicable also: +SynchronizationState -# +``` -# ```text +Example: -# UpdatedTimestamp +```text -# ServerTimestamp +12:00 history synchronized -# ``` +12:03 communication lost -# +17:24 communication restored -# Their meanings are: +12:00 → 17:24 -# + identified as recovery interval -# | Timestamp | Meaning | +``` -# |---|---| +The gateway then retrieves available AquaCom history for that interval. -# | SourceTimestamp | Time associated with the value at the source | +--- -# | ReceivedTimestamp | Time the gateway received the information | +# 21. Watermark rule -# | UpdatedTimestamp | Time current gateway state was changed | +A history synchronization watermark shall only advance after the corresponding interval has been successfully handled according to the configured persistence policy. -# | ServerTimestamp | Timestamp used by the OPC UA server | +"Successfully handled" is defined precisely as: **the samples of the interval have been durably spooled**, meaning the records have been committed to persistent spool storage and would be re-delivered after a restart. Delivery to the final storage destinations is tracked independently through the per-destination acknowledgements described in §24 and does not gate the watermark. -# +This distinction matters for restart behavior: if the watermark required final delivery, a gateway restart during a storage outage would force re-retrieval of the entire interval from the AquaCom station (or loss of it, if the station has since overwritten its own history). Advancing the watermark on durable spool, combined with at-least-once idempotent delivery, keeps recovery bounded to the spool rather than the field device. -# The following must never be assumed: +The following is prohibited: -# +```text -# ```text +receive historical packet -# ReceivedTimestamp = SourceTimestamp + │ -# ``` + ▼ -# +advance watermark -# \--- + │ -# + ▼ -# \# 15. Fundamental historical rule +storage fails -# +``` -# The fundamental historical-data rule is: +Preferred behavior is: -# +```text -# > A historical sample represents the process at its SourceTimestamp, not at the time the gateway receives the sample. +receive -# + ↓ -# Example: +validate -# + ↓ -# ```text +persist/durably queue -# Historical value: + ↓ -# SourceTimestamp = 14:05 +advance synchronization state -# ReceivedTimestamp = 18:30 +``` -# ``` +--- -# +# 22. Historical pipeline -# The value belongs to history at: +The historical pipeline shall conceptually perform: -# +```text -# ```text +HistoricalSample -# 14:05 + │ -# ``` + ▼ -# +Validation -# It shall not become a new current process value at: + │ -# + ▼ -# ```text +Normalization -# 18:30 + │ -# ``` + ▼ -# +Deduplication -# \--- + │ -# + ▼ -# \# 16. Live-data architecture +Durable Spool -# + │ -# Live AquaCom values follow: + ▼ -# +Distribution -# ```text +``` -# AquaCom response +This pipeline provides a stable boundary between acquisition and storage. -# │ +**Arrival order is not chronological.** Live historization produces samples at the current time while backfill delivers older intervals afterwards, so the pipeline must accept samples in arbitrary order relative to `SourceTimestamp`. Storage adapters shall therefore support out-of-order writes, and historical queries shall order results by `SourceTimestamp`, never by storage arrival order. -# ▼ +--- -# Protocol decode +# 23. Deduplication -# │ +The default historical logical identity is: -# ▼ +```text -# Point mapping +StationId -# │ ++ -# ▼ +PointId -# Canonical CurrentValue ++ -# │ +SourceTimestamp -# ▼ +``` -# Current-State Service +A stronger AquaCom-provided identifier may be used where verified. -# │ +Identical samples shall not create unintended duplicate history. -# ├────────► OPC UA +Conflicting samples sharing the same logical identity shall generate diagnostics. -# │ +The default conflict strategy should favor explicit conflict detection rather than silent overwrite. -# └────────► Historical pipeline +Deduplication state shall survive gateway restart, at least for the interval of any in-progress synchronization. Otherwise a restart during backfill would cause the already-received portion of the interval to be stored twice. Two implementation options satisfy this: persisting deduplication state alongside the synchronization watermark, or relying on idempotent storage (the at-least-once delivery described in §24) and treating deduplication as an efficiency measure for steady-state operation. The chosen option shall be documented in the relevant ExecPlan. -# if live historization is enabled +The deduplication window shall be bounded; an unbounded identity cache violates §51 (resource boundaries). -# ``` +Deduplication operates on samples in **arrival order**, which during backfill is generally not chronological order (see §22 ordering note). The implementation shall not assume chronological input. -# +--- -# The Current-State Service maintains the latest accepted value for each configured point. +# 24. Durable spool -# +The historical spool provides restart-safe delivery. -# \--- +Conceptually: -# +```text -# \# 17. Current-State Service +HistoricalSample -# + │ -# Responsibilities: + ▼ -# +Persistent spool -# \- maintain the latest current value; + │ -# \- reject obsolete current-state updates where applicable; + ├────► Repository -# \- expose current values to OPC UA; + ├────► Sink A -# \- notify subscribers of valid changes; + └────► Sink B -# \- preserve quality and timestamps. +``` -# +Each destination maintains independent acknowledgement. -# Conceptual interface: +Example: -# +```text -# ```text +Record 4711 -# update(CurrentValue) +Repository ACK -# +CentralHistorian ACK -# get(PointId) +Archive PENDING -# +``` -# getAll(StationId) +The record remains until configured delivery requirements have been fulfilled. -# +--- -# subscribe(PointId) +# 25. History Repository -# ``` +A History Repository is a queryable historical source. -# +It provides operations conceptually equivalent to: -# Historical backfill shall not directly call `update(CurrentValue)`. +```text -# +store -# \--- +storeBatch -# +readRaw -# \# 18. Historical-data architecture +readAtTime -# +getBounds -# Historical AquaCom data follows: +health -# +``` -# ```text +The repository used by OPC UA Historical Access is selected by configuration. -# AquaCom history +The OPC UA implementation shall not know whether the repository is: -# │ +```text -# ▼ +relational database -# Historical Acquisition Manager +time-series database -# │ +embedded historian -# ▼ +industrial historian -# Canonical HistoricalSample +other queryable implementation -# │ +``` -# ▼ +--- -# Historical Pipeline +# 26. Storage Sink -# │ +A Storage Sink is a destination receiving historical records. -# ├────────► History Repository +Conceptual interface: -# ├────────► Storage Sink A +```text -# ├────────► Storage Sink B +initialize -# └────────► Other configured destinations +write -# ``` +writeBatch -# +flush -# The historical pipeline is independent from OPC UA. +health -# +shutdown -# \--- +``` -# +A sink does not need to support historical queries. -# \# 19. Historical Acquisition Manager +Examples: -# +```text -# The Historical Acquisition Manager is responsible for: +SQL database -# +time-series database -# \- determining missing intervals; +enterprise historian -# \- requesting AquaCom history; +file archive -# \- processing historical responses; +message broker -# \- maintaining synchronization state; +data lake connector -# \- managing watermarks; +``` -# \- performing reconnect/backfill; +--- -# \- throttling history retrieval; +# 27. Repository versus sink -# \- avoiding interference with live polling. +These shall remain separate concepts: -# +```text -# Live acquisition shall normally have higher communication priority than bulk history recovery. +Repository -# + = query historical data -# \--- +Sink -# + = receive historical data -# \# 20. Historical synchronization +``` -# +One implementation may support both. -# Each station may maintain persistent synchronization state. +Example: -# +```text -# Conceptually: +Local historian -# + Repository: yes -# ```text + Sink: yes -# StationId +Archive export -# HistoryStreamId + Repository: no -# LastCompleteTimestamp + Sink: yes -# LastAttemptTimestamp +``` -# LastSuccessfulTimestamp +--- -# SynchronizationState +# 28. Multiple storage destinations -# ``` +A historical record may be delivered to multiple independent destinations. -# +Example: -# Example: +```text -# + Historical Pipeline -# ```text + │ -# 12:00 history synchronized + ┌───────────────┼────────────────┐ -# + │ │ │ -# 12:03 communication lost + ▼ ▼ ▼ -# + Local Repository Central DB Archive -# 17:24 communication restored +``` -# +Failure of one destination shall not unnecessarily block successful destinations. -# 12:00 → 17:24 +--- -# identified as recovery interval +# 29. OPC UA architecture -# ``` +The OPC UA subsystem provides northbound access to normalized gateway information. -# +Logical components: -# The gateway then retrieves available AquaCom history for that interval. +```text -# +OPC UA Server -# \--- + │ -# + ├── Namespace Manager -# \# 21. Watermark rule + ├── Current Value Provider -# + ├── Historical Provider -# A history synchronization watermark shall only advance after the corresponding interval has been successfully handled according to the configured persistence policy. + ├── Quality Mapper -# + ├── Diagnostics Provider -# The following is prohibited: + └── Security -# +``` -# ```text +--- -# receive historical packet +# 30. OPC UA current data -# │ +Current values are exposed through normal OPC UA Data Access. -# ▼ +Mapping: -# advance watermark +```text -# │ +CurrentValue.Value -# ▼ + ↓ -# storage fails +OPC UA Value -# ``` +CurrentValue.SourceTimestamp -# + ↓ -# Preferred behavior is: +OPC UA SourceTimestamp -# +Gateway server time -# ```text + ↓ -# receive +OPC UA ServerTimestamp -# ↓ +CurrentValue.Quality -# validate + ↓ -# ↓ +OPC UA StatusCode -# persist/durably queue +``` -# ↓ +--- -# advance synchronization state +# 31. OPC UA Historical Access -# ``` +Historical Access shall query the configured History Repository. -# +Correct flow: -# \--- +```text -# +OPC UA Client -# \# 22. Historical pipeline + │ -# + │ HistoryRead -# The historical pipeline shall conceptually perform: + ▼ -# +OPC UA Server -# ```text + │ -# HistoricalSample + ▼ -# │ +Historical Provider -# ▼ + │ -# Validation + ▼ -# │ +History Repository -# ▼ + │ -# Normalization + ▼ -# │ +HistoricalSample[] -# ▼ + │ -# Deduplication + ▼ -# │ +OPC UA DataValue[] -# ▼ +``` -# Durable Spool +--- -# │ +# 32. Prohibited HistoryRead architecture -# ▼ +The following architecture is prohibited: -# Distribution +```text -# ``` +OPC UA HistoryRead -# + │ -# This pipeline provides a stable boundary between acquisition and storage. + ▼ -# +AquaCom request -# \--- + │ -# + ▼ -# \# 23. Deduplication +wait for RTU -# + │ -# The default historical logical identity is: + ▼ -# +OPC UA response -# ```text +``` -# StationId +Historical synchronization and historical client access must be decoupled. -# \+ +--- -# PointId +# 33. OPC UA namespace -# \+ +The default logical namespace shall resemble: -# SourceTimestamp +```text -# ``` +AquaCom -# +│ -# A stronger AquaCom-provided identifier may be used where verified. +├── Gateway -# +│ ├── Status -# Identical samples shall not create unintended duplicate history. +│ ├── Version -# +│ ├── StartTime -# Conflicting samples sharing the same logical identity shall generate diagnostics. +│ ├── ConfigurationVersion -# +│ ├── HistoricalQueueDepth -# The default conflict strategy should favor explicit conflict detection rather than silent overwrite. +│ └── Storage -# +│ -# \--- +└── Stations -# + │ -# \# 24. Durable spool + ├── PS001 -# + │ ├── Process -# The historical spool provides restart-safe delivery. + │ │ ├── Level -# + │ │ ├── Pump01Running -# Conceptually: + │ │ └── Pump02Running -# + │ │ -# ```text + │ └── Diagnostics -# HistoricalSample + │ ├── CommunicationState -# │ + │ ├── LastCommunication -# ▼ + │ ├── HistoryState -# Persistent spool + │ └── HistoryWatermark -# │ + │ -# ├────► Repository + └── PS002 -# ├────► Sink A +``` -# └────► Sink B +Exact organization may be configurable. -# ``` +--- -# +# 34. Stable OPC UA NodeIds -# Each destination maintains independent acknowledgement. +OPC UA NodeIds shall derive from stable engineering identifiers. -# +Example: -# Example: +```text -# +station/PS001/point/LEVEL -# ```text +``` -# Record 4711 +Changing: -# +```text -# Repository ACK +Wet well level -# CentralHistorian ACK +``` -# Archive PENDING +to: -# ``` +```text -# +Inlet sump level -# The record remains until configured delivery requirements have been fulfilled. +``` -# +shall not normally change the NodeId. -# \--- +Stable NodeIds are required to preserve client configuration. -# +--- -# \# 25. History Repository +# 35. Diagnostics architecture -# +Diagnostics are separate from process data. -# A History Repository is a queryable historical source. +Conceptually: -# +```text -# It provides operations conceptually equivalent to: +Diagnostics -# +│ -# ```text +├── Gateway -# store +│ -# storeBatch +├── Station -# readRaw +│ -# readAtTime +├── AquaCom Protocol -# getBounds +│ -# health +├── History -# ``` +│ -# +├── Storage -# The repository used by OPC UA Historical Access is selected by configuration. +└── OPC UA -# +``` -# The OPC UA implementation shall not know whether the repository is: +Diagnostics may be exposed through: -# +- logs; -# ```text +- metrics; -# relational database +- OPC UA diagnostic nodes; -# time-series database +- future management interfaces. -# embedded historian +--- -# industrial historian +# 36. Gateway diagnostics -# other queryable implementation +Examples: -# ``` +```text -# +GatewayState -# \--- +Version -# +ConfigurationVersion -# \# 26. Storage Sink +StartTime -# +Uptime -# A Storage Sink is a destination receiving historical records. +StationCount -# +OnlineStationCount -# Conceptual interface: +HistoricalQueueDepth -# +SpoolDiskUsage -# ```text +OPCUAState -# initialize +``` -# write +--- -# writeBatch +# 37. Station diagnostics -# flush +Examples: -# health +```text -# shutdown +CommunicationState -# ``` +LastRequest -# +LastResponse -# A sink does not need to support historical queries. +LastSuccessfulCommunication -# +ConsecutiveFailures -# Examples: +ProtocolErrors -# +HistorySynchronizationState -# ```text +HistoryWatermark -# SQL database +HistoricalBacklog -# time-series database +``` -# enterprise historian +--- -# file archive +# 38. Storage diagnostics -# message broker +Per destination: -# data lake connector +```text -# ``` +State -# +LastSuccessfulWrite -# \--- +LastFailure -# +PendingRecords -# \# 27. Repository versus sink +RetryCount -# +Latency -# These shall remain separate concepts: +``` -# +A storage failure shall be identifiable independently from an AquaCom communication failure. -# ```text +--- -# Repository +# 39. Logging -# = query historical data +Structured logging is preferred. -# +Useful context includes: -# Sink +```text -# = receive historical data +timestamp -# ``` +severity -# +component -# One implementation may support both. +stationId -# +pointId -# Example: +operation -# +correlationId -# ```text +message -# Local historian +error -# Repository: yes +``` -# Sink: yes +Raw AquaCom telegram logging shall be separately configurable. -# +--- -# Archive export +# 40. Configuration architecture -# Repository: no +Configuration shall be external and machine-readable. -# Sink: yes +Conceptual structure: -# ``` +```text -# +Gateway Configuration -# \--- +│ -# +├── Gateway -# \# 28. Multiple storage destinations +├── AquaCom -# +├── Stations -# A historical record may be delivered to multiple independent destinations. +├── Points -# +├── History -# Example: +├── Storage -# +├── OPC UA -# ```text +├── Security -# Historical Pipeline +├── Diagnostics -# │ +└── Logging -# ┌───────────────┼────────────────┐ +``` -# │ │ │ +Configuration validation occurs before activation. -# ▼ ▼ ▼ +--- -# Local Repository Central DB Archive +# 41. Station configuration -# ``` +A station definition contains concepts equivalent to: -# +```text -# Failure of one destination shall not unnecessarily block successful destinations. +StationId -# +Enabled -# \--- +Transport -# +AquaComAddress -# \# 29. OPC UA architecture +ProtocolProfile -# +LiveAcquisition -# The OPC UA subsystem provides northbound access to normalized gateway information. +HistoricalAcquisition -# +PointDefinitionSource -# Logical components: +Timezone -# +``` -# ```text +Station configuration shall not be embedded directly in protocol source code. -# OPC UA Server +--- -# │ +# 42. Point configuration -# ├── Namespace Manager +A point contains concepts equivalent to: -# ├── Current Value Provider +```text -# ├── Historical Provider +PointId -# ├── Quality Mapper +DisplayName -# ├── Diagnostics Provider +Description -# └── Security +SourceReference -# ``` +DataType -# +EngineeringUnit -# \--- +Scaling -# +LiveEnabled -# \# 30. OPC UA current data +HistoryEnabled -# +OPCUAEnabled -# Current values are exposed through normal OPC UA Data Access. +Historizing -# +StoragePolicy -# Mapping: +``` -# +`PointId` shall remain stable even if presentation metadata changes. -# ```text +--- -# CurrentValue.Value +# 43. Security boundaries -# ↓ +The architecture assumes AquaCom is a legacy field protocol. -# OPC UA Value +Security shall therefore be applied around the AquaCom communication environment. -# +Conceptually: -# CurrentValue.SourceTimestamp +```text -# ↓ +SCADA / client network -# OPC UA SourceTimestamp + │ -# + ▼ -# Gateway server time +Firewall / segmentation -# ↓ + │ -# OPC UA ServerTimestamp + ▼ -# +AquaCom Gateway -# CurrentValue.Quality + │ -# ↓ + ▼ -# OPC UA StatusCode +OT communication network -# ``` + │ -# + ▼ -# \--- +Site firewall/router/VPN -# + │ -# \# 31. OPC UA Historical Access + ▼ -# +AquaCom RTU -# Historical Access shall query the configured History Repository. +``` -# +The gateway shall not assume that AquaCom itself provides modern confidentiality, authentication or authorization. -# Correct flow: +--- -# +# 44. OPC UA security -# ```text +OPC UA security is handled within the OPC UA subsystem. -# OPC UA Client +Production architecture shall support: -# │ +```text -# │ HistoryRead +application certificates -# ▼ +trusted client management -# OPC UA Server +secure channels -# │ +authentication -# ▼ +authorization -# Historical Provider +encrypted communication -# │ +``` -# ▼ +Insecure endpoints shall not be the normal production default. -# History Repository +--- -# │ +# 45. Secrets boundary -# ▼ +Secrets shall not form part of normal version-controlled configuration. -# HistoricalSample\[] +Examples: -# │ +```text -# ▼ +database passwords -# OPC UA DataValue\[] +API tokens -# ``` +private keys -# +certificate private keys -# \--- +``` -# +These shall be injected through an external secret mechanism. -# \# 32. Prohibited HistoryRead architecture +--- -# +# 46. Host subsystem -# The following architecture is prohibited: +The `host` subsystem composes the application. -# +Responsibilities include: -# ```text +- configuration loading; -# OPC UA HistoryRead +- dependency construction; -# │ +- subsystem startup; -# ▼ +- lifecycle management; -# AquaCom request +- shutdown; -# │ +- top-level health state. -# ▼ +The host shall not become a location for protocol or business logic. -# wait for RTU +--- -# │ +# 47. Startup architecture -# ▼ +Conceptual startup sequence: -# OPC UA response +```text -# ``` +Load configuration -# + │ -# Historical synchronization and historical client access must be decoupled. + ▼ -# +Validate configuration -# \--- + │ -# + ▼ -# \# 33. OPC UA namespace +Load persistent state -# + │ -# The default logical namespace shall resemble: + ▼ -# +Initialize storage -# ```text + │ -# AquaCom + ▼ -# │ +Initialize historical services -# ├── Gateway + │ -# │ ├── Status + ▼ -# │ ├── Version +Initialize OPC UA -# │ ├── StartTime + │ -# │ ├── ConfigurationVersion + ▼ -# │ ├── HistoricalQueueDepth +Initialize AquaCom stations -# │ └── Storage + │ -# │ + ▼ -# └── Stations +Start live acquisition -# │ + │ -# ├── PS001 + ▼ -# │ ├── Process +Start historical synchronization -# │ │ ├── Level +``` -# │ │ ├── Pump01Running +A temporary failure of a non-critical secondary storage destination should not necessarily prevent the gateway from starting. -# │ │ └── Pump02Running +--- -# │ │ +# 48. Shutdown architecture -# │ └── Diagnostics +Conceptual controlled shutdown: -# │ ├── CommunicationState +```text -# │ ├── LastCommunication +Stop accepting administrative changes -# │ ├── HistoryState + │ -# │ └── HistoryWatermark + ▼ -# │ +Stop scheduling new acquisition -# └── PS002 + │ -# ``` + ▼ -# +Complete/cancel active transactions safely -# Exact organization may be configurable. + │ -# + ▼ -# \--- +Persist synchronization state -# + │ -# \# 34. Stable OPC UA NodeIds + ▼ -# +Flush durable state -# OPC UA NodeIds shall derive from stable engineering identifiers. + │ -# + ▼ -# Example: +Stop OPC UA -# + │ -# ```text + ▼ -# station/PS001/point/LEVEL +Close storage -# ``` + │ -# + ▼ -# Changing: +Terminate -# +``` -# ```text +--- -# Wet well level +# 49. Restart recovery -# ``` +After an unclean restart: -# +```text -# to: +Load configuration -# + │ -# ```text + ▼ -# Inlet sump level +Recover persistent spool -# ``` + │ -# + ▼ -# shall not normally change the NodeId. +Restore synchronization watermarks -# + │ -# Stable NodeIds are required to preserve client configuration. + ▼ -# +Reconnect AquaCom stations -# \--- + │ -# + ▼ -# \# 35. Diagnostics architecture +Resume live acquisition -# + │ -# Diagnostics are separate from process data. + ▼ -# +Resume historical recovery -# Conceptually: +``` -# +Live acquisition should resume before large backlog processing. -# ```text +--- -# Diagnostics +# 50. Scheduling priorities -# │ +Where AquaCom communication capacity is limited, scheduling shall conceptually prioritize: -# ├── Gateway +```text -# │ +1. communication/session health -# ├── Station +2. current/live acquisition -# │ +3. alarms/events when implemented -# ├── AquaCom Protocol +4. explicit configuration operations -# │ +5. historical backfill -# ├── History +6. low-priority maintenance -# │ +``` -# ├── Storage +Historical synchronization shall be throttleable. -# └── OPC UA +--- -# ``` +# 51. Resource boundaries -# +All queues and caches shall be bounded. -# Diagnostics may be exposed through: +This applies to: -# +```text -# \- logs; +AquaCom request queues -# \- metrics; +history queues -# \- OPC UA diagnostic nodes; +storage retry queues -# \- future management interfaces. +OPC UA historical result sets -# +continuation state -# \--- +logging buffers -# +spool storage -# \# 36. Gateway diagnostics +``` -# +Overflow behavior shall be explicit and observable. -# Examples: +--- -# +# 52. Testing architecture -# ```text +The system shall support independent testing of each architectural boundary. -# GatewayState +Conceptually: -# Version +```text -# ConfigurationVersion +Protocol Fixtures -# StartTime + │ -# Uptime + ▼ -# StationCount +AquaCom Codec Tests -# OnlineStationCount +Simulator -# HistoricalQueueDepth + │ -# SpoolDiskUsage + ▼ -# OPCUAState +Session / Acquisition Tests -# ``` +Historical Fixtures -# + │ -# \--- + ▼ -# +History Pipeline Tests -# \# 37. Station diagnostics +Repository -# + │ -# Examples: + ▼ -# +OPC UA HA Tests -# ```text +Current-State Service -# CommunicationState + │ -# LastRequest + ▼ -# LastResponse +OPC UA DA Tests -# LastSuccessfulCommunication +``` -# ConsecutiveFailures +--- -# ProtocolErrors +# 53. AquaCom simulator -# HistorySynchronizationState +The simulator shall permit development without operational AquaCom equipment. -# HistoryWatermark +Supported scenarios shall eventually include: -# HistoricalBacklog +```text -# ``` +normal live values -# +normal history -# \--- +timeouts -# +disconnects -# \# 38. Storage diagnostics +delayed response -# +malformed telegram -# Per destination: +bad checksum -# +duplicate history -# ```text +conflicting history -# State +clock error -# LastSuccessfulWrite +reconnect -# LastFailure +backfill -# PendingRecords +``` -# RetryCount +The simulator is part of the architecture, not merely a temporary development tool. -# Latency +--- -# ``` +# 54. Protocol evidence architecture -# +AquaCom implementation is linked to documented evidence. -# A storage failure shall be identifiable independently from an AquaCom communication failure. +Relevant repository area: -# +```text -# \--- +docs/protocol/ -# +``` -# \# 39. Logging +Recommended structure: -# +```text -# Structured logging is preferred. +aquacom.md -# +known-operations.md -# Useful context includes: +protocol-gaps.md -# +test-vectors.md -# ```text +``` -# timestamp +Every implemented AquaCom behavior should be traceable to: -# severity +```text -# component +Specification -# stationId +Capture -# pointId +Known-good implementation -# operation +Approved test vector -# correlationId +``` -# message +Unknown behavior remains explicitly unresolved. -# error +--- -# ``` +# 55. Error isolation -# +Failures shall remain contained where possible. -# Raw AquaCom telegram logging shall be separately configurable. +Examples: -# +| Failure | Expected isolation | -# \--- +|---|---| -# +| Station A offline | Station B continues | -# \# 40. Configuration architecture +| Storage A offline | Storage B continues | -# +| OPC UA client disconnect | Acquisition continues | -# Configuration shall be external and machine-readable. +| OPC UA server problem | AquaCom acquisition should continue where possible | -# +| Historical backfill fails | Live acquisition continues | -# Conceptual structure: +| Malformed AquaCom telegram | Telegram rejected, station diagnostics updated | -# +| Secondary storage failure | Durable spool retains pending delivery | -# ```text +--- -# Gateway Configuration +# 56. Future extension points -# │ +The architecture shall permit future addition of: -# ├── Gateway +```text -# ├── AquaCom +OPC UA Alarms & Conditions -# ├── Stations +AquaCom alarms/events -# ├── Points +AquaCom commands -# ├── History +setpoint changes -# ├── Storage +MQTT -# ├── OPC UA +REST API -# ├── Security +external analytics -# ├── Diagnostics +central configuration -# └── Logging +redundant gateways -# ``` +high availability -# +additional history repositories -# Configuration validation occurs before activation. +additional storage sinks -# +``` -# \--- +These shall be added behind explicit subsystem interfaces. -# +--- -# \# 41. Station configuration +# 57. Future process-control architecture -# +If control functionality is introduced later, it shall use a separate command pipeline: -# A station definition contains concepts equivalent to: +```text -# +Northbound request -# ```text + │ -# StationId + ▼ -# Enabled +Authentication -# Transport + │ -# AquaComAddress + ▼ -# ProtocolProfile +Authorization -# LiveAcquisition + │ -# HistoricalAcquisition + ▼ -# PointDefinitionSource +Command validation -# Timezone + │ -# ``` + ▼ -# +Command service -# Station configuration shall not be embedded directly in protocol source code. + │ -# + ▼ -# \--- +AquaCom command -# + │ -# \# 42. Point configuration + ▼ -# +Confirmation / readback -# A point contains concepts equivalent to: + │ -# + ▼ -# ```text +Audit -# PointId +``` -# DisplayName +A direct mapping from arbitrary OPC UA writes to raw AquaCom telegrams is not an acceptable control architecture. -# Description +--- -# SourceReference +# 58. Architectural invariants -# DataType +The following invariants shall hold throughout implementation. -# EngineeringUnit +## INV-001 -# Scaling +AquaCom-specific logic exists only behind the AquaCom abstraction. -# LiveEnabled +## INV-002 -# HistoryEnabled +The canonical model contains no AquaCom telegram representation. -# OPCUAEnabled +## INV-003 -# Historizing +The canonical model contains no OPC UA-specific representation. -# StoragePolicy +## INV-004 -# ``` +Historical samples do not update current state solely because they were recently retrieved. -# +## INV-005 -# `PointId` shall remain stable even if presentation metadata changes. +Historical SourceTimestamp is preserved. -# +## INV-006 -# \--- +OPC UA HistoryRead queries a repository, not an AquaCom device. -# +## INV-007 -# \# 43. Security boundaries +A storage failure does not directly stop unrelated live AquaCom acquisition. -# +## INV-008 -# The architecture assumes AquaCom is a legacy field protocol. +Point identity remains stable across display-name changes. -# +## INV-009 -# Security shall therefore be applied around the AquaCom communication environment. +OPC UA NodeIds remain stable across restart and normal upgrades. -# +## INV-010 -# Conceptually: +Unverified AquaCom protocol details are not implemented as guesses. -# +## INV-011 -# ```text +Synchronization state required for recovery is persistent. -# SCADA / client network +## INV-012 -# │ +Historical delivery supports bounded persistent buffering. -# ▼ +## INV-013 -# Firewall / segmentation +Historical samples may arrive out of chronological order; storage adapters support out-of-order writes and queries order by SourceTimestamp. -# │ +## INV-014 -# ▼ +The history synchronization watermark advances only on durable spool commit, and restart recovery of undelivered samples is bounded to the spool rather than the field device. -# AquaCom Gateway +## INV-015 -# │ +Deduplication remains correct across restart; a restart during synchronization does not silently introduce duplicate historical records. -# ▼ +## INV-016 -# OT communication network +A historical record without a usable SourceTimestamp is quarantined with diagnostics, never silently stored as process history. -# │ +--- -# ▼ +# 59. Primary end-to-end data flows -# Site firewall/router/VPN +## 59.1 Live data -# │ +```text -# ▼ +AquaCom RTU -# AquaCom RTU + │ -# ``` + ▼ -# +Transport -# The gateway shall not assume that AquaCom itself provides modern confidentiality, authentication or authorization. + │ -# + ▼ -# \--- +Protocol Engine -# + │ -# \# 44. OPC UA security + ▼ -# +Station Manager -# OPC UA security is handled within the OPC UA subsystem. + │ -# + ▼ -# Production architecture shall support: +CurrentValue -# + │ -# ```text + ▼ -# application certificates +Current-State Service -# trusted client management + │ -# secure channels + ├────► OPC UA -# authentication + │ -# authorization + └────► Historical Pipeline -# encrypted communication + when live historization is enabled -# ``` +``` -# +--- -# Insecure endpoints shall not be the normal production default. +## 59.2 AquaCom historical recovery -# +```text -# \--- +AquaCom RTU -# + │ -# \# 45. Secrets boundary + ▼ -# +Historical Acquisition -# Secrets shall not form part of normal version-controlled configuration. + │ -# + ▼ -# Examples: +HistoricalSample -# + │ -# ```text + ▼ -# database passwords +Validation -# API tokens + │ -# private keys + ▼ -# certificate private keys +Deduplication -# ``` + │ -# + ▼ -# These shall be injected through an external secret mechanism. +Durable Spool -# + │ -# \--- + ├────► Repository -# + ├────► Storage A -# \# 46. Host subsystem + └────► Storage B -# +``` -# The `host` subsystem composes the application. +--- -# +## 59.3 OPC UA historical request -# Responsibilities include: +```text -# +OPC UA Client -# \- configuration loading; + │ -# \- dependency construction; + ▼ -# \- subsystem startup; +HistoryRead -# \- lifecycle management; + │ -# \- shutdown; + ▼ -# \- top-level health state. +OPC UA Historical Provider -# + │ -# The host shall not become a location for protocol or business logic. + ▼ -# +History Repository -# \--- + │ -# + ▼ -# \# 47. Startup architecture +HistoricalSample[] -# + │ -# Conceptual startup sequence: + ▼ -# +OPC UA DataValue[] -# ```text +``` -# Load configuration +--- -# │ +# 60. Reference architecture -# ▼ +The complete reference architecture is: -# Validate configuration +```text -# │ + FIELD / AQUACOM -# ▼ + │ -# Load persistent state + ▼ -# │ + ┌─────────────────────┐ -# ▼ + │ AquaCom Transport │ -# Initialize storage + └──────────┬──────────┘ -# │ + │ -# ▼ + ▼ -# Initialize historical services + ┌─────────────────────┐ -# │ + │ Protocol Engine │ -# ▼ + └──────────┬──────────┘ -# Initialize OPC UA + │ -# │ + ▼ -# ▼ + ┌─────────────────────┐ -# Initialize AquaCom stations + │ Station Manager │ -# │ + └──────────┬──────────┘ -# ▼ + │ -# Start live acquisition + ▼ -# │ + ┌─────────────────────┐ -# ▼ + │ Canonical Model │ -# Start historical synchronization + └───────┬───────┬─────┘ -# ``` + │ │ -# + LIVE HISTORY -# A temporary failure of a non-critical secondary storage destination should not necessarily prevent the gateway from starting. + │ │ -# + ▼ ▼ -# \--- + ┌──────────────┐ ┌──────────────────┐ -# + │Current-State │ │Historical │ -# \# 48. Shutdown architecture + │Service │ │Acquisition │ -# + └──────┬───────┘ └────────┬─────────┘ -# Conceptual controlled shutdown: + │ │ -# + │ ▼ -# ```text + │ ┌──────────────────┐ -# Stop accepting administrative changes + │ │Historical │ -# │ + │ │Pipeline │ -# ▼ + │ └────────┬─────────┘ -# Stop scheduling new acquisition + │ │ -# │ + │ ┌──────────┼───────────┐ -# ▼ + │ │ │ │ -# Complete/cancel active transactions safely + │ ▼ ▼ ▼ -# │ + │ Repository Sink A Sink B -# ▼ + │ │ -# Persist synchronization state + │ │ -# │ + ▼ ▼ -# ▼ + ┌────────────────────────────┐ -# Flush durable state + │ OPC UA Server │ -# │ + │ │ -# ▼ + │ Data Access │ -# Stop OPC UA + │ Historical Access │ -# │ + │ Diagnostics │ -# ▼ + └──────────────┬─────────────┘ -# Close storage + │ -# │ + ▼ -# ▼ + SCADA / Clients -# Terminate +``` -# ``` +--- -# +# 61. Architecture decision summary -# \--- +The gateway is intentionally designed as an: -# +```text -# \# 49. Restart recovery +AquaCom Integration Gateway -# +``` -# After an unclean restart: +rather than merely an: -# +```text -# ```text +AquaCom OPC UA Server -# Load configuration +``` -# │ +This allows: -# ▼ +- OPC UA to be replaced or supplemented; -# Recover persistent spool +- storage technology to change; -# │ +- multiple historians to coexist; -# ▼ +- AquaCom equipment to be gradually retired; -# Restore synchronization watermarks +- other northbound interfaces to be added; -# │ +- historical acquisition to continue independently of OPC UA clients. -# ▼ +The long-term architectural asset is therefore the normalized integration layer and historical recovery mechanism, not any single external interface. -# Reconnect AquaCom stations +--- -# │ +# 62. Final architectural rule -# ▼ +The most important data rule is: -# Resume live acquisition +> Historical data is stored and exposed according to when the process value was valid, not when the gateway happened to retrieve it. -# │ +The most important protocol rule is: -# ▼ +> AquaCom behavior must be based on verified protocol knowledge and must never be invented merely to complete an implementation. -# Resume historical recovery +The most important software rule is: -# ``` - -# - -# Live acquisition should resume before large backlog processing. - -# - -# \--- - -# - -# \# 50. Scheduling priorities - -# - -# Where AquaCom communication capacity is limited, scheduling shall conceptually prioritize: - -# - -# ```text - -# 1\. communication/session health - -# 2\. current/live acquisition - -# 3\. alarms/events when implemented - -# 4\. explicit configuration operations - -# 5\. historical backfill - -# 6\. low-priority maintenance - -# ``` - -# - -# Historical synchronization shall be throttleable. - -# - -# \--- - -# - -# \# 51. Resource boundaries - -# - -# All queues and caches shall be bounded. - -# - -# This applies to: - -# - -# ```text - -# AquaCom request queues - -# history queues - -# storage retry queues - -# OPC UA historical result sets - -# continuation state - -# logging buffers - -# spool storage - -# ``` - -# - -# Overflow behavior shall be explicit and observable. - -# - -# \--- - -# - -# \# 52. Testing architecture - -# - -# The system shall support independent testing of each architectural boundary. - -# - -# Conceptually: - -# - -# ```text - -# Protocol Fixtures - -# │ - -# ▼ - -# AquaCom Codec Tests - -# - -# Simulator - -# │ - -# ▼ - -# Session / Acquisition Tests - -# - -# Historical Fixtures - -# │ - -# ▼ - -# History Pipeline Tests - -# - -# Repository - -# │ - -# ▼ - -# OPC UA HA Tests - -# - -# Current-State Service - -# │ - -# ▼ - -# OPC UA DA Tests - -# ``` - -# - -# \--- - -# - -# \# 53. AquaCom simulator - -# - -# The simulator shall permit development without operational AquaCom equipment. - -# - -# Supported scenarios shall eventually include: - -# - -# ```text - -# normal live values - -# normal history - -# timeouts - -# disconnects - -# delayed response - -# malformed telegram - -# bad checksum - -# duplicate history - -# conflicting history - -# clock error - -# reconnect - -# backfill - -# ``` - -# - -# The simulator is part of the architecture, not merely a temporary development tool. - -# - -# \--- - -# - -# \# 54. Protocol evidence architecture - -# - -# AquaCom implementation is linked to documented evidence. - -# - -# Relevant repository area: - -# - -# ```text - -# docs/protocol/ - -# ``` - -# - -# Recommended structure: - -# - -# ```text - -# aquacom.md - -# known-operations.md - -# protocol-gaps.md - -# test-vectors.md - -# ``` - -# - -# Every implemented AquaCom behavior should be traceable to: - -# - -# ```text - -# Specification - -# Capture - -# Known-good implementation - -# Approved test vector - -# ``` - -# - -# Unknown behavior remains explicitly unresolved. - -# - -# \--- - -# - -# \# 55. Error isolation - -# - -# Failures shall remain contained where possible. - -# - -# Examples: - -# - -# | Failure | Expected isolation | - -# |---|---| - -# | Station A offline | Station B continues | - -# | Storage A offline | Storage B continues | - -# | OPC UA client disconnect | Acquisition continues | - -# | OPC UA server problem | AquaCom acquisition should continue where possible | - -# | Historical backfill fails | Live acquisition continues | - -# | Malformed AquaCom telegram | Telegram rejected, station diagnostics updated | - -# | Secondary storage failure | Durable spool retains pending delivery | - -# - -# \--- - -# - -# \# 56. Future extension points - -# - -# The architecture shall permit future addition of: - -# - -# ```text - -# OPC UA Alarms \& Conditions - -# AquaCom alarms/events - -# AquaCom commands - -# setpoint changes - -# MQTT - -# REST API - -# external analytics - -# central configuration - -# redundant gateways - -# high availability - -# additional history repositories - -# additional storage sinks - -# ``` - -# - -# These shall be added behind explicit subsystem interfaces. - -# - -# \--- - -# - -# \# 57. Future process-control architecture - -# - -# If control functionality is introduced later, it shall use a separate command pipeline: - -# - -# ```text - -# Northbound request - -# │ - -# ▼ - -# Authentication - -# │ - -# ▼ - -# Authorization - -# │ - -# ▼ - -# Command validation - -# │ - -# ▼ - -# Command service - -# │ - -# ▼ - -# AquaCom command - -# │ - -# ▼ - -# Confirmation / readback - -# │ - -# ▼ - -# Audit - -# ``` - -# - -# A direct mapping from arbitrary OPC UA writes to raw AquaCom telegrams is not an acceptable control architecture. - -# - -# \--- - -# - -# \# 58. Architectural invariants - -# - -# The following invariants shall hold throughout implementation. - -# - -# \## INV-001 - -# - -# AquaCom-specific logic exists only behind the AquaCom abstraction. - -# - -# \## INV-002 - -# - -# The canonical model contains no AquaCom telegram representation. - -# - -# \## INV-003 - -# - -# The canonical model contains no OPC UA-specific representation. - -# - -# \## INV-004 - -# - -# Historical samples do not update current state solely because they were recently retrieved. - -# - -# \## INV-005 - -# - -# Historical SourceTimestamp is preserved. - -# - -# \## INV-006 - -# - -# OPC UA HistoryRead queries a repository, not an AquaCom device. - -# - -# \## INV-007 - -# - -# A storage failure does not directly stop unrelated live AquaCom acquisition. - -# - -# \## INV-008 - -# - -# Point identity remains stable across display-name changes. - -# - -# \## INV-009 - -# - -# OPC UA NodeIds remain stable across restart and normal upgrades. - -# - -# \## INV-010 - -# - -# Unverified AquaCom protocol details are not implemented as guesses. - -# - -# \## INV-011 - -# - -# Synchronization state required for recovery is persistent. - -# - -# \## INV-012 - -# - -# Historical delivery supports bounded persistent buffering. - -# - -# \--- - -# - -# \# 59. Primary end-to-end data flows - -# - -# \## 59.1 Live data - -# - -# ```text - -# AquaCom RTU - -# │ - -# ▼ - -# Transport - -# │ - -# ▼ - -# Protocol Engine - -# │ - -# ▼ - -# Station Manager - -# │ - -# ▼ - -# CurrentValue - -# │ - -# ▼ - -# Current-State Service - -# │ - -# ├────► OPC UA - -# │ - -# └────► Historical Pipeline - -# when live historization is enabled - -# ``` - -# - -# \--- - -# - -# \## 59.2 AquaCom historical recovery - -# - -# ```text - -# AquaCom RTU - -# │ - -# ▼ - -# Historical Acquisition - -# │ - -# ▼ - -# HistoricalSample - -# │ - -# ▼ - -# Validation - -# │ - -# ▼ - -# Deduplication - -# │ - -# ▼ - -# Durable Spool - -# │ - -# ├────► Repository - -# ├────► Storage A - -# └────► Storage B - -# ``` - -# - -# \--- - -# - -# \## 59.3 OPC UA historical request - -# - -# ```text - -# OPC UA Client - -# │ - -# ▼ - -# HistoryRead - -# │ - -# ▼ - -# OPC UA Historical Provider - -# │ - -# ▼ - -# History Repository - -# │ - -# ▼ - -# HistoricalSample\[] - -# │ - -# ▼ - -# OPC UA DataValue\[] - -# ``` - -# - -# \--- - -# - -# \# 60. Reference architecture - -# - -# The complete reference architecture is: - -# - -# ```text - -# FIELD / AQUACOM - -# │ - -# ▼ - -# ┌─────────────────────┐ - -# │ AquaCom Transport │ - -# └──────────┬──────────┘ - -# │ - -# ▼ - -# ┌─────────────────────┐ - -# │ Protocol Engine │ - -# └──────────┬──────────┘ - -# │ - -# ▼ - -# ┌─────────────────────┐ - -# │ Station Manager │ - -# └──────────┬──────────┘ - -# │ - -# ▼ - -# ┌─────────────────────┐ - -# │ Canonical Model │ - -# └───────┬───────┬─────┘ - -# │ │ - -# LIVE HISTORY - -# │ │ - -# ▼ ▼ - -# ┌──────────────┐ ┌──────────────────┐ - -# │Current-State │ │Historical │ - -# │Service │ │Acquisition │ - -# └──────┬───────┘ └────────┬─────────┘ - -# │ │ - -# │ ▼ - -# │ ┌──────────────────┐ - -# │ │Historical │ - -# │ │Pipeline │ - -# │ └────────┬─────────┘ - -# │ │ - -# │ ┌──────────┼───────────┐ - -# │ │ │ │ - -# │ ▼ ▼ ▼ - -# │ Repository Sink A Sink B - -# │ │ - -# │ │ - -# ▼ ▼ - -# ┌────────────────────────────┐ - -# │ OPC UA Server │ - -# │ │ - -# │ Data Access │ - -# │ Historical Access │ - -# │ Diagnostics │ - -# └──────────────┬─────────────┘ - -# │ - -# ▼ - -# SCADA / Clients - -# ``` - -# - -# \--- - -# - -# \# 61. Architecture decision summary - -# - -# The gateway is intentionally designed as an: - -# - -# ```text - -# AquaCom Integration Gateway - -# ``` - -# - -# rather than merely an: - -# - -# ```text - -# AquaCom OPC UA Server - -# ``` - -# - -# This allows: - -# - -# \- OPC UA to be replaced or supplemented; - -# \- storage technology to change; - -# \- multiple historians to coexist; - -# \- AquaCom equipment to be gradually retired; - -# \- other northbound interfaces to be added; - -# \- historical acquisition to continue independently of OPC UA clients. - -# - -# The long-term architectural asset is therefore the normalized integration layer and historical recovery mechanism, not any single external interface. - -# - -# \--- - -# - -# \# 62. Final architectural rule - -# - -# The most important data rule is: - -# - -# > Historical data is stored and exposed according to when the process value was valid, not when the gateway happened to retrieve it. - -# - -# The most important protocol rule is: - -# - -# > AquaCom behavior must be based on verified protocol knowledge and must never be invented merely to complete an implementation. - -# - -# The most important software rule is: - -# - -# > AquaCom, canonical data handling, historical persistence and OPC UA shall remain separate architectural concerns. +> AquaCom, canonical data handling, historical persistence and OPC UA shall remain separate architectural concerns. diff --git a/docs/configuration/configuration-reference.md b/docs/configuration/configuration-reference.md index b662466..6aec5f8 100644 --- a/docs/configuration/configuration-reference.md +++ b/docs/configuration/configuration-reference.md @@ -1,173 +1,113 @@ -\# AquaCom Integration Gateway – Configuration Reference - - - -\## 1. Purpose - - - -This document defines the configuration model for the \*\*AquaCom Integration Gateway\*\*. +# AquaCom Integration Gateway – Configuration Reference +## 1. Purpose +This document defines the configuration model for the **AquaCom Integration Gateway**. It is the authoritative human-readable reference for: +- configuration sections; +- configuration fields; -\- configuration sections; +- required and optional values; -\- configuration fields; +- defaults; -\- required and optional values; - -\- defaults; - -\- validation rules; - -\- relationships between configuration areas; - -\- example configuration. +- validation rules; +- relationships between configuration areas; +- example configuration. The configuration model shall remain implementation-language independent. - - Machine-readable validation schemas shall be maintained separately under: - - ```text schemas/ ``` - - This document explains what the configuration means. - - The schema files define what configuration is syntactically and structurally valid. +--- - -\--- - - - -\# 2. Configuration principles - - +# 2. Configuration principles Gateway configuration shall be: +- external to the application executable; +- machine-readable; -\- external to the application executable; +- versioned; -\- machine-readable; +- schema-validated; -\- versioned; +- suitable for source control; -\- schema-validated; - -\- suitable for source control; - -\- portable between environments; - -\- explicit about identifiers; - -\- independent of runtime object allocation. +- portable between environments; +- explicit about identifiers; +- independent of runtime object allocation. Production secrets shall not be stored directly in normal version-controlled configuration. - - Configuration syntax shown in this document uses YAML for readability. - - The final implementation may use YAML, JSON, TOML, or another structured format, provided that the semantic model defined here is preserved. +--- - -\--- - - - -\# 3. Configuration hierarchy - - +# 3. Configuration hierarchy The gateway configuration shall conceptually contain: - - ```yaml configVersion: 1 - - gateway: - ... - - + ... logging: - ... - - + ... diagnostics: - ... - - + ... security: - ... - - + ... opcua: - ... - - + ... history: - ... - - + ... storage: - ... - - + ... stations: - ... + ... ``` - - The top-level configuration sections are: - - | Section | Purpose | |---|---| @@ -190,320 +130,198 @@ The top-level configuration sections are: | `stations` | AquaCom station definitions | +--- +# 4. Configuration version -\--- - - - -\# 4. Configuration version - - - -\## 4.1 `configVersion` - - - -\*\*Type:\*\* integer - -\*\*Required:\*\* yes +## 4.1 `configVersion` +**Type:** integer +**Required:** yes Defines the version of the gateway configuration schema. - - Example: - - ```yaml configVersion: 1 ``` - - The application shall reject configuration using an unsupported schema version. - - Configuration migration between incompatible schema versions shall be explicit. - - The application shall not silently reinterpret an incompatible configuration format. +--- - -\--- - - - -\# 5. Gateway configuration - - +# 5. Gateway configuration Example: - - ```yaml gateway: - id: aquacom-gateway-01 + id: aquacom-gateway-01 - displayName: Central AquaCom Gateway + displayName: Central AquaCom Gateway - timezone: Europe/Stockholm + timezone: Europe/Stockholm + startup: + allowDegradedStartup: true - startup: + shutdown: - allowDegradedStartup: true - - - - shutdown: - - gracefulTimeout: 30s + gracefulTimeout: 30s ``` +--- +## 5.1 `gateway.id` -\--- - - - -\## 5.1 `gateway.id` - - - -\*\*Type:\*\* string - -\*\*Required:\*\* yes - +**Type:** string +**Required:** yes Stable identifier for this gateway instance. - - Example: - - ```yaml id: aquacom-gateway-01 ``` - - Requirements: +- must be unique within the relevant operational environment; +- shall remain stable across software restart; -\- must be unique within the relevant operational environment; - -\- shall remain stable across software restart; - -\- shall normally remain stable across software upgrade; - -\- shall not be based on runtime-generated identifiers. - +- shall normally remain stable across software upgrade; +- shall not be based on runtime-generated identifiers. This identifier may be used in: +- diagnostics; +- logging; -\- diagnostics; +- historical metadata; -\- logging; +- OPC UA namespace generation; -\- historical metadata; +- audit records. -\- OPC UA namespace generation; +--- -\- audit records. - - - -\--- - - - -\## 5.2 `gateway.displayName` - - - -\*\*Type:\*\* string - -\*\*Required:\*\* no +## 5.2 `gateway.displayName` +**Type:** string +**Required:** no Human-readable name. - - Example: - - ```yaml displayName: Central AquaCom Gateway ``` - - Changing this value shall not change the gateway's stable identity. +--- +## 5.3 `gateway.timezone` -\--- - - - -\## 5.3 `gateway.timezone` - - - -\*\*Type:\*\* timezone identifier - -\*\*Required:\*\* yes - +**Type:** timezone identifier +**Required:** yes Defines the gateway's configured local timezone. - - Example: - - ```yaml timezone: Europe/Stockholm ``` - - UTC shall normally be used internally. - - This timezone may be used for: +- presentation; +- interpretation of timestamps where the source does not include timezone information; -\- presentation; - -\- interpretation of timestamps where the source does not include timezone information; - -\- diagnostics. - - +- diagnostics. A station may override this value. +--- +# 6. Startup configuration -\--- - - - -\# 6. Startup configuration - - - -\## 6.1 `gateway.startup.allowDegradedStartup` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `true` +## 6.1 `gateway.startup.allowDegradedStartup` +**Type:** boolean +**Default:** `true` Defines whether the gateway may start when non-critical external dependencies are unavailable. - - Example: - - ```yaml startup: - allowDegradedStartup: true + allowDegradedStartup: true ``` - - If enabled, failures such as an unavailable optional storage destination shall not necessarily prevent live AquaCom acquisition or OPC UA startup. - - Critical configuration errors shall always prevent startup. +--- +# 7. Shutdown configuration -\--- - - - -\# 7. Shutdown configuration - - - -\## 7.1 `gateway.shutdown.gracefulTimeout` - - - -\*\*Type:\*\* duration - -\*\*Default:\*\* implementation-defined +## 7.1 `gateway.shutdown.gracefulTimeout` +**Type:** duration +**Default:** implementation-defined Defines the maximum time allowed for controlled shutdown. - - Example: - - ```yaml shutdown: - gracefulTimeout: 30s + gracefulTimeout: 30s ``` - - During this period the gateway should attempt to: - - ```text stop new acquisition @@ -520,68 +338,44 @@ close storage ``` +--- - -\--- - - - -\# 8. Logging configuration - - +# 8. Logging configuration Example: - - ```yaml logging: - level: information + level: information + structured: true + telegramLogging: - structured: true + enabled: false + includePayload: false + retention: - telegramLogging: + maxFiles: 14 - enabled: false - - includePayload: false - - - - retention: - - maxFiles: 14 - - maxFileSize: 100MB + maxFileSize: 100MB ``` +--- +## 8.1 `logging.level` -\--- - - - -\## 8.1 `logging.level` - - - -\*\*Type:\*\* enumeration - -\*\*Required:\*\* no - +**Type:** enumeration +**Required:** no Typical values: - - ```text trace @@ -598,42 +392,26 @@ critical ``` - - Production default should normally be: - - ```yaml level: information ``` +--- +## 8.2 `logging.structured` -\--- - - - -\## 8.2 `logging.structured` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `true` - +**Type:** boolean +**Default:** `true` Enables structured logging where supported by the implementation. - - Structured log records should support fields such as: - - ```text timestamp @@ -656,282 +434,168 @@ error ``` +--- - -\--- - - - -\# 9. AquaCom telegram logging - - +# 9. AquaCom telegram logging Example: - - ```yaml telegramLogging: - enabled: false + enabled: false - includePayload: false + includePayload: false ``` - - Telegram logging shall be treated as a diagnostic capability. - - It shall not be enabled by default in production. +--- +## 9.1 `logging.telegramLogging.enabled` -\--- - - - -\## 9.1 `logging.telegramLogging.enabled` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `false` - +**Type:** boolean +**Default:** `false` Enables AquaCom telegram-level logging. +--- +## 9.2 `logging.telegramLogging.includePayload` -\--- - - - -\## 9.2 `logging.telegramLogging.includePayload` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `false` - +**Type:** boolean +**Default:** `false` Defines whether raw telegram payload is included. - - If enabled, operators shall consider: +- storage consumption; +- operational sensitivity; -\- storage consumption; - -\- operational sensitivity; - -\- possible exposure of process information. - - - -\--- - - - -\# 10. Diagnostics configuration +- possible exposure of process information. +--- +# 10. Diagnostics configuration Example: - - ```yaml diagnostics: - enabled: true + enabled: true + opcua: + exposeGatewayDiagnostics: true - opcua: + exposeStationDiagnostics: true - exposeGatewayDiagnostics: true + exposeStorageDiagnostics: true - exposeStationDiagnostics: true + metrics: - exposeStorageDiagnostics: true - - - - metrics: - - enabled: true + enabled: true ``` +--- +## 10.1 `diagnostics.enabled` -\--- - - - -\## 10.1 `diagnostics.enabled` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `true` - +**Type:** boolean +**Default:** `true` Enables gateway diagnostics. - - Disabling this shall not disable required internal error handling. +--- +## 10.2 `diagnostics.opcua.exposeGatewayDiagnostics` -\--- - - - -\## 10.2 `diagnostics.opcua.exposeGatewayDiagnostics` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `true` - +**Type:** boolean +**Default:** `true` Exposes gateway-level diagnostic nodes through OPC UA. +--- +## 10.3 `diagnostics.opcua.exposeStationDiagnostics` -\--- - - - -\## 10.3 `diagnostics.opcua.exposeStationDiagnostics` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `true` - +**Type:** boolean +**Default:** `true` Exposes per-station communication and synchronization diagnostics. +--- +## 10.4 `diagnostics.opcua.exposeStorageDiagnostics` -\--- - - - -\## 10.4 `diagnostics.opcua.exposeStorageDiagnostics` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `true` - +**Type:** boolean +**Default:** `true` Exposes historical-storage diagnostics. +--- +## 10.5 `diagnostics.metrics.enabled` -\--- - - - -\## 10.5 `diagnostics.metrics.enabled` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* implementation-defined - +**Type:** boolean +**Default:** implementation-defined Enables runtime metrics. - - The metrics transport or technology is implementation-specific. +--- - -\--- - - - -\# 11. Shared security configuration - - +# 11. Shared security configuration Example: - - ```yaml security: - secretsProvider: + secretsProvider: - type: environment + type: environment + certificates: - - certificates: - - directory: ./certificates + directory: ./certificates ``` - - This section contains shared security behavior. - - Protocol-specific settings belong under their corresponding subsystem. +--- +# 12. Secret handling -\--- - - - -\# 12. Secret handling - - - -\## 12.1 `security.secretsProvider.type` - - +## 12.1 `security.secretsProvider.type` Defines how secrets are supplied. - - Possible implementations may include: - - ```text environment @@ -946,590 +610,360 @@ external secret service ``` - - Production secrets shall not normally appear directly in the gateway configuration. - - Example: - - ```yaml database: - username: gateway\_user + username: gateway_user - passwordSecret: HISTORY\_DB\_PASSWORD + passwordSecret: HISTORY_DB_PASSWORD ``` - - rather than: - - ```yaml password: SuperSecretPassword123 ``` +--- - -\--- - - - -\# 13. OPC UA configuration - - +# 13. OPC UA configuration Example: - - ```yaml opcua: - enabled: true + enabled: true + application: + name: AquaCom Integration Gateway - application: + uri: urn:example:aquacom-gateway - name: AquaCom Integration Gateway + endpoint: - uri: urn:example:aquacom-gateway + bindAddress: 0.0.0.0 + port: 4840 + path: /aquacom - endpoint: + namespace: - bindAddress: 0.0.0.0 + uri: urn:example:aquacom - port: 4840 + rootName: AquaCom - path: /aquacom + dataAccess: + enabled: true + historicalAccess: - namespace: + enabled: true - uri: urn:example:aquacom + repository: local-history - rootName: AquaCom + maxValuesPerRead: 10000 + maxConcurrentReads: 10 + security: - dataAccess: + allowNone: false - enabled: true + certificate: gateway-opcua - - - historicalAccess: - - enabled: true - - repository: local-history - - maxValuesPerRead: 10000 - - maxConcurrentReads: 10 - - - - security: - - allowNone: false - - certificate: gateway-opcua - - trustMode: explicit + trustMode: explicit ``` +--- +# 14. `opcua.enabled` -\--- - - - -\# 14. `opcua.enabled` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `true` - +**Type:** boolean +**Default:** `true` Controls whether the OPC UA server is started. - - The gateway architecture shall permit AquaCom acquisition and historical storage to operate independently where practical if OPC UA is disabled. +--- +# 15. OPC UA application identity -\--- - - - -\# 15. OPC UA application identity - - - -\## 15.1 `opcua.application.name` - - +## 15.1 `opcua.application.name` Human-readable OPC UA application name. - - Example: - - ```yaml name: AquaCom Integration Gateway ``` +--- - -\--- - - - -\## 15.2 `opcua.application.uri` - - +## 15.2 `opcua.application.uri` Stable application URI. - - Example: - - ```yaml uri: urn:example:aquacom-gateway ``` - - The URI should remain stable for an installed gateway instance. +--- - -\--- - - - -\# 16. OPC UA endpoint - - +# 16. OPC UA endpoint Example: - - ```yaml endpoint: - bindAddress: 0.0.0.0 + bindAddress: 0.0.0.0 - port: 4840 + port: 4840 - path: /aquacom + path: /aquacom ``` +--- - -\--- - - - -\## 16.1 `opcua.endpoint.bindAddress` - - +## 16.1 `opcua.endpoint.bindAddress` Defines the interface on which the OPC UA endpoint listens. - - Production deployments should consider binding only to the required OT network interface. +--- +## 16.2 `opcua.endpoint.port` -\--- - - - -\## 16.2 `opcua.endpoint.port` - - - -\*\*Type:\*\* integer - - +**Type:** integer Example: - - ```yaml port: 4840 ``` - - Port conflict shall be detected during startup. +--- - -\--- - - - -\## 16.3 `opcua.endpoint.path` - - +## 16.3 `opcua.endpoint.path` Optional OPC UA endpoint path. - - Example: - - ```yaml path: /aquacom ``` +--- - -\--- - - - -\# 17. OPC UA namespace - - +# 17. OPC UA namespace Example: - - ```yaml namespace: - uri: urn:example:aquacom + uri: urn:example:aquacom - rootName: AquaCom + rootName: AquaCom ``` +--- - -\--- - - - -\## 17.1 `opcua.namespace.uri` - - +## 17.1 `opcua.namespace.uri` Stable namespace URI used for gateway-specific nodes. - - Changing this value may affect client configurations and shall therefore be treated as a compatibility-impacting change. +--- - -\--- - - - -\## 17.2 `opcua.namespace.rootName` - - +## 17.2 `opcua.namespace.rootName` Human-readable root object name. - - Changing this value shall not automatically alter stable NodeIds. +--- +# 18. OPC UA Data Access -\--- - - - -\# 18. OPC UA Data Access - - - -\## 18.1 `opcua.dataAccess.enabled` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `true` +## 18.1 `opcua.dataAccess.enabled` +**Type:** boolean +**Default:** `true` Enables exposure of configured current values. +--- - -\--- - - - -\# 19. OPC UA Historical Access - - +# 19. OPC UA Historical Access Example: - - ```yaml historicalAccess: - enabled: true + enabled: true - repository: local-history + repository: local-history - maxValuesPerRead: 10000 + maxValuesPerRead: 10000 - maxConcurrentReads: 10 + maxConcurrentReads: 10 ``` +--- +## 19.1 `opcua.historicalAccess.enabled` -\--- - - - -\## 19.1 `opcua.historicalAccess.enabled` - - - -\*\*Type:\*\* boolean - - +**Type:** boolean Enables OPC UA Historical Access. - - If enabled, a queryable history repository must be configured. +--- +## 19.2 `opcua.historicalAccess.repository` -\--- - - - -\## 19.2 `opcua.historicalAccess.repository` - - - -\*\*Type:\*\* storage repository identifier - -\*\*Required when Historical Access is enabled\*\* - +**Type:** storage repository identifier +**Required when Historical Access is enabled** Example: - - ```yaml repository: local-history ``` - - The referenced destination must implement the History Repository capability. - - A sink-only destination is invalid here. +--- +## 19.3 `opcua.historicalAccess.maxValuesPerRead` -\--- - - - -\## 19.3 `opcua.historicalAccess.maxValuesPerRead` - - - -\*\*Type:\*\* positive integer - - +**Type:** positive integer Limits the number of values returned in one historical response. - - Example: - - ```yaml maxValuesPerRead: 10000 ``` - - Larger responses shall use OPC UA continuation where required. +--- +## 19.4 `opcua.historicalAccess.maxConcurrentReads` -\--- - - - -\## 19.4 `opcua.historicalAccess.maxConcurrentReads` - - - -\*\*Type:\*\* positive integer - - +**Type:** positive integer Limits concurrent historical requests. - - This protects the gateway and historical repository against excessive resource usage. +--- - -\--- - - - -\# 20. OPC UA security configuration - - +# 20. OPC UA security configuration Example: - - ```yaml security: - allowNone: false + allowNone: false - certificate: gateway-opcua + certificate: gateway-opcua - trustMode: explicit + trustMode: explicit + userAuthentication: - - userAuthentication: - - enabled: false + enabled: false ``` +--- +## 20.1 `opcua.security.allowNone` -\--- - - - -\## 20.1 `opcua.security.allowNone` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `false` - +**Type:** boolean +**Default:** `false` Controls whether an unsecured OPC UA endpoint is permitted. - - Production configuration should normally use: - - ```yaml allowNone: false ``` +--- - -\--- - - - -\## 20.2 `opcua.security.certificate` - - +## 20.2 `opcua.security.certificate` Reference to an externally configured certificate identity. - - Private keys shall not be embedded directly in configuration committed to source control. +--- - -\--- - - - -\## 20.3 `opcua.security.trustMode` - - +## 20.3 `opcua.security.trustMode` Possible values may include: - - ```text explicit @@ -1538,206 +972,128 @@ managed ``` - - `explicit` means client certificates must be explicitly trusted. - - The exact supported trust modes depend upon the selected OPC UA implementation. +--- - -\--- - - - -\# 21. Historical subsystem configuration - - +# 21. Historical subsystem configuration Example: - - ```yaml history: - enabled: true + enabled: true + liveHistorization: + enabled: true - liveHistorization: + deduplication: - enabled: true + enabled: true + conflictPolicy: reject + synchronization: - deduplication: + enabled: true - enabled: true + resumeAfterRestart: true - conflictPolicy: reject + backfill: + enabled: true + priority: low - synchronization: + maxRequestsPerSecond: 2 - enabled: true + spool: - resumeAfterRestart: true + enabled: true + maxSize: 20GB + maxAge: 30d - backfill: - - enabled: true - - priority: low - - maxRequestsPerSecond: 2 - - - - spool: - - enabled: true - - maxSize: 20GB - - maxAge: 30d - - overflowPolicy: stop-history + overflowPolicy: stop-history ``` +--- +# 22. `history.enabled` -\--- - - - -\# 22. `history.enabled` - - - -\*\*Type:\*\* boolean - - +**Type:** boolean Enables the gateway historical subsystem. - - This includes historical pipeline functionality. - - Individual stations and points may still enable or disable history independently. +--- +# 23. Live historization -\--- - - - -\# 23. Live historization - - - -\## 23.1 `history.liveHistorization.enabled` - - - -\*\*Type:\*\* boolean - +## 23.1 `history.liveHistorization.enabled` +**Type:** boolean Determines whether live current values may also be submitted to the historical pipeline. - - Example: - - ```yaml liveHistorization: - enabled: true + enabled: true ``` - - This setting does not replace per-point history configuration. - - Both global and point-level configuration shall permit history before a live value is historized. +--- - -\--- - - - -\# 24. Deduplication - - +# 24. Deduplication Example: - - ```yaml deduplication: - enabled: true + enabled: true - conflictPolicy: reject + conflictPolicy: reject ``` +--- +## 24.1 `history.deduplication.enabled` -\--- - - - -\## 24.1 `history.deduplication.enabled` - - - -\*\*Type:\*\* boolean - -\*\*Recommended:\*\* `true` - +**Type:** boolean +**Recommended:** `true` Historical processing should normally be idempotent. +--- - -\--- - - - -\## 24.2 `history.deduplication.conflictPolicy` - - +## 24.2 `history.deduplication.conflictPolicy` Defines behavior when two different values have the same logical historical identity. - - Allowed conceptual values: - - ```text reject @@ -1750,144 +1106,88 @@ version ``` - - Recommended default: - - ```yaml conflictPolicy: reject ``` - - A conflict shall generate diagnostics. +--- - -\--- - - - -\# 25. History synchronization - - +# 25. History synchronization Example: - - ```yaml synchronization: - enabled: true + enabled: true - resumeAfterRestart: true + resumeAfterRestart: true ``` +--- - -\--- - - - -\## 25.1 `history.synchronization.enabled` - - +## 25.1 `history.synchronization.enabled` Enables synchronization-state management. +--- +## 25.2 `history.synchronization.resumeAfterRestart` -\--- - - - -\## 25.2 `history.synchronization.resumeAfterRestart` - - - -\*\*Type:\*\* boolean - -\*\*Recommended:\*\* `true` - +**Type:** boolean +**Recommended:** `true` If enabled, the gateway restores historical synchronization watermarks and continues incomplete synchronization after restart. +--- - -\--- - - - -\# 26. Backfill configuration - - +# 26. Backfill configuration Example: - - ```yaml backfill: - enabled: true + enabled: true - priority: low + priority: low - maxRequestsPerSecond: 2 + maxRequestsPerSecond: 2 ``` +--- - -\--- - - - -\## 26.1 `history.backfill.enabled` - - +## 26.1 `history.backfill.enabled` Enables retrieval of missing AquaCom history following communication interruption. - - Actual availability depends upon: +- AquaCom protocol support; +- RTU capabilities; -\- AquaCom protocol support; - -\- RTU capabilities; - -\- verified station configuration. - - - -\--- - - - -\## 26.2 `history.backfill.priority` +- verified station configuration. +--- +## 26.2 `history.backfill.priority` Defines scheduling priority relative to other AquaCom operations. - - Supported values may include: - - ```text low @@ -1896,162 +1196,94 @@ normal ``` - - Recommended: - - ```yaml priority: low ``` - - Live acquisition shall normally take precedence. +--- - -\--- - - - -\## 26.3 `history.backfill.maxRequestsPerSecond` - - +## 26.3 `history.backfill.maxRequestsPerSecond` Optional history-retrieval rate limit. - - Used to prevent large backfill operations from consuming all available station communication capacity. +--- - -\--- - - - -\# 27. Durable spool configuration - - +# 27. Durable spool configuration Example: - - ```yaml spool: - enabled: true + enabled: true - path: ./data/spool + path: ./data/spool - maxSize: 20GB + maxSize: 20GB - maxAge: 30d + maxAge: 30d - overflowPolicy: stop-history + overflowPolicy: stop-history ``` +--- +## 27.1 `history.spool.enabled` -\--- - - - -\## 27.1 `history.spool.enabled` - - - -\*\*Recommended:\*\* `true` - - +**Recommended:** `true` Enables persistent delivery buffering. +--- - -\--- - - - -\## 27.2 `history.spool.path` - - +## 27.2 `history.spool.path` Location of persistent spool data where the selected implementation requires an explicit path. - - The path must be writable. +--- - -\--- - - - -\## 27.3 `history.spool.maxSize` - - +## 27.3 `history.spool.maxSize` Maximum spool size. - - Example: - - ```yaml maxSize: 20GB ``` - - The gateway shall provide diagnostics before reaching this limit. +--- - -\--- - - - -\## 27.4 `history.spool.maxAge` - - +## 27.4 `history.spool.maxAge` Optional maximum retention time for pending spool records. - - Care shall be taken before enabling automatic age-based deletion. +--- - -\--- - - - -\## 27.5 `history.spool.overflowPolicy` - - +## 27.5 `history.spool.overflowPolicy` Defines behavior if the spool cannot accept additional data. - - Conceptual values: - - ```text stop-history @@ -2064,166 +1296,108 @@ fail-gateway ``` - - Recommended production default: - - ```yaml overflowPolicy: stop-history ``` +Historical information shall never be dropped silently. Policies that do discard data (`drop-oldest`, and age-based deletion via `maxAge`) shall emit a diagnostic event per discarded record (or per discarded batch, for bounded diagnostics) so the loss is observable and auditable. `reject-new` and `stop-history` remain the recommended policies because they never lose already-accepted records. +--- -Dropping historical information silently shall not be permitted. - - - -\--- - - - -\# 28. Storage configuration - - +# 28. Storage configuration Example: - - ```yaml storage: - destinations: + destinations: + - id: local-history + type: repository - - id: local-history + driver: example-local-db - type: repository + required: true - driver: example-local-db + capabilities: - required: true + write: true + query: true + connection: - capabilities: + path: ./data/history - write: true + - id: enterprise-historian - query: true + type: sink + driver: example-historian + required: true - connection: + connection: - path: ./data/history + endpoint: historian.example.local + credentialSecret: HISTORIAN_CREDENTIAL + - id: archive - - id: enterprise-historian + type: sink - type: sink + driver: ndjson - driver: example-historian + required: false - required: true + connection: - - - connection: - - endpoint: historian.example.local - - credentialSecret: HISTORIAN\_CREDENTIAL - - - - - id: archive - - type: sink - - driver: ndjson - - required: false - - - - connection: - - directory: ./archive + directory: ./archive ``` +--- +# 29. Storage destination identifiers -\--- - - - -\# 29. Storage destination identifiers - - - -\## 29.1 `storage.destinations\[].id` - - - -\*\*Type:\*\* string - -\*\*Required:\*\* yes +## 29.1 `storage.destinations[].id` +**Type:** string +**Required:** yes Stable identifier for the storage destination. - - Example: - - ```yaml id: local-history ``` - - Must be unique. - - Other configuration sections reference destinations using this identifier. +--- +# 30. Storage destination type -\--- - - - -\# 30. Storage destination type - - - -\## 30.1 `storage.destinations\[].type` - - +## 30.1 `storage.destinations[].type` Defines the architectural role. - - Conceptual values: - - ```text repository @@ -2234,12 +1408,8 @@ repository-and-sink ``` - - Meaning: - - | Type | Query | Write | |---|---|---| @@ -2250,298 +1420,194 @@ Meaning: | `repository-and-sink` | yes | yes | - - The exact schema may represent capabilities instead of this combined enumeration. +--- +# 31. Storage driver -\--- - - - -\# 31. Storage driver - - - -\## 31.1 `storage.destinations\[].driver` - - +## 31.1 `storage.destinations[].driver` Defines the implementation adapter. - - Example: - - ```yaml driver: example-local-db ``` - - The value is implementation-specific. - - Business logic shall not depend directly on this value. +--- +# 32. Required and optional destinations -\--- - - - -\# 32. Required and optional destinations - - - -\## 32.1 `storage.destinations\[].required` - - - -\*\*Type:\*\* boolean - +## 32.1 `storage.destinations[].required` +**Type:** boolean Defines whether successful delivery to the destination is required before a spool record may be considered fully delivered. - - Example: - - ```yaml required: true ``` - - If: - - ```yaml required: false ``` - - failure shall generate diagnostics but shall not indefinitely block cleanup of records that have reached all required destinations. +--- - -\--- - - - -\# 33. Storage capabilities - - +# 33. Storage capabilities Example: - - ```yaml capabilities: - write: true + write: true - query: true + query: true ``` - - This may be validated against the selected driver. - - A repository used for OPC UA Historical Access must have: - - ```yaml query: true ``` +--- - -\--- - - - -\# 34. Storage retry configuration - - +# 34. Storage retry configuration A storage destination may contain: - - ```yaml retry: - enabled: true + enabled: true - initialDelay: 2s + initialDelay: 2s - maximumDelay: 5m + maximumDelay: 5m - multiplier: 2 + multiplier: 2 ``` - - Retry policy shall be independent for each destination. - - Storage retries shall not block AquaCom live acquisition. +--- - -\--- - - - -\# 35. Station configuration - - +# 35. Station configuration Example: - - ```yaml stations: + - id: PS001 + displayName: Wastewater Pump Station 001 - - id: PS001 + enabled: true - displayName: Wastewater Pump Station 001 + timezone: Europe/Stockholm - enabled: true + transport: - timezone: Europe/Stockholm + type: tcp + host: 192.0.2.10 + port: 1234 - transport: + aquacom: - type: tcp + address: 101 - host: 192.0.2.10 + profile: default - port: 1234 + acquisition: + live: + enabled: true - aquacom: + pollingInterval: 5s - address: 101 + requestTimeout: 3s - profile: default + retryCount: 2 + history: + enabled: true - acquisition: + backfillEnabled: true - live: + pointDefinitions: - enabled: true + sourceType: atf - pollingInterval: 5s - - requestTimeout: 3s - - retryCount: 2 - - - - history: - - enabled: true - - backfillEnabled: true - - - - pointDefinitions: - - sourceType: atf - - path: ./stations/PS001.atf + path: ./stations/PS001.atf ``` +--- +# 36. `stations[].id` -\--- - - - -\# 36. `stations\[].id` - - - -\*\*Type:\*\* string - -\*\*Required:\*\* yes - +**Type:** string +**Required:** yes Stable gateway identifier for the station. - - Example: - - ```yaml id: PS001 ``` - - Requirements: +- unique within the gateway; +- stable across restart; -\- unique within the gateway; - -\- stable across restart; - -\- stable across IP-address changes; - -\- stable across display-name changes. - +- stable across IP-address changes; +- stable across display-name changes. The station ID may form part of: - - ```text PointId @@ -2556,144 +1622,86 @@ diagnostics ``` +--- - -\--- - - - -\# 37. `stations\[].displayName` - - +# 37. `stations[].displayName` Human-readable station name. - - Example: - - ```yaml displayName: Wastewater Pump Station 001 ``` - - Changing this shall not change the stable station identity. +--- +# 38. `stations[].enabled` -\--- - - - -\# 38. `stations\[].enabled` - - - -\*\*Type:\*\* boolean - -\*\*Default:\*\* `true` - +**Type:** boolean +**Default:** `true` Controls whether the gateway actively communicates with the station. - - A disabled station configuration may remain visible diagnostically. +--- +# 39. Station timezone -\--- - - - -\# 39. Station timezone - - - -\## 39.1 `stations\[].timezone` - - +## 39.1 `stations[].timezone` Optional station-specific timezone. - - If omitted, the gateway timezone is inherited. - - Example: - - ```yaml timezone: Europe/Stockholm ``` - - This configuration is important where AquaCom timestamps lack explicit timezone information. +--- - -\--- - - - -\# 40. Transport configuration - - +# 40. Transport configuration Example: - - ```yaml transport: - type: tcp + type: tcp - host: 192.0.2.10 + host: 192.0.2.10 - port: 1234 + port: 1234 - connectTimeout: 5s + connectTimeout: 5s - idleTimeout: 30s + idleTimeout: 30s ``` - - Transport-specific fields are conditional on `type`. +--- - -\--- - - - -\# 41. `transport.type` - - +# 41. `transport.type` Defines the AquaCom transport implementation. - - Potential supported values may include: - - ```text tcp @@ -2704,442 +1712,262 @@ serial-over-ip ``` - - Only implemented and verified transport types shall be accepted. +--- - -\--- - - - -\# 42. TCP transport - - +# 42. TCP transport Example: - - ```yaml transport: - type: tcp + type: tcp - host: 192.0.2.10 + host: 192.0.2.10 - port: 1234 + port: 1234 - connectTimeout: 5s + connectTimeout: 5s ``` +--- - -\--- - - - -\## 42.1 `transport.host` - - +## 42.1 `transport.host` Hostname or IP address. - - This value is not the station's stable identity. +--- - -\--- - - - -\## 42.2 `transport.port` - - +## 42.2 `transport.port` TCP port used for AquaCom communication. - - The default shall not be invented if it is not formally defined for the target equipment. - - Where no verified standard value exists, explicit configuration shall be required. +--- - -\--- - - - -\## 42.3 `transport.connectTimeout` - - +## 42.3 `transport.connectTimeout` Maximum connection establishment time. +--- - -\--- - - - -\# 43. Serial transport - - +# 43. Serial transport Conceptual example: - - ```yaml transport: - type: serial + type: serial + port: /dev/ttyS0 + baudRate: 9600 - port: /dev/ttyS0 + dataBits: 8 - baudRate: 9600 + stopBits: 1 - dataBits: 8 - - stopBits: 1 - - parity: none + parity: none ``` - - Actual values shall be determined from validated AquaCom documentation and target-device requirements. - - Codex shall not insert assumed AquaCom serial defaults without evidence. +--- - -\--- - - - -\# 44. AquaCom station protocol configuration - - +# 44. AquaCom station protocol configuration Example: - - ```yaml aquacom: - address: 101 + address: 101 - profile: default + profile: default ``` - - This section contains AquaCom-specific station configuration. - - Only verified protocol concepts shall be introduced here. +--- - -\--- - - - -\## 44.1 `aquacom.address` - - +## 44.1 `aquacom.address` Station address or AquaCom identifier required by the verified protocol implementation. - - The representation is protocol-dependent. +It shall not be assumed to be identical to `stations[].id`. +--- -It shall not be assumed to be identical to `stations\[].id`. - - - -\--- - - - -\## 44.2 `aquacom.profile` - - +## 44.2 `aquacom.profile` Optional reference to an AquaCom protocol profile. - - Profiles may be useful where different controller generations or configurations require different verified protocol behaviors. - - Example: - - ```yaml profile: app700 ``` - - A profile shall never enable guessed protocol behavior. +--- - -\--- - - - -\# 45. Acquisition configuration - - +# 45. Acquisition configuration Each station contains: - - ```yaml acquisition: - live: + live: - ... + ... + history: - - history: - - ... + ... ``` - - Live acquisition and historical acquisition shall be independently configurable. +--- - -\--- - - - -\# 46. Live acquisition - - +# 46. Live acquisition Example: - - ```yaml live: - enabled: true + enabled: true - pollingInterval: 5s + pollingInterval: 5s - requestTimeout: 3s + requestTimeout: 3s - retryCount: 2 + retryCount: 2 ``` +--- - -\--- - - - -\## 46.1 `acquisition.live.enabled` - - +## 46.1 `acquisition.live.enabled` Enables current-value acquisition for the station. +--- - -\--- - - - -\## 46.2 `acquisition.live.pollingInterval` - - +## 46.2 `acquisition.live.pollingInterval` Defines the normal polling period. - - The implementation shall validate unreasonable values. - - Example: - - ```yaml pollingInterval: 5s ``` +--- - -\--- - - - -\## 46.3 `acquisition.live.requestTimeout` - - +## 46.3 `acquisition.live.requestTimeout` Maximum allowed time for an AquaCom live transaction before timeout handling. +--- - -\--- - - - -\## 46.4 `acquisition.live.retryCount` - - +## 46.4 `acquisition.live.retryCount` Number of immediate or near-immediate retries permitted before normal communication-failure logic takes over. - - Retries shall not create an uncontrolled retry storm. +--- - -\--- - - - -\# 47. Station historical acquisition - - +# 47. Station historical acquisition Example: - - ```yaml history: - enabled: true + enabled: true - backfillEnabled: true + backfillEnabled: true - maxBackfillAge: 7d + maxBackfillAge: 7d ``` +--- - -\--- - - - -\## 47.1 `acquisition.history.enabled` - - +## 47.1 `acquisition.history.enabled` Enables AquaCom historical retrieval for the station. - - This requires verified support for the station/controller type. +--- - -\--- - - - -\## 47.2 `acquisition.history.backfillEnabled` - - +## 47.2 `acquisition.history.backfillEnabled` Enables automatic recovery following communication interruption. +--- - -\--- - - - -\## 47.3 `acquisition.history.maxBackfillAge` - - +## 47.3 `acquisition.history.maxBackfillAge` Optional limit on how far back the gateway will automatically request history. - - Example: - - ```yaml maxBackfillAge: 7d ``` - - This protects against unexpected very large historical queries. - - A value beyond this range may require an explicit administrative operation. +--- - -\--- - - - -\# 48. Point definition source - - +# 48. Point definition source Points may originate from sources such as: - - ```text ATF @@ -3152,36 +1980,24 @@ future external configuration service ``` - - Example: - - ```yaml pointDefinitions: - sourceType: atf + sourceType: atf - path: ./stations/PS001.atf + path: ./stations/PS001.atf ``` +--- - -\--- - - - -\## 48.1 `pointDefinitions.sourceType` - - +## 48.1 `pointDefinitions.sourceType` Potential values: - - ```text atf @@ -3192,238 +2008,142 @@ file ``` - - Only implemented source types shall be accepted. +--- - -\--- - - - -\## 48.2 `pointDefinitions.path` - - +## 48.2 `pointDefinitions.path` Path to an external point-definition file. - - File changes shall be validated before activation. +--- - -\--- - - - -\# 49. Inline point definitions - - +# 49. Inline point definitions Conceptual example: - - ```yaml points: + - id: LEVEL + displayName: Wet well level - - id: LEVEL + description: Measured wet well level - displayName: Wet well level + source: - description: Measured wet well level + reference: ... + dataType: float + engineeringUnit: m - source: + scaling: - reference: ... + enabled: false + live: + enabled: true - dataType: float + history: - engineeringUnit: m + enabled: true + opcua: + enabled: true - scaling: + historizing: true - enabled: false + storage: - - - live: - - enabled: true - - - - history: - - enabled: true - - - - opcua: - - enabled: true - - historizing: true - - - - storage: - - enabled: true + enabled: true ``` - - The exact AquaCom source-reference syntax shall only be defined when supported by verified protocol or ATF semantics. +--- +# 50. Point identifier -\--- - - - -\# 50. Point identifier - - - -\## 50.1 `points\[].id` - - - -\*\*Required:\*\* yes - +## 50.1 `points[].id` +**Required:** yes Stable point identifier within the station. - - Example: - - ```yaml id: LEVEL ``` - - Logical identity may become: - - ```text station:PS001/point:LEVEL ``` - - Changing `displayName` shall not change this identifier. +--- +# 51. Point display name -\--- - - - -\# 51. Point display name - - - -\## 51.1 `points\[].displayName` - - +## 51.1 `points[].displayName` Human-readable presentation name. - - Example: - - ```yaml displayName: Wet well level ``` - - May be changed without creating a new logical process point. +--- - -\--- - - - -\# 52. Point source reference - - +# 52. Point source reference Example: - - ```yaml source: - reference: ... + reference: ... ``` - - Defines how the point maps to AquaCom. - - The content is intentionally unspecified until AquaCom mapping rules are verified. - - Codex shall not invent source-reference syntax. +--- +# 53. Point data type -\--- - - - -\# 53. Point data type - - - -\## 53.1 `points\[].dataType` - - +## 53.1 `points[].dataType` Logical canonical data type. - - Possible values may include: - - ```text boolean @@ -3440,326 +2160,207 @@ string ``` - - The final supported set shall be defined by the canonical data model. +--- +# 54. Engineering unit -\--- - - - -\# 54. Engineering unit - - - -\## 54.1 `points\[].engineeringUnit` - - +## 54.1 `points[].engineeringUnit` Optional engineering unit. - - Examples: - - ```yaml engineeringUnit: m ``` - - ```yaml engineeringUnit: l/s ``` - - ```yaml engineeringUnit: °C ``` - - Engineering units should use a controlled representation where feasible. +--- - -\--- - - - -\# 55. Point scaling - - +# 55. Point scaling Example: - - ```yaml scaling: - enabled: true + enabled: true - multiplier: 0.1 + multiplier: 0.1 - offset: 0 + offset: 0 ``` - - Scaling is applied after verified AquaCom decoding and before creation of the canonical value. - - The raw value may optionally be retained diagnostically. +--- - -\--- - - - -\# 56. Point live configuration - - +# 56. Point live configuration Example: - - ```yaml live: - enabled: true + enabled: true ``` - - Controls whether the point participates in current-state processing. +--- - -\--- - - - -\# 57. Point history configuration - - +# 57. Point history configuration Example: - - ```yaml history: - enabled: true + enabled: true ``` - - Controls whether historical information is processed for the point. +--- - -\--- - - - -\# 58. Point OPC UA configuration - - +# 58. Point OPC UA configuration Example: - - ```yaml opcua: - enabled: true + enabled: true - historizing: true + historizing: true ``` +--- - -\--- - - - -\## 58.1 `points\[].opcua.enabled` - - +## 58.1 `points[].opcua.enabled` Controls whether the point is exposed through OPC UA. +--- - -\--- - - - -\## 58.2 `points\[].opcua.historizing` - - +## 58.2 `points[].opcua.historizing` Controls whether the OPC UA Variable advertises historical capability. - - If enabled: +- point history must be enabled; +- OPC UA Historical Access must be enabled; -\- point history must be enabled; - -\- OPC UA Historical Access must be enabled; - -\- a queryable repository must be configured. - - - -\--- - - - -\# 59. Explicit NodeId override +- a queryable repository must be configured. +--- +# 59. Explicit NodeId override Optional example: - - ```yaml opcua: - enabled: true + enabled: true - nodeId: station/PS001/point/LEVEL + nodeId: station/PS001/point/LEVEL ``` - - Normally the gateway should generate deterministic NodeIds from stable station and point identifiers. - - Manual override should only be required for compatibility or migration scenarios. - - Duplicate NodeIds shall be rejected. +--- - -\--- - - - -\# 60. Point storage configuration - - +# 60. Point storage configuration Example: - - ```yaml storage: - enabled: true + enabled: true + destinations: + - local-history - destinations: - - - local-history - - - enterprise-historian + - enterprise-historian ``` +If `destinations` is omitted, the **default routing policy** applies. The default routing shall be deterministic and explicit: +- if exactly one destination is configured, points route to it; +- if more than one destination is configured, validation requires either a `history.defaultDestinations` list or per-point `destinations` — an implicit "deliver to everything" or "deliver to nothing" default is not permitted, because both silently change delivery behavior when destinations are added or removed. -If `destinations` is omitted, a global/default routing policy may be used. - - - -\--- - - - -\# 61. Storage routing +The exact form of the global default is implementation-defined but shall be validated; ambiguity shall fail configuration validation rather than resolve silently. +--- +# 61. Storage routing The system shall support routing history to selected destinations. - - Example: - - ```yaml storage: - enabled: true + enabled: true + destinations: + - local-history - destinations: - - - local-history - - - enterprise-historian + - enterprise-historian ``` - - Different points may therefore use different storage policies if required. +--- - -\--- - - - -\# 62. History source behavior - - +# 62. History source behavior A point may theoretically receive historical data from: - - ```text live historization @@ -3772,132 +2373,78 @@ future import ``` - - The canonical `AcquisitionSource` shall preserve the origin. - - Storage routing shall not normally depend upon raw AquaCom message types. +--- - -\--- - - - -\# 63. Timestamp configuration - - +# 63. Timestamp configuration Station timestamp behavior may include: - - ```yaml timestamps: - sourceTimezone: Europe/Stockholm + sourceTimezone: Europe/Stockholm - maximumFutureDeviation: 5m + maximumFutureDeviation: 5m - maximumPastDeviation: 365d + maximumPastDeviation: 365d - ambiguousDstPolicy: reject + ambiguousDstPolicy: reject ``` +--- - -\--- - - - -\## 63.1 `timestamps.sourceTimezone` - - +## 63.1 `timestamps.sourceTimezone` Used to interpret device timestamps without explicit timezone information. - - Defaults to: - - ```text station timezone ``` - - then: - - ```text gateway timezone ``` - - if no station timezone exists. +--- - -\--- - - - -\## 63.2 `timestamps.maximumFutureDeviation` - - +## 63.2 `timestamps.maximumFutureDeviation` Maximum permitted source timestamp ahead of gateway time before it is flagged. +--- - -\--- - - - -\## 63.3 `timestamps.maximumPastDeviation` - - +## 63.3 `timestamps.maximumPastDeviation` Optional sanity threshold for implausibly old values. - - Historical backfill behavior may legitimately require values much older than normal live values. - - The implementation shall distinguish history from erroneous live timestamps. +--- - -\--- - - - -\## 63.4 `timestamps.ambiguousDstPolicy` - - +## 63.4 `timestamps.ambiguousDstPolicy` Defines behavior when a source local timestamp is ambiguous during daylight-saving transition. - - Possible conceptual values: - - ```text reject @@ -3910,500 +2457,328 @@ mark-uncertain ``` - - Recommended behavior is either: - - ```text mark-uncertain ``` - - or: - - ```text reject ``` - - depending on operational requirements. +--- - -\--- - - - -\# 64. Communication health configuration - - +# 64. Communication health configuration A station may define thresholds such as: - - ```yaml health: - degradedAfterFailures: 2 + degradedAfterFailures: 2 - offlineAfterFailures: 5 + offlineAfterFailures: 5 - staleAfter: 30s + staleAfter: 30s ``` +--- - -\--- - - - -\## 64.1 `health.degradedAfterFailures` - - +## 64.1 `health.degradedAfterFailures` Consecutive failed transactions before communication changes to a degraded state. +--- - -\--- - - - -\## 64.2 `health.offlineAfterFailures` - - +## 64.2 `health.offlineAfterFailures` Consecutive failed transactions before the station is considered offline. - - This value must be greater than or equal to the degraded threshold. +--- - -\--- - - - -\## 64.3 `health.staleAfter` - - +## 64.3 `health.staleAfter` Maximum age of a current value before quality may be changed to stale/uncertain/bad according to configured behavior. +--- - -\--- - - - -\# 65. Retry behavior - - +# 65. Retry behavior Transport, AquaCom transactions and storage destinations may each have separate retry policies. - - These retries shall not be combined into an uncontrolled nested retry loop. - - Retry configuration should define: - - ```yaml retry: - initialDelay: 1s + initialDelay: 1s - maximumDelay: 1m + maximumDelay: 1m - multiplier: 2 + multiplier: 2 - jitter: true + jitter: true ``` - - Exact supported fields are implementation-specific. +--- - -\--- - - - -\# 66. Configuration inheritance - - +# 66. Configuration inheritance Where practical, the gateway may support default configuration inheritance. - - Example: - - ```yaml defaults: - station: + station: - acquisition: + acquisition: - live: + live: - pollingInterval: 5s + pollingInterval: 5s + health: + degradedAfterFailures: 2 - health: - - degradedAfterFailures: 2 - - offlineAfterFailures: 5 + offlineAfterFailures: 5 ``` - - Then a station may override: - - ```yaml stations: - - id: PS001 + - id: PS001 + acquisition: + live: - acquisition: - - live: - - pollingInterval: 2s + pollingInterval: 2s ``` - - If implemented, inheritance rules must be deterministic and documented. - - Implicit multi-level inheritance should be avoided if it makes effective configuration difficult to understand. +--- - -\--- - - - -\# 67. Configuration validation rules - - +# 67. Configuration validation rules The configuration validator shall detect at least: - - ```text unsupported configVersion - - duplicate gateway/station/point/storage identifiers - - duplicate OPC UA NodeIds - - missing referenced storage destinations - - Historical Access enabled without queryable repository - - point historizing enabled without history capability - - unsupported transport type - - unsupported storage driver - - invalid timezone - - invalid durations - - invalid numeric limits - - invalid security configuration - - required secrets not available - - nonexistent required files - - invalid ATF or point-definition references - - history enabled for unsupported station profile - - configuration contradictions ``` +--- - -\--- - - - -\# 68. Cross-field validation - - +# 68. Cross-field validation Examples of required relationships follow. - - If: - - ```yaml opcua: - historicalAccess: + historicalAccess: - enabled: true + enabled: true ``` - - then: - - ```text a queryable history repository must exist. ``` - - If: - - ```yaml point: - opcua: + opcua: - historizing: true + historizing: true ``` - - then: - - ```text point history must be enabled. ``` - - If: - - ```yaml history: - enabled: false + enabled: false ``` - - then point-level historical acquisition shall not become active. - - If: - - ```yaml station: - acquisition: + acquisition: - history: + history: - enabled: true + enabled: true ``` - - but the configured AquaCom profile has no verified historical implementation, validation shall fail or the station history feature shall be explicitly marked unavailable. - - It shall not silently use guessed AquaCom behavior. +--- - -\--- - - - -\# 69. Configuration activation - - +# 69. Configuration activation Configuration shall follow: - - ```text Load - ↓ + ↓ Parse - ↓ + ↓ Schema validation - ↓ + ↓ Semantic validation - ↓ + ↓ Dependency validation - ↓ + ↓ Activation ``` - - Invalid configuration shall not partially activate. - - Where dynamic reload is supported: - - ```text Current valid configuration - │ + │ - ▼ + ▼ - New configuration + New configuration - │ + │ - ▼ + ▼ - Validate + Validate - ┌───────┴────────┐ + ┌───────┴────────┐ - │ │ + │ │ - valid invalid + valid invalid - │ │ + │ │ - ▼ ▼ + ▼ ▼ - activate retain existing + activate retain existing ``` +--- - -\--- - - - -\# 70. Dynamic configuration - - +# 70. Dynamic configuration Changes may be classified as: - - ```text runtime-safe @@ -4414,12 +2789,8 @@ full-restart ``` - - Examples that may eventually be runtime-safe: - - ```text logging level @@ -4432,12 +2803,8 @@ storage retry intervals ``` - - Changes likely requiring controlled restart or resource reconstruction: - - ```text OPC UA application certificate @@ -4452,24 +2819,14 @@ persistent storage path ``` - - Exact behavior shall be defined by the implementation. +--- - -\--- - - - -\# 71. Configuration compatibility - - +# 71. Configuration compatibility The gateway shall distinguish: - - ```text configuration schema compatibility @@ -4482,554 +2839,410 @@ protocol profile compatibility ``` - - A configuration schema change shall not automatically require an OPC UA namespace change. +--- - -\--- - - - -\# 72. Example minimal configuration - - +# 72. Example minimal configuration ```yaml configVersion: 1 - - gateway: - id: aquacom-gateway-01 - - timezone: Europe/Stockholm - + id: aquacom-gateway-01 + timezone: Europe/Stockholm logging: - level: information - - + level: information diagnostics: - enabled: true - - + enabled: true opcua: - enabled: true + enabled: true + application: + name: AquaCom Integration Gateway - application: + uri: urn:example:aquacom-gateway - name: AquaCom Integration Gateway + endpoint: - uri: urn:example:aquacom-gateway + bindAddress: 0.0.0.0 + port: 4840 + namespace: - endpoint: + uri: urn:example:aquacom - bindAddress: 0.0.0.0 + dataAccess: - port: 4840 + enabled: true + historicalAccess: + enabled: false - namespace: - - uri: urn:example:aquacom - - - - dataAccess: - - enabled: true - - - - historicalAccess: - - enabled: false - - - - security: - - allowNone: false - - certificate: gateway-opcua + security: + allowNone: false + certificate: gateway-opcua history: - enabled: false - - + enabled: false storage: - destinations: \[] - - + destinations: [] stations: + - id: PS001 + enabled: true - - id: PS001 + transport: - enabled: true + type: tcp + host: 192.0.2.10 + port: 1234 - transport: + aquacom: - type: tcp + address: 101 - host: 192.0.2.10 + acquisition: - port: 1234 + live: + enabled: true + pollingInterval: 5s - aquacom: + pointDefinitions: - address: 101 + sourceType: atf - - - acquisition: - - live: - - enabled: true - - pollingInterval: 5s - - - - pointDefinitions: - - sourceType: atf - - path: ./stations/PS001.atf + path: ./stations/PS001.atf ``` - - This configuration represents: - - ```text AquaCom live acquisition - ↓ + ↓ OPC UA current values ``` - - without historical functionality. +--- - -\--- - - - -\# 73. Example configuration with history - - +# 73. Example configuration with history ```yaml configVersion: 1 - - gateway: - id: aquacom-gateway-01 - - timezone: Europe/Stockholm - + id: aquacom-gateway-01 + timezone: Europe/Stockholm logging: - level: information - - + level: information diagnostics: - enabled: true - - + enabled: true opcua: - enabled: true + enabled: true + application: + name: AquaCom Integration Gateway - application: + uri: urn:example:aquacom-gateway - name: AquaCom Integration Gateway + endpoint: - uri: urn:example:aquacom-gateway + bindAddress: 0.0.0.0 + port: 4840 + namespace: - endpoint: + uri: urn:example:aquacom - bindAddress: 0.0.0.0 + dataAccess: - port: 4840 + enabled: true + historicalAccess: + enabled: true - namespace: + repository: local-history - uri: urn:example:aquacom + maxValuesPerRead: 10000 + security: + allowNone: false - dataAccess: - - enabled: true - - - - historicalAccess: - - enabled: true - - repository: local-history - - maxValuesPerRead: 10000 - - - - security: - - allowNone: false - - certificate: gateway-opcua - - + certificate: gateway-opcua history: - enabled: true + enabled: true + liveHistorization: + enabled: true - liveHistorization: + deduplication: - enabled: true + enabled: true + conflictPolicy: reject + synchronization: - deduplication: + enabled: true - enabled: true + resumeAfterRestart: true - conflictPolicy: reject + backfill: + enabled: true + priority: low - synchronization: + maxRequestsPerSecond: 2 - enabled: true + spool: - resumeAfterRestart: true + enabled: true + path: ./data/spool + maxSize: 20GB - backfill: - - enabled: true - - priority: low - - maxRequestsPerSecond: 2 - - - - spool: - - enabled: true - - path: ./data/spool - - maxSize: 20GB - - overflowPolicy: stop-history - - + overflowPolicy: stop-history storage: - destinations: + destinations: + - id: local-history + type: repository-and-sink - - id: local-history + driver: local-history - type: repository-and-sink + required: true - driver: local-history + capabilities: - required: true + write: true + query: true + connection: - capabilities: + path: ./data/history - write: true + - id: central-historian - query: true + type: sink + driver: enterprise-historian + required: true - connection: + capabilities: - path: ./data/history + write: true + query: false + connection: - - id: central-historian + endpoint: historian.example.local - type: sink + credentialSecret: CENTRAL_HISTORIAN_CREDENTIAL - driver: enterprise-historian + - id: archive - required: true + type: sink + driver: ndjson + required: false - capabilities: + capabilities: - write: true + write: true - query: false - - - - connection: - - endpoint: historian.example.local - - credentialSecret: CENTRAL\_HISTORIAN\_CREDENTIAL - - - - - id: archive - - type: sink - - driver: ndjson - - required: false - - - - capabilities: - - write: true - - query: false - - - - connection: - - directory: ./archive + query: false + connection: + directory: ./archive stations: + - id: PS001 + displayName: Wastewater Pump Station 001 - - id: PS001 + enabled: true - displayName: Wastewater Pump Station 001 + timezone: Europe/Stockholm - enabled: true + transport: - timezone: Europe/Stockholm + type: tcp + host: 192.0.2.10 + port: 1234 - transport: + connectTimeout: 5s - type: tcp + aquacom: - host: 192.0.2.10 + address: 101 - port: 1234 + profile: default - connectTimeout: 5s + acquisition: + live: + enabled: true - aquacom: + pollingInterval: 5s - address: 101 + requestTimeout: 3s - profile: default + retryCount: 2 + history: + enabled: true - acquisition: + backfillEnabled: true + maxBackfillAge: 7d + timestamps: - live: + sourceTimezone: Europe/Stockholm - enabled: true + maximumFutureDeviation: 5m - pollingInterval: 5s + health: - requestTimeout: 3s + degradedAfterFailures: 2 - retryCount: 2 + offlineAfterFailures: 5 + staleAfter: 30s + pointDefinitions: - history: + sourceType: atf - enabled: true - - backfillEnabled: true - - maxBackfillAge: 7d - - - - timestamps: - - sourceTimezone: Europe/Stockholm - - maximumFutureDeviation: 5m - - - - health: - - degradedAfterFailures: 2 - - offlineAfterFailures: 5 - - staleAfter: 30s - - - - pointDefinitions: - - sourceType: atf - - path: ./stations/PS001.atf + path: ./stations/PS001.atf ``` +--- - -\--- - - - -\# 74. Example resulting data flow - - +# 74. Example resulting data flow Using the configuration above: - - ```text PS001 - │ + │ - │ AquaCom + │ AquaCom - ▼ + ▼ Gateway - │ + │ - ├────────► Current-State Service + ├────────► Current-State Service - │ │ + │ │ - │ ▼ + │ ▼ - │ OPC UA DA + │ OPC UA DA - │ + │ - └────────► Historical Pipeline + └────────► Historical Pipeline - │ + │ - ▼ + ▼ - Durable Spool + Durable Spool - │ + │ - ┌─────────┼─────────────┐ + ┌─────────┼─────────────┐ - │ │ │ + │ │ │ - ▼ ▼ ▼ + ▼ ▼ ▼ - Local History Central Archive + Local History Central Archive - Repository Historian + Repository Historian - │ + │ - ▼ + ▼ - OPC UA HA + OPC UA HA ``` +--- - -\--- - - - -\# 75. Recommended configuration file organization - - +# 75. Recommended configuration file organization Small installations may use one file: - - ```text config/ @@ -5038,12 +3251,8 @@ config/ ``` - - Larger installations should be able to split configuration logically: - - ```text config/ @@ -5060,268 +3269,192 @@ config/ └── stations/ - ├── PS001.yaml + ├── PS001.yaml - ├── PS002.yaml + ├── PS002.yaml - ├── PS003.yaml + ├── PS003.yaml - └── ... + └── ... ``` - - The implementation may support file inclusion or configuration-directory loading. - - Regardless of physical layout, the resulting configuration model shall be equivalent to the logical model described in this document. +--- - -\--- - - - -\# 76. Recommended station-file organization - - +# 76. Recommended station-file organization For a larger number of AquaCom installations: - - ```text config/ └── stations/ - │ + │ - ├── PS001/ + ├── PS001/ - │ ├── station.yaml + │ ├── station.yaml - │ └── station.atf + │ └── station.atf - │ + │ - ├── PS002/ + ├── PS002/ - │ ├── station.yaml + │ ├── station.yaml - │ └── station.atf + │ └── station.atf - │ + │ - └── PS003/ + └── PS003/ - ├── station.yaml + ├── station.yaml - └── station.atf + └── station.atf ``` - - This allows station-specific engineering configuration to remain grouped. +--- - -\--- - - - -\# 77. Relationship to ATF - - +# 77. Relationship to ATF Where ATF files are used, configuration and ATF have separate responsibilities. - - Conceptually: - - ```text station.yaml - │ + │ - ├── transport + ├── transport - ├── communication + ├── communication - ├── acquisition + ├── acquisition - ├── history + ├── history - ├── security/context + ├── security/context - └── ATF reference + └── ATF reference - │ + │ - ▼ + ▼ - station.atf + station.atf - │ + │ - └── AquaCom station capability / + └── AquaCom station capability / - point definition + point definition ``` - - ATF-specific semantics shall be documented under: - - ```text docs/protocol/ ``` - - once verified. - - The generic configuration model shall not duplicate all information contained in the ATF unless required for overrides or normalization. +--- - -\--- - - - -\# 78. Point overrides - - +# 78. Point overrides The architecture should permit gateway-specific metadata to augment imported point definitions. - - Conceptual example: - - ```yaml pointOverrides: + LEVEL: + displayName: Wet well level - LEVEL: + engineeringUnit: m - displayName: Wet well level + opcua: - engineeringUnit: m + enabled: true + historizing: true + INTERNAL_DIAGNOSTIC_17: - opcua: + opcua: - enabled: true + enabled: false - historizing: true + history: - - - INTERNAL\_DIAGNOSTIC\_17: - - opcua: - - enabled: false - - - - history: - - enabled: false + enabled: false ``` - - This permits an imported ATF point model to remain unchanged while gateway behavior is customized. +--- - -\--- - - - -\# 79. Configuration precedence - - +# 79. Configuration precedence If multiple configuration sources are supported, precedence shall be explicit. - - Recommended conceptual order: - - ```text built-in safe defaults - ↓ + ↓ global configuration - ↓ + ↓ station configuration - ↓ + ↓ point definition source - ↓ + ↓ explicit point override ``` - - Later layers may override earlier values only where permitted. - - Secrets are resolved independently and shall not be written back into the effective configuration. +--- - -\--- - - - -\# 80. Effective configuration - - +# 80. Effective configuration The gateway should provide a diagnostic mechanism for viewing the effective non-secret configuration. - - For example: - - ```text configured polling interval @@ -5338,32 +3471,18 @@ selected OPC UA NodeId ``` - - This significantly improves troubleshooting. - - Secret values shall be redacted. +--- - -\--- - - - -\# 81. Configuration audit - - +# 81. Configuration audit Changes to active configuration should be auditable. - - An audit event should identify: - - ```text configuration version @@ -5380,158 +3499,104 @@ operator/source where known ``` - - The exact implementation is technology-dependent. +### Administrative channel +Audit events require an administrative actor, but no interactive administration interface is defined in the initial scope. Until one is introduced, administrative actions (configuration activation, certificate trust decisions, manually initiated history synchronization) are performed through the deployment mechanism — controlled configuration-file changes and process restart/reload — and the audit record shall identify that source accordingly (for example, `operator: config-file`). A dedicated administrative interface (CLI, API, or similar) may be added later as a separately specified and separately authorized capability; implementations shall not invent one ad hoc. -\--- - - - -\# 82. Safe configuration behavior - +--- +# 82. Safe configuration behavior When configuration is incomplete or contradictory, prefer explicit failure over implicit behavior. - - Examples: - - Do not automatically select an arbitrary repository if: - - ```yaml opcua: - historicalAccess: + historicalAccess: - enabled: true + enabled: true ``` - - and multiple repositories exist but none is identified. - - Do not silently enable insecure OPC UA because certificate configuration failed. - - Do not invent AquaCom transport defaults if no verified defaults exist. - - Do not activate historical retrieval for an unsupported AquaCom controller profile. +--- +# 83. Configuration design rule -\--- - - - -\# 83. Configuration design rule - - - -Configuration should express \*\*intent\*\*, not implementation internals. - - +Configuration should express **intent**, not implementation internals. Prefer: - - ```yaml history: - enabled: true + enabled: true ``` - - over implementation-specific concepts such as: - - ```yaml instantiateHistoryWorkerThread: true ``` - - Prefer: - - ```yaml required: true ``` - - over: - - ```yaml deleteSpoolRecordAfterDestination2: false ``` - - The configuration model shall remain understandable independently of implementation language. +--- - -\--- - - - -\# 84. Configuration schema files - - +# 84. Configuration schema files Machine-readable schemas should be separated approximately as: - - ```text schemas/ -├── gateway-config.schema.\* +├── gateway-config.schema.* -├── station-config.schema.\* +├── station-config.schema.* -├── point-config.schema.\* +├── point-config.schema.* -└── storage-config.schema.\* +└── storage-config.schema.* ``` - - Schemas shall encode: - - ```text required properties @@ -5548,211 +3613,125 @@ basic structural constraints ``` - - Cross-component semantic validation may require application-level validation in addition to schema validation. +--- - -\--- - - - -\# 85. Configuration tests - - +# 85. Configuration tests Automated tests shall verify at least: - - ```text minimal valid configuration accepted - - complete valid configuration accepted - - duplicate station ID rejected - - duplicate point ID rejected - - duplicate OPC UA NodeId rejected - - missing History Repository rejected when OPC UA HA enabled - - unknown storage destination rejected - - invalid timezone rejected - - invalid duration rejected - - unsupported transport rejected - - unsupported AquaCom profile rejected - - history configuration rejected where implementation support is absent - - insecure OPC UA production default prevented - - missing required secret reported - - point override applied correctly - - configuration reload failure leaves previous valid configuration active ``` +--- - -\--- - - - -\# 86. Initial implementation recommendation - - +# 86. Initial implementation recommendation The first configuration implementation should support only the fields required for the initial production scope. - - The model in this document defines the intended architecture but does not require every optional field to be implemented immediately. - - Unsupported fields shall be: - - ```text rejected ``` - - or explicitly documented as: - - ```text not implemented ``` - - They shall not be silently ignored unless the configuration model explicitly permits forward-compatible unknown fields. +--- - -\--- - - - -\# 87. Final configuration rule - - +# 87. Final configuration rule The configuration system shall preserve the architectural separation of the gateway. - - In particular: - - ```text station transport configuration - ≠ + ≠ station identity - - display name - ≠ + ≠ PointId - - OPC UA NodeId - ≠ + ≠ runtime object identifier - - history retrieval - ≠ + ≠ OPC UA HistoryRead - - storage destination - ≠ + ≠ AquaCom protocol implementation ``` - - The configuration model shall describe how components are connected without coupling those components internally. - - The most important configuration principles are: - - > Stable identifiers shall remain stable even when presentation or transport details change. - - > Invalid or unsupported AquaCom behavior shall never be activated through configuration by relying on guessed protocol semantics. - - > Historical acquisition, historical persistence, and OPC UA Historical Access shall remain independently configurable concerns. diff --git a/docs/product/codex-implementation-specification.md b/docs/product/codex-implementation-specification.md index 2d6248e..65cd1ce 100644 --- a/docs/product/codex-implementation-specification.md +++ b/docs/product/codex-implementation-specification.md @@ -1,10830 +1,3752 @@ -\\# Codex Implementation Specification +# Implementation Specification +## AquaCom Integration Gateway +**Document type:** Implementation specification -\\## AquaCom Integration Gateway +**Target implementation agent:** any AI coding agent (Codex, Copilot, or equivalent) +**System:** AquaCom Integration Gateway +**Primary southbound interface:** AquaCom +**Primary northbound interface:** OPC UA +**Additional capability:** Pluggable historical storage +**Programming language:** To be selected during implementation planning +**Version:** 0.1 -\\\*\\\*Document type:\\\*\\\* Implementation specification - - - -\\\*\\\*Target implementation agent:\\\*\\\* Codex - - - -\\\*\\\*System:\\\*\\\* AquaCom Integration Gateway - - - -\\\*\\\*Primary southbound interface:\\\*\\\* AquaCom - - - -\\\*\\\*Primary northbound interface:\\\*\\\* OPC UA - - - -\\\*\\\*Additional capability:\\\*\\\* Pluggable historical storage - - - -\\\*\\\*Programming language:\\\*\\\* To be selected during implementation planning - - - -\\\*\\\*Version:\\\*\\\* 0.1 - - - -\\\*\\\*Status:\\\*\\\* Draft - - - - - - - -\\--- - - - - - - - -\\# 1. Purpose - - - - - - - -This document defines how Codex shall implement the AquaCom Integration Gateway. - - - +**Status:** Draft +--- +# 1. Purpose +This document defines how the implementation agent shall implement the AquaCom Integration Gateway. The gateway shall: +- communicate with AquaCom-compatible devices; +- acquire current process information; +- acquire historical process information; +- maintain a protocol-independent internal data model; +- expose current values through OPC UA; +- expose historical values through OPC UA Historical Access; -\\- communicate with AquaCom-compatible devices; - - - -\\- acquire current process information; - - - -\\- acquire historical process information; - - - -\\- maintain a protocol-independent internal data model; - - - -\\- expose current values through OPC UA; - - - -\\- expose historical values through OPC UA Historical Access; - - - -\\- persist historical information through one or several configurable storage providers; - - - -\\- recover historical data after communication interruptions where AquaCom equipment supports this; - - - -\\- maintain diagnostic information; - - - -\\- provide a simulation environment for development and automated testing. - - - +- persist historical information through one or several configurable storage providers; +- recover historical data after communication interruptions where AquaCom equipment supports this; +- maintain diagnostic information; +- provide a simulation environment for development and automated testing. This document describes: +- repository organization; +- architectural boundaries; +- required modules; +- internal interfaces; +- data contracts; +- state machines; -\\- repository organization; - - - -\\- architectural boundaries; - - - -\\- required modules; - - - -\\- internal interfaces; - - - -\\- data contracts; - - - -\\- state machines; - - - -\\- configuration; - - - -\\- persistence; - - - -\\- testing; - - - -\\- milestones; - - - -\\- Codex working rules; - - - -\\- acceptance criteria. - +- configuration; +- persistence; +- testing; +- milestones; +- implementation-agent working rules; +- acceptance criteria. It does not prescribe the implementation programming language. +--- - - - - - -\\--- - - - - - - - -\\# 2. Fundamental implementation rule - - - - - - +# 2. Fundamental implementation rule The implementation shall be based on the following processing architecture: - - - - - - ```text - - AquaCom + │ - -\ │ - - - -\ ▼ - - + ▼ Transport + │ - -\ │ - - - -\ ▼ - - + ▼ AquaCom protocol engine + │ - -\ │ - - - -\ ▼ - - + ▼ Station/session manager + │ - -\ │ - - - -\ ▼ - - + ▼ Canonical data model + │ + ├──────────────► Current-state service -\ │ + │ │ + │ ▼ + │ OPC UA DA -\ ├──────────────► Current-state service + │ + └──────────────► Historical pipeline + │ -\ │ │ + ┌──────────┼──────────────┐ + │ │ │ + ▼ ▼ ▼ -\ │ ▼ + Repository Storage A Storage B + │ + ▼ -\ │ OPC UA DA - - - -\ │ - - - -\ └──────────────► Historical pipeline - - - -\ │ - - - -\ ┌──────────┼──────────────┐ - - - -\ │ │ │ - - - -\ ▼ ▼ ▼ - - - -\ Repository Storage A Storage B - - - -\ │ - - - -\ ▼ - - - -\ OPC UA HA - - + OPC UA HA ``` - - - - - - No component above the AquaCom protocol layer shall parse or construct AquaCom telegrams. - - - - - - No AquaCom component shall depend upon OPC UA. - - - - - - No AquaCom component shall depend upon a particular database. - - - - - - No storage adapter shall depend upon AquaCom telegram structure. +--- +# 3. Implementation agent principles +The implementation agent shall follow these principles throughout development. +## 3.1 Do not invent AquaCom protocol details +The implementation agent shall never invent: +- byte positions; -\\--- +- telegram structure; +- field identifiers; +- checksums; +- addressing rules; +- commands; +- response codes; +- scaling; -\\# 3. Codex implementation principles - - - - - - - -Codex shall follow these principles throughout development. - - - - - - - -\\## 3.1 Do not invent AquaCom protocol details - - - - - - - -Codex shall never invent: - - - - - - - -\\- byte positions; - - - -\\- telegram structure; - - - -\\- field identifiers; - - - -\\- checksums; - - - -\\- addressing rules; - - - -\\- commands; - - - -\\- response codes; - - - -\\- scaling; - - - -\\- timestamp encoding; - - - -\\- historical-data behavior. - - - - - +- timestamp encoding; +- historical-data behavior. Protocol behavior may only be implemented when supported by at least one of: +1. authoritative AquaCom documentation; +2. validated packet/serial captures; +3. existing known-good implementation behavior; - - - -1\\. authoritative AquaCom documentation; - - - -2\\. validated packet/serial captures; - - - -3\\. existing known-good implementation behavior; - - - -4\\. approved test vectors. - - - - - - +4. approved test vectors. Unknown protocol details shall be represented as: - - - - - - ```text - - -NOT\\\_IMPLEMENTED - - +NOT_IMPLEMENTED ``` - - - - - - or documented protocol gaps. - - - - - - A compilation-successful guess is not an acceptable implementation. +--- - - - - - -\\--- - - - - - - - -\\# 4. Repository documentation strategy - - - - - - +# 4. Repository documentation strategy The repository shall contain a short root `AGENTS.md`. - - - - - - Detailed requirements shall live under `docs/`. +AGENTS.md shall primarily direct the implementation agent to the authoritative project documentation rather than attempt to contain the complete project specification. +This follows current agent guidance favoring concise repository instructions with deeper documentation maintained separately. +Complex implementation work shall use an execution plan maintained in the repository. Current agent guidance recommends plan documents for significant features or refactors. +--- - - -`AGENTS.md` shall primarily direct Codex to the authoritative project documentation rather than attempt to contain the complete project specification. - - - - - - - -This follows current Codex guidance favoring concise repository instructions with deeper documentation maintained separately. - - - - - - - -Complex implementation work shall use an execution plan maintained in the repository. OpenAI's current Codex guidance explicitly recommends plan documents for significant features or refactors. - - - - - - - -\\--- - - - - - - - -\\# 5. Repository structure - - - - - - +# 5. Repository structure The initial repository shall use the following logical organization: - - - - - - ```text - - / - - ├── AGENTS.md - - ├── README.md - - ├── ARCHITECTURE.md - - ├── CHANGELOG.md - - ├── LICENSE - - │ - - ├── docs/ - - │ ├── product/ - - │ │ └── software-specification.md - - │ │ - - │ ├── architecture/ - - │ │ ├── overview.md - - │ │ ├── data-flow.md - - │ │ ├── live-data.md - - │ │ ├── historical-data.md - - │ │ ├── opcua.md - - │ │ ├── storage.md - - │ │ └── security.md - - │ │ - - │ ├── protocol/ - - │ │ ├── aquacom.md - - │ │ ├── known-operations.md - - │ │ ├── protocol-gaps.md - - │ │ └── test-vectors.md - - │ │ - - │ ├── configuration/ - - │ │ └── configuration-reference.md - - │ │ - - │ ├── operations/ - - │ │ ├── deployment.md - - │ │ ├── diagnostics.md - - │ │ ├── backup-recovery.md - - │ │ └── troubleshooting.md - - │ │ - - │ └── exec-plans/ - - │ ├── active/ - - │ └── completed/ - - │ - - ├── schemas/ +│ ├── gateway-config.schema.* +│ ├── station-config.schema.* -│ ├── gateway-config.schema.\\\* - - - -│ ├── station-config.schema.\\\* - - - -│ └── point-config.schema.\\\* - - +│ └── point-config.schema.* │ - - ├── config/ - - │ └── examples/ - - │ - - ├── src/ - - │ ├── core/ - - │ ├── aquacom/ - - │ ├── history/ - - │ ├── storage/ - - │ ├── opcua/ - - │ ├── diagnostics/ - - │ └── host/ - - │ - - ├── tests/ - - │ ├── unit/ - - │ ├── integration/ - - │ ├── protocol/ - - │ ├── opcua/ - - │ ├── history/ - - │ └── acceptance/ - - │ - - ├── simulator/ - - │ - - ├── fixtures/ - - │ ├── aquacom/ - - │ ├── history/ - - │ └── configuration/ - - │ - - ├── tools/ - - │ - - └── deployment/ - - ``` - - - - - - File extensions shall be selected once the implementation language and configuration formats are chosen. +--- +# 6. Root AGENTS.md +The root `AGENTS.md` implements this strategy: it is the short entry point that directs the implementation agent to the authoritative documents, lists the non-negotiable architecture and evidence rules, and never restates full specifications. The current root `AGENTS.md` in this repository is the reference implementation of that form and takes precedence over any example restated here. +Nested `AGENTS.md` files may later be added where a specific module requires additional constraints. Current coding agents support hierarchical `AGENTS.md` instructions scoped to directory trees, making this suitable for stricter protocol or storage-specific rules if the repository grows. +--- - -\\--- - - - - - - - -\\# 6. Root AGENTS.md - - - - - - - -The root `AGENTS.md` shall contain approximately the following instructions. - - - - - - - -```text - - - -\\# AquaCom Integration Gateway - - - - - - - -Read ARCHITECTURE.md and the relevant documents under docs/ before - - - -changing implementation behavior. - - - - - - - -For substantial features, protocol changes, persistence changes, or - - - -cross-module refactors, create or update an ExecPlan under - - - -docs/exec-plans/active/. - - - - - - - -\\## Non-negotiable architecture rules - - - - - - - -1\\. AquaCom protocol parsing belongs only in src/aquacom. - - - -2\\. OPC UA must never parse AquaCom telegrams. - - - -3\\. Storage adapters consume canonical historical records only. - - - -4\\. Historical samples must never be replayed as current process values. - - - -5\\. Preserve source timestamps separately from gateway receipt timestamps. - - - -6\\. Stable point IDs and OPC UA NodeIds must survive restart and renaming. - - - -7\\. Do not invent undocumented AquaCom protocol behavior. - - - -8\\. Every protocol implementation requires automated test vectors. - - - -9\\. Every bug fix requires a regression test where practical. - - - -10\\. Live acquisition takes priority over historical backfill. - - - - - - - -\\## AquaCom protocol evidence - - - - - - - -Protocol behavior may only be implemented using documented or validated - - - -evidence stored or referenced from docs/protocol/. - - - - - - - -If information is insufficient, document the gap in - - - -docs/protocol/protocol-gaps.md instead of guessing. - - - - - - - -\\## Testing - - - - - - - -Run applicable unit, integration and acceptance tests before considering - - - -a task complete. - - - - - - - -Do not weaken or remove tests merely to make a change pass. - - - - - - - -\\## Security - - - - - - - -Never commit secrets, private keys, production certificates, passwords - - - -or production network addresses. - - - - - - - -OPC UA insecure modes must not become the production default. - - - - - - - -\\## Documentation - - - - - - - -Update relevant architecture or configuration documentation whenever - - - -observable behavior changes. - - - -``` - - - - - - - -Nested `AGENTS.md` files may later be added where a specific module requires additional constraints. - - - - - - - -Codex supports hierarchical `AGENTS.md` instructions scoped to directory trees, making this suitable for stricter protocol or storage-specific rules if the repository grows. - - - - - - - -\\--- - - - - - - - -\\# 7. Architecture document - - - - - - +# 7. Architecture document `ARCHITECTURE.md` shall act as the primary architecture map. - - - - - - It shall contain: +- system purpose; +- module diagram; +- dependency direction; +- canonical data model; +- live-data flow; +- historical-data flow; -\\- system purpose; - - - -\\- module diagram; - - - -\\- dependency direction; - - - -\\- canonical data model; - - - -\\- live-data flow; - - - -\\- historical-data flow; - - - -\\- security boundaries; - - - -\\- links to detailed documents. - - - - - +- security boundaries; +- links to detailed documents. The following dependency direction is mandatory: - - - - - - ```text + ┌──────────────┐ + │ host │ -\ ┌──────────────┐ + └──────┬───────┘ + │ + ┌────────────┼────────────┐ -\ │ host │ + ▼ ▼ ▼ + OPC UA history diagnostics + │ │ -\ └──────┬───────┘ + │ storage + │ │ + └──────┬─────┘ -\ │ + ▼ + core + ▲ -\ ┌────────────┼────────────┐ - - - -\ ▼ ▼ ▼ - - - -\ OPC UA history diagnostics - - - -\ │ │ - - - -\ │ storage - - - -\ │ │ - - - -\ └──────┬─────┘ - - - -\ ▼ - - - -\ core - - - -\ ▲ - - - -\ │ - - - -\ aquacom - + │ + aquacom ``` - - - - - - `core` shall contain no dependencies on AquaCom, OPC UA or a particular storage implementation. +--- - - - - - -\\--- - - - - - - - -\\# 8. Core domain module - - - - - - +# 8. Core domain module The `core` module shall contain protocol-neutral data models. - - - - - - Required logical entities include: - - - - - - ```text - - StationId - - PointId - - CurrentValue - - HistoricalSample - - DataQuality - - AcquisitionSource - - StationState - - CommunicationState - - HistorySynchronizationState - - ``` +--- +# 9. Stable identifiers - - - - -\\--- - - - - - - - -\\# 9. Stable identifiers - - - - - - - -\\## 9.1 StationId - - - - - - +## 9.1 StationId `StationId` shall be an immutable logical gateway identifier. - - - - - - It shall not depend solely upon: +- IP address; +- hostname; +- serial port; - - - -\\- IP address; - - - -\\- hostname; - - - -\\- serial port; - - - -\\- display name. - - - - - - +- display name. Example: - - - - - - ```text - - station:PS001 - - ``` +--- - - - - - -\\--- - - - - - - - -\\## 9.2 PointId - - - - - - +## 9.2 PointId `PointId` shall be an immutable logical identifier. - - - - - - Example: - - - - - - ```text - - station:PS001/point:LEVEL - - ``` - - - - - - Changing: - - - - - - ```text - - Wet well level - - ``` - - - - - - to: - - - - - - ```text - - Inlet sump level - - ``` - - - - - - shall not change `PointId`. +--- - - - - - -\\--- - - - - - - - -\\# 10. CurrentValue contract - - - - - - +# 10. CurrentValue contract The canonical current-value model shall conceptually contain: - - - - - - ```text - - CurrentValue - - - - - - StationId - - PointId - - Value - - DataType - - Quality - - SourceTimestamp? - - ReceivedTimestamp - - UpdatedTimestamp - - AcquisitionSource - - ``` - - - - - - `SourceTimestamp` may be absent when AquaCom provides no trustworthy source timestamp. - - - - - - `ReceivedTimestamp` shall always be gateway-generated. +--- - - - - - -\\--- - - - - - - - -\\# 11. HistoricalSample contract - - - - - - +# 11. HistoricalSample contract Historical samples shall contain: - - - - - - ```text - - HistoricalSample - - - - - - StationId - - PointId - - Value - - DataType - - Quality - - SourceTimestamp - - ReceivedTimestamp - - AcquisitionSource - - AcquisitionId? - - Metadata? - - ``` - - - - - - `SourceTimestamp` is mandatory for normal process history. +A record without a reliable historical timestamp shall not silently be interpreted as historical process data. Such a record shall be **quarantined with diagnostics**: rejected from normal history, recorded distinctly for investigation, and never stored with a fabricated or gateway-generated timestamp (see `software-specification.md` §10). +--- - - - - -A record without a reliable historical timestamp shall not silently be interpreted as historical process data. - - - - - - - -\\--- - - - - - - - -\\# 12. DataQuality - - - - - - +# 12. DataQuality Internal quality shall be independent of OPC UA. - - - - - - At minimum: - - - - - - ```text - - GOOD - - UNCERTAIN - - BAD - - UNKNOWN - - ``` - - - - - - Optional reason/substatus shall support values equivalent to: - - - - - - ```text - - -COMMUNICATION\\\_FAILURE - - +COMMUNICATION_FAILURE STALE +INVALID_VALUE +INVALID_TIMESTAMP -INVALID\\\_VALUE +CONFIGURATION_ERROR +CONVERSION_ERROR +SOURCE_UNCERTAIN -INVALID\\\_TIMESTAMP - - - -CONFIGURATION\\\_ERROR - - - -CONVERSION\\\_ERROR - - - -SOURCE\\\_UNCERTAIN - - - -HISTORY\\\_GAP - - +HISTORY_GAP ``` - - - - - - The OPC UA module shall map this model to OPC UA StatusCodes. +--- - - - - - -\\--- - - - - - - - -\\# 13. AcquisitionSource - - - - - - +# 13. AcquisitionSource Historical and current information shall identify origin. - - - - - - Examples: - - - - - - ```text - - LIVE +AQUACOM_HISTORY - -AQUACOM\\\_HISTORY - - - -AQUACOM\\\_REPORT - - +AQUACOM_REPORT RECOVERY - - IMPORT - - SIMULATOR - - ``` - - - - - - Implementation shall permit additional values. +--- - - - - - -\\--- - - - - - - - -\\# 14. AquaCom transport interface - - - - - - +# 14. AquaCom transport interface The AquaCom transport abstraction shall provide functions equivalent to: - - - - - - ```text - - open() - - close() - - send(bytes) - - receive() - - isConnected() - - getDiagnostics() - - ``` - - - - - - The interface shall not interpret AquaCom fields. - - - - - - Potential transport implementations may include: +- TCP; +- serial; +- modem abstraction; - - - -\\- TCP; - - - -\\- serial; - - - -\\- modem abstraction; - - - -\\- serial-over-IP. - - - - - - +- serial-over-IP. The initial implementation only needs transports required by the target environment. +--- - - - - - -\\--- - - - - - - - -\\# 15. AquaCom protocol codec - - - - - - +# 15. AquaCom protocol codec The protocol codec shall provide deterministic transformation between: - - - - - - ```text - - raw transport bytes - - ``` - - - - - - and: - - - - - - ```text - - AquaComMessage - - ``` - - - - - - Logical interfaces: - - - - - - ```text - - encode(request) -> bytes - - - - - - decode(bytes) -> message - - - - - - -validate(bytes) -> validation\\\_result - - +validate(bytes) -> validation_result ``` - - - - - - The codec shall have no network dependency. - - - - - - This is required so protocol behavior can be exhaustively unit tested. +--- - - - - - -\\--- - - - - - - - -\\# 16. Protocol test vectors - - - - - - +# 16. Protocol test vectors Every implemented AquaCom telegram type shall have test fixtures. - - - - - - Fixture format shall contain conceptually: - - - - - - ```text - - name - - source - - direction - - raw-data - - decoded-operation - - decoded-fields - - expected-validation - - notes - - ``` - - - - - - Example source classifications: - - - - - - ```text - - SPECIFICATION - - CAPTURE +KNOWN_GOOD_IMPLEMENTATION - -KNOWN\\\_GOOD\\\_IMPLEMENTATION - - - -SYNTHETIC\\\_FROM\\\_SPECIFICATION - - +SYNTHETIC_FROM_SPECIFICATION ``` - - - - - - The test suite shall prove both: - - - - - - ```text - - raw → decoded - - ``` - - - - - - and where applicable: - - - - - - ```text - - object → encoded raw - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 17. Protocol gap registry - - - - - - +# 17. Protocol gap registry `docs/protocol/protocol-gaps.md` shall maintain unresolved protocol issues. - - - - - - Each entry shall contain: - - - - - - ```text - - Gap ID - - Description - - Affected operation - - Evidence available - - Evidence missing - - Impact - - Implementation status - - ``` - - - - - - Example: - - - - - - ```text - - AQ-GAP-007 - - - - - - Exact encoding of history timestamp field for controller type X - - is not yet verified. - - - - - - Impact: - - History retrieval cannot be enabled for this controller type. - - - - - - Required evidence: - - Protocol specification or validated capture. - - ``` +The implementation agent shall not close a protocol gap without evidence. +### Evidence acquisition plan +Protocol evidence does not currently exist in this repository. The evidence rule above is therefore not executable until evidence is acquired. Until then, Phase 3 (codec framework) may proceed with the framework, fixture and simulation scaffolding, but no telegram behavior may be implemented. +Evidence shall be acquired through at least one of: +1. obtaining the authoritative AquaCom protocol specification from the equipment vendor or system owner; +2. capturing traffic from a known-good AquaCom deployment (sanitized per §64 before committing); +3. deriving behavior from an existing known-good integration. +The responsible party for obtaining the authoritative documentation shall be identified before Phase 3 starts, and the acquisition status shall be tracked in `docs/protocol/protocol-gaps.md` until resolved. Implementation milestones that depend on specific operations (live polling, history retrieval) are blocked by missing evidence for those operations and shall be sequenced after the evidence exists. -Codex shall not close a protocol gap without evidence. - - - - - - - -\\--- - - - - - - - -\\# 18. AquaCom session manager - - - - - +--- +# 18. AquaCom session manager Each configured station shall use an independent session/state abstraction. - - - - - - Conceptual states: - - - - - - ```text - - DISABLED - - INITIALIZING - - CONNECTING - - ONLINE - - DEGRADED - - OFFLINE - - RECOVERING - - ``` - - - - - - Basic transition: - - - - - - ```text - - -\ successful communication - - + successful communication OFFLINE ──────────────────────────────► ONLINE - - - - - - ONLINE + │ + │ repeated communication failures -\ │ - - - -\ │ repeated communication failures - - - -\ ▼ - - + ▼ DEGRADED + │ + │ threshold exceeded -\ │ - - - -\ │ threshold exceeded - - - -\ ▼ - - + ▼ OFFLINE + │ + │ transport available -\ │ - - - -\ │ transport available - - - -\ ▼ - - + ▼ RECOVERING + │ + │ successful synchronization -\ │ - - - -\ │ successful synchronization - - - -\ ▼ - - + ▼ ONLINE - - ``` - - - - - - Thresholds shall be configurable. +--- - - - - - -\\--- - - - - - - - -\\# 19. Live acquisition scheduler - - - - - - +# 19. Live acquisition scheduler The live acquisition scheduler shall: +- schedule AquaCom live requests; +- apply per-station polling configuration; +- prevent overlapping invalid station transactions; +- prioritize live communication over history backfill; +- implement retry policy; - -\\- schedule AquaCom live requests; - - - -\\- apply per-station polling configuration; - - - -\\- prevent overlapping invalid station transactions; - - - -\\- prioritize live communication over history backfill; - - - -\\- implement retry policy; - - - -\\- update communication diagnostics. - - - - - - +- update communication diagnostics. The scheduler shall not write directly to OPC UA. - - - - - - It shall publish canonical `CurrentValue` updates. +--- - - - - - -\\--- - - - - - - - -\\# 20. Current-state service - - - - - - +# 20. Current-state service The Current-State Service shall maintain the latest accepted current value for each PointId. - - - - - - Interface equivalent: - - - - - - ```text - - update(CurrentValue) - - - - - - get(PointId) - - - - - - getAll(StationId) - - - - - - subscribe(PointId) - - ``` - - - - - - The service shall reject obsolete updates where appropriate. - - - - - - A recovered historical sample shall not invoke: - - - - - - ```text - - CurrentState.update() - - ``` - - - - - - unless it has independently been determined to represent current state. +--- - - - - - -\\--- - - - - - - - -\\# 21. Historical acquisition manager - - - - - - +# 21. Historical acquisition manager The Historical Acquisition Manager shall manage history retrieval from AquaCom stations. - - - - - - Responsibilities: +- determine retrieval windows; +- initiate historical requests; +- maintain synchronization watermark; +- process partial responses; +- resume interrupted synchronization; +- throttle retrieval; -\\- determine retrieval windows; - - - -\\- initiate historical requests; - - - -\\- maintain synchronization watermark; - - - -\\- process partial responses; - - - -\\- resume interrupted synchronization; - - - -\\- throttle retrieval; - - - -\\- deduplicate; - - - -\\- send canonical historical samples to the historical pipeline. - - - - - +- deduplicate; +- send canonical historical samples to the historical pipeline. Conceptual interface: - - - - - - ```text - - synchronize(station) - - - - - - synchronize(station, start, end) - - - - - - getSynchronizationState(station) - - - - - - pause(station) - - - - - - resume(station) - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 22. Synchronization watermark - - - - - - +# 22. Synchronization watermark Each station shall maintain persistent history synchronization state. - - - - - - Example: - - - - - - ```text - - StationId - - HistoryStreamId - - LastCompleteTimestamp - - LastAttemptTimestamp - - LastSuccessfulTimestamp - - SynchronizationState - - ``` +The watermark shall only advance when the implementation has sufficient evidence that the interval has been processed successfully. The precise advancement condition — durably spooled, not finally delivered — is defined in `ARCHITECTURE.md` §21; final delivery is tracked independently through per-destination spool acknowledgements. +--- - - - - -The watermark shall only advance when the implementation has sufficient evidence that the interval has been processed successfully. - - - - - - - -\\--- - - - - - - - -\\# 23. Historical recovery state machine - - - - - - +# 23. Historical recovery state machine Conceptual history states: - - - - - - ```text - - IDLE - - -CHECK\\\_REQUIRED\\\_RANGE - - +CHECK_REQUIRED_RANGE REQUESTING - - RECEIVING - - PERSISTING - - VERIFYING - - COMPLETE - - -RETRY\\\_WAIT - - +RETRY_WAIT FAILED - - ``` - - - - - - Typical sequence: - - - - - - ```text - - IDLE + │ + ▼ -\ │ - - - -\ ▼ - - - -CHECK\\\_REQUIRED\\\_RANGE - - - -\ │ - - - -\ ▼ +CHECK_REQUIRED_RANGE + │ + ▼ REQUESTING + │ - -\ │ - - - -\ ▼ - - + ▼ RECEIVING + │ - -\ │ - - - -\ ▼ - - + ▼ PERSISTING + │ - -\ │ - - - -\ ▼ - - + ▼ VERIFYING + │ + ├── incomplete ──► REQUESTING -\ │ - - - -\ ├── incomplete ──► REQUESTING - - - -\ │ - - - -\ └── complete ───► COMPLETE - + │ + └── complete ───► COMPLETE ``` - - - - - - A restart during synchronization shall resume from persistent state without requiring complete duplication of the historical interval. +--- - - - - - -\\--- - - - - - - - -\\# 24. Historical pipeline - - - - - - +# 24. Historical pipeline The historical pipeline shall receive: - - - - - - ```text - - HistoricalSample - - ``` - - - - - - and perform: - - - - - - ```text - - validation - - -\ ↓ - - + ↓ normalization - - -\ ↓ - - + ↓ deduplication - - -\ ↓ - - + ↓ durable enqueue - - -\ ↓ - - + ↓ distribution - - ``` - - - - - - Conceptually: - - - - - - ```text - - HistoricalSample + │ - -\ │ - - - -\ ▼ - - + ▼ Validator + │ - -\ │ - - - -\ ▼ - - + ▼ Deduplicator + │ - -\ │ - - - -\ ▼ - - + ▼ Durable spool + │ + ┌─────┼─────────┐ -\ │ - - - -\ ┌─────┼─────────┐ - - - -\ ▼ ▼ ▼ - - + ▼ ▼ ▼ Repo Store A Store B - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 25. Deduplication key - - - - - - +# 25. Deduplication key Default logical identity: - - - - - - ```text - - StationId - - -\\+ - - ++ PointId - - -\\+ - - ++ SourceTimestamp - - ``` - - - - - - If AquaCom supplies a stronger identity such as a sequence number, the implementation may include it. - - - - - - The deduplication algorithm shall be deterministic. - - - - - - Collision behavior shall be configurable: - - - - - - ```text +KEEP_FIRST +KEEP_LAST -KEEP\\\_FIRST - - - -KEEP\\\_LAST - - - -REJECT\\\_CONFLICT - - - -VERSION\\\_CONFLICT - +REJECT_CONFLICT +VERSION_CONFLICT ``` - - - - - +The configuration reference (`docs/configuration/configuration-reference.md` §24.2) names these policies `keep-first`, `keep-last`, `reject`, and `version`; the semantic mapping between the two namings shall be kept in the configuration model so the same policy cannot drift apart between documents. Recommended default: - - - - - - ```text - - -REJECT\\\_CONFLICT - - +REJECT_CONFLICT ``` - - - - - - where conflicting values are stored diagnostically rather than silently overwritten. +Deduplication state shall remain correct across restart (see `ARCHITECTURE.md` §23): a restart during synchronization shall not introduce duplicate records, either through persisted deduplication state or through idempotent storage. +The deduplication window shall be bounded. +--- - - - -\\--- - - - - - - - -\\# 26. History repository interface - - - - - - +# 26. History repository interface A History Repository is a queryable source used by OPC UA Historical Access. - - - - - - Required logical interface: - - - - - - ```text - - store(sample) - - - - - - storeBatch(samples) - - - - - - readRaw( + pointId, + startTime, -\ pointId, - - - -\ startTime, - - - -\ endTime, - - - -\ limit, - - - -\ continuation + endTime, + limit, + continuation ) - - - - - - readAtTime( + pointId, - -\ pointId, - - - -\ timestamps - - + timestamps ) - - - - - - getBounds(pointId) - - - - - - health() - - ``` - - - - - - The repository may be: +- local; +- external; +- relational; +- time-series; - - -\\- local; - - - -\\- external; - - - -\\- relational; - - - -\\- time-series; - - - -\\- industrial historian. - - - - - - +- industrial historian. OPC UA shall interact only through this abstraction. +--- - - - - - -\\--- - - - - - - - -\\# 27. Storage sink interface - - - - - - +# 27. Storage sink interface A storage sink is an independent history destination. - - - - - - Required interface: - - - - - - ```text - - initialize() - - - - - - write(sample) - - - - - - writeBatch(samples) - - - - - - flush() - - - - - - health() - - - - - - shutdown() - - ``` - - - - - - A storage sink does not need to support queries. - - - - - - Examples: - - - - - - ```text - - SQL sink - - time-series sink - - industrial historian sink - - file/archive sink - - message-stream sink - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 28. Repository versus sink - - - - - - +# 28. Repository versus sink These concepts shall remain distinct. - - - - - - ```text - - Repository - - -\ = historical data can be queried - - - - - - + = historical data can be queried Sink - - -\ = historical data can be delivered - - + = historical data can be delivered ``` - - - - - - A component may implement both interfaces. - - - - - - Example: - - - - - - ```text - - Local historian: + Repository = yes - -\ Repository = yes - - - -\ Sink = yes - - - - - - + Sink = yes Archive file: + Repository = no - -\ Repository = no - - - -\ Sink = yes - - + Sink = yes ``` +--- - - - - - -\\--- - - - - - - - -\\# 29. Durable spool - - - - - - +# 29. Durable spool Historical delivery shall pass through persistent buffering. - - - - - - Conceptual record: - - - - - - ```text - - SpoolRecord - - - - - - RecordId - - HistoricalSample - - CreatedTimestamp +Destinations[] - -Destinations\\\[] - - - -Acknowledgements\\\[] - - +Acknowledgements[] RetryState - - ``` - - - - - - Example: - - - - - - ```text - - record 123 - - - - - - StorageLocal ACK - - CentralHistorian PENDING - - Archive ACK - - ``` - - - - - - Record 123 shall remain available until required delivery policy has been fulfilled. +--- - - - - - -\\--- - - - - - - - -\\# 30. Delivery policy - - - - - - +# 30. Delivery policy Configuration shall define whether a destination is: - - - - - - ```text - - REQUIRED - - OPTIONAL - - ``` - - - - - - Example: - - - - - - ```text - - LocalHistory: - - -\ REQUIRED - - - - - - + REQUIRED CentralHistorian: - - -\ REQUIRED - - - - - - + REQUIRED ArchiveExport: - - -\ OPTIONAL - - + OPTIONAL ``` - - - - - - A failed optional destination shall generate diagnostics but shall not indefinitely block spool cleanup. +--- - - - - - -\\--- - - - - - - - -\\# 31. Storage failure handling - - - - - - +# 31. Storage failure handling Each storage destination shall implement independent retry. - - - - - - Recommended conceptual retry progression: - - - - - - ```text - - immediate - - short delay - - increasing delay - - maximum delay - - periodic retry - - ``` - - - - - - Exact timing shall be configurable. - - - - - - Retries shall not block live acquisition. +--- - - - - - -\\--- - - - - - - - -\\# 32. OPC UA module - - - - - - +# 32. OPC UA module The OPC UA module shall contain: - - - - - - ```text - - Server lifecycle - - Namespace manager - - Node mapping - - Current-value provider - - Historical provider - - StatusCode mapper - - Security configuration - - Diagnostics exposure - - ``` - - - - - - It shall depend upon: - - - - - - ```text - - core - - current-state service - - history repository - - diagnostics - - ``` - - - - - - It shall not depend directly upon AquaCom transport or AquaCom codec. +--- - - - - - -\\--- - - - - - - - -\\# 33. OPC UA namespace mapping - - - - - - +# 33. OPC UA namespace mapping Default structure: - - - - - - ```text - - AquaCom - - │ - - ├── Gateway - - │ ├── Status - - │ ├── Version - - │ ├── StartTime - - │ ├── ConfigurationVersion - - │ ├── HistoricalQueueDepth - - │ └── Storage - - │ - - └── Stations + │ + ├── PS001 -\ │ + │ ├── Process + │ │ ├── Level + │ │ ├── Pump01Running -\ ├── PS001 + │ │ └── Pump02Running + │ │ + │ └── Diagnostics -\ │ ├── Process + │ ├── CommunicationState + │ ├── LastCommunication + │ ├── HistoryState -\ │ │ ├── Level - - - -\ │ │ ├── Pump01Running - - - -\ │ │ └── Pump02Running - - - -\ │ │ - - - -\ │ └── Diagnostics - - - -\ │ ├── CommunicationState - - - -\ │ ├── LastCommunication - - - -\ │ ├── HistoryState - - - -\ │ └── HistoryWatermark - - - -\ │ - - - -\ └── PS002 + │ └── HistoryWatermark + │ + └── PS002 ``` - - - - - - Namespace generation shall be deterministic. +--- - - - - - -\\--- - - - - - - - -\\# 34. OPC UA NodeId generation - - - - - - +# 34. OPC UA NodeId generation OPC UA NodeIds shall derive from stable internal identifiers. - - - - - - Example: - - - - - - ```text - - ns=;s=station/PS001/point/LEVEL - - ``` - - - - - - The exact syntax may differ. - - - - - - The critical requirement is: - - - - - - ```text - - same logical point - - -\ = - - + = same NodeId - - ``` - - - - - - across: +- restart; +- upgrade; +- rename; +- transport address change. +--- - -\\- restart; - - - -\\- upgrade; - - - -\\- rename; - - - -\\- transport address change. - - - - - - - -\\--- - - - - - - - -\\# 35. OPC UA current-value mapping - - - - - - +# 35. OPC UA current-value mapping For each configured live point: - - - - - - ```text - - CurrentValue.Value - - -\ → OPC UA Value - - - - - - + → OPC UA Value CurrentValue.SourceTimestamp - - -\ → OPC UA SourceTimestamp - - - - - - + → OPC UA SourceTimestamp gateway processing time - - -\ → OPC UA ServerTimestamp - - - - - - + → OPC UA ServerTimestamp CurrentValue.Quality - - -\ → OPC UA StatusCode - - + → OPC UA StatusCode ``` - - - - - - If communication fails, the node shall transition to a suitable non-Good status according to configuration. +--- - - - - - -\\--- - - - - - - - -\\# 36. OPC UA history provider - - - - - - +# 36. OPC UA history provider When an OPC UA client sends: - - - - - - ```text - - HistoryRead - - ``` - - - - - - the OPC UA layer shall call: - - - - - - ```text - - HistoryRepository.readRaw() - - ``` - - - - - - It shall never directly cause: - - - - - - ```text - - AquaCom history request - - ``` - - - - - - The following is prohibited: - - - - - - ```text - - OPC UA HistoryRead + │ - -\ │ - - - -\ ▼ - - + ▼ AquaCom WGH request + │ - -\ │ - - - -\ ▼ - - + ▼ wait for RTU - - ``` - - - - - - Correct behavior: - - - - - - ```text - - AquaCom synchronization + │ - -\ │ - - - -\ ▼ - - + ▼ Repository - - - - - - OPC UA HistoryRead + │ - -\ │ - - - -\ ▼ - - + ▼ Repository - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 37. OPC UA historical continuation - - - - - - +# 37. OPC UA historical continuation The implementation shall support bounded historical reads. - - - - - - Configuration: - - - - - - ```text - - -maximumValuesPerHistoryRead - - +opcua.historicalAccess.maxValuesPerRead ``` - - - - - +(This is the field defined in `docs/configuration/configuration-reference.md` §19.3. Earlier revisions of this document called it `maximumValuesPerHistoryRead`; the configuration-reference name is authoritative.) If more values exist than can be returned: - - - - - - ```text - - HistoryRead + │ - -\ │ - - - -\ ▼ - - + ▼ batch 1 + continuation token + │ - -\ │ - - - -\ ▼ - - + ▼ HistoryRead continuation + │ - -\ │ - - - -\ ▼ - - + ▼ batch 2 - - ``` - - - - - - Continuation state shall be resource-limited. +--- - - - - - -\\--- - - - - - - - -\\# 38. Time handling - - - - - - +# 38. Time handling Internal canonical timestamps shall be timezone-unambiguous. - - - - - - Preferred internal representation: - - - - - - ```text - - UTC timestamp - - ``` - - - - - - Station configuration shall define AquaCom timezone interpretation where required. - - - - - - Configuration fields shall include concepts equivalent to: - - - - - - ```text - - timezone - - timestampSource - - maximumFutureDeviation - - maximumPastDeviation - - dstPolicy - - ``` - - - - - - The raw source timestamp may optionally also be retained for diagnostics. +--- - - - - - -\\--- - - - - - - - -\\# 39. Configuration model - - - - - - +# 39. Configuration model A gateway configuration shall contain approximately: - - - - - - ```text - - gateway - - logging - - opcua - - history - - storage - - stations - - security - - diagnostics - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 40. Gateway configuration - - - - - - +# 40. Gateway configuration Conceptual example: - - - - - - ```text - - gateway: + id: aquacom-gateway-01 + site: central-ot -\ id: aquacom-gateway-01 - - - -\ site: central-ot - - - -\ timezone: Europe/Stockholm - - + timezone: Europe/Stockholm ``` +--- - - - - - -\\--- - - - - - - - -\\# 41. Station configuration - - - - - - +# 41. Station configuration Conceptual configuration: - - - - - - ```text - - station: + id: PS001 + enabled: true -\ id: PS001 + transport: + type: tcp + ... -\ enabled: true + aquacom: + address: ... + profile: ... + acquisition: + live: + enabled: true + interval: ... -\ transport: + history: + enabled: true + synchronization: ... -\ type: tcp - - - -\ ... - - - - - - - -\ aquacom: - - - -\ address: ... - - - -\ profile: ... - - - - - - - -\ acquisition: - - - -\ live: - - - -\ enabled: true - - - -\ interval: ... - - - - - - - -\ history: - - - -\ enabled: true - - - -\ synchronization: ... - - - - - - - -\ points: - - - -\ source: ... - + points: + source: ... ``` - - - - - - Exact syntax shall be selected during technology selection. +--- - - - - - -\\--- - - - - - - - -\\# 42. Point configuration - - - - - - +# 42. Point configuration Conceptual: - - - - - - ```text - - point: + id: LEVEL + name: Wet well level -\ id: LEVEL + source: + aquacomReference: ... + datatype: float -\ name: Wet well level + engineeringUnit: m + live: + enabled: true + history: + enabled: true + opcua: + enabled: true -\ source: - - - -\ aquacomReference: ... - - - - - - - -\ datatype: float - - - -\ engineeringUnit: m - - - - - - - -\ live: - - - -\ enabled: true - - - - - - - -\ history: - - - -\ enabled: true - - - - - - - -\ opcua: - - - -\ enabled: true - - - -\ historizing: true - - - - - - - -\ storage: - - - -\ enabled: true + historizing: true + storage: + enabled: true ``` +--- - - - - - -\\--- - - - - - - - -\\# 43. Configuration schema validation - - - - - - +# 43. Configuration schema validation Machine-readable schemas shall validate configuration before startup. - - - - - - Validation shall detect: +- duplicate station IDs; +- duplicate point IDs; +- invalid references; +- impossible polling intervals; +- unsupported transports; +- missing required storage; -\\- duplicate station IDs; - - - -\\- duplicate point IDs; - - - -\\- invalid references; - - - -\\- impossible polling intervals; - - - -\\- unsupported transports; - - - -\\- missing required storage; - - - -\\- invalid timezone; - - - -\\- duplicate OPC UA NodeIds; - - - -\\- invalid security configuration. - - - - +- invalid timezone; +- duplicate OPC UA NodeIds; +- invalid security configuration. Invalid configuration shall fail safely. +--- - - - - - -\\--- - - - - - - - -\\# 44. Secrets - - - - - - +# 44. Secrets Secrets shall be supplied through an external secret mechanism appropriate to the selected deployment environment. - - - - - - The repository shall never contain production: - - - - - - ```text - - passwords - - private keys - - API tokens - - certificate private keys - - database credentials - - ``` - - - - - - Example files shall use placeholders. +--- - - - - - -\\--- - - - - - - - -\\# 45. Diagnostics model - - - - - - +# 45. Diagnostics model Gateway-level diagnostics: - - - - - - ```text - - GatewayState - - Version - - ConfigurationVersion - - StartTime - - Uptime - - StationCount - - OnlineStationCount - - HistoricalQueueDepth - - SpoolDiskUsage - - OPCUAState - - ``` - - - - - - Per-station: - - - - - - ```text - - CommunicationState - - LastRequest - - LastResponse - - LastSuccessfulCommunication - - ConsecutiveFailures - - ProtocolErrors - - HistorySynchronizationState - - HistoryWatermark - - HistoricalBacklog - - ``` - - - - - - Per-storage destination: - - - - - - ```text - - State - - LastSuccessfulWrite - - LastFailure - - PendingRecords - - Retries - - Latency - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 46. Logging requirements - - - - - - +# 46. Logging requirements Structured logging shall be preferred. - - - - - - Every log record should support context equivalent to: - - - - - - ```text - - timestamp - - severity - - component - - stationId? - - pointId? - - operation? - - correlationId? - - message - - error? - - ``` - - - - - - Raw telegram logging shall be a separate debug facility. - - - - - - Production default shall not log every telegram. +--- - - - - - -\\--- - - - - - - - -\\# 47. Correlation IDs - - - - - - +# 47. Correlation IDs AquaCom transactions shall have a transaction/correlation identifier. - - - - - - Example flow: - - - - - - ```text - - AQ transaction 81245 + │ + ├── send request -\ │ + ├── receive response + ├── decode + ├── update values -\ ├── send request - - - -\ ├── receive response - - - -\ ├── decode - - - -\ ├── update values - - - -\ └── diagnostics - - + └── diagnostics ``` - - - - - - This ID should be propagated through relevant diagnostics. +--- - - - - - -\\--- - - - - - - - -\\# 48. Metrics - - - - - - +# 48. Metrics The implementation shall expose or internally maintain metrics equivalent to: - - - - - - ```text +aquacom_requests_total +aquacom_errors_total -aquacom\\\_requests\\\_total +aquacom_request_duration +station_online +live_values_processed -aquacom\\\_errors\\\_total +history_samples_received +history_samples_deduplicated +history_samples_pending -aquacom\\\_request\\\_duration +storage_writes_total +storage_write_errors +opcua_clients_connected -station\\\_online - - - -live\\\_values\\\_processed - - - -history\\\_samples\\\_received - - - -history\\\_samples\\\_deduplicated - - - -history\\\_samples\\\_pending - - - -storage\\\_writes\\\_total - - - -storage\\\_write\\\_errors - - - -opcua\\\_clients\\\_connected - - - -opcua\\\_history\\\_reads - - - -spool\\\_bytes - +opcua_history_reads +spool_bytes ``` - - - - - - Specific metrics technology shall not be prescribed. +--- - - - - - -\\--- - - - - - - - -\\# 49. Simulator - - - - - - +# 49. Simulator A simulator shall be treated as a first-class component. - - - - - - It shall support scripted station behavior. - - - - - - Required simulation scenarios: - - - - - - ```text - - normal live operation - - communication timeout - - connection loss - - malformed response - - bad checksum - - delayed response - - historical values - - historical gap - - duplicate history - - conflicting duplicate - - station clock error - - reconnect and backfill - - ``` - - - - - - The simulator shall not share parsing code with the production AquaCom decoder in ways that make protocol tests self-validating. +--- - - - - - -\\--- - - - - - - - -\\# 50. Test pyramid - - - - - - +# 50. Test pyramid The project shall implement: - - - - - - ```text + Acceptance + /\ -\ Acceptance + / \ + Integration + / \ -\ /\\\\ - - - -\ / \\\\ - - - -\ Integration - - - -\ / \\\\ - - - -\ / \\\\ - - - -\ Unit tests - + / \ + Unit tests ``` - - - - - - The majority of tests should be deterministic unit and component tests. +--- +# 51. Required unit tests - - - - -\\--- - - - - - - - -\\# 51. Required unit tests - - - - - - - -\\## AquaCom codec - - - - - - +## AquaCom codec ```text - - framing - - checksum - - address encoding - - telegram decoding - - invalid telegram rejection - - unexpected field handling - - ``` - - - - - - -\\## Canonical model - - - - - - +## Canonical model ```text - - quality mapping - - timestamp validation - - stable identifier handling - - ``` - - - - - - -\\## History - - - - - - +## History ```text - - deduplication - - watermark progression - - gap handling - - conflict handling - - spool acknowledgement - - ``` - - - - - - -\\## OPC UA - - - - - - +## OPC UA ```text - - NodeId mapping - - quality mapping - - timestamp mapping - - history result conversion - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 52. Required integration tests - - - - - - +# 52. Required integration tests At minimum: - - - - - - ```text - - Simulator → AquaCom engine - - - - - - AquaCom engine → Current State - - - - - - AquaCom history → Historical pipeline - - - - - - Historical pipeline → Repository - - - - - - Historical pipeline → multiple sinks - - - - - - Repository → OPC UA HistoryRead - - - - - - Current state → OPC UA subscription - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 53. Restart recovery tests - - - - - - +# 53. Restart recovery tests Automated tests shall cover gateway restart during: - - - - - - ```text - - normal operation - - history retrieval - - database outage - - spool backlog - - partial delivery - - reconnect recovery - - ``` - - - - - - Expected result: - - - - - - No acknowledged historical data is silently lost. +--- - - - - - -\\--- - - - - - - - -\\# 54. End-to-end mandatory acceptance test - - - - - - +# 54. End-to-end mandatory acceptance test Scenario: - - - - - - ```text - - T0: - - Station online. - - - - - - Gateway receives live level values. - - - - - - OPC UA client receives current values. - - - - - - Values are persisted historically. - - - - - - T1: - - Station communication is interrupted. - - - - - - Gateway marks communication degraded/bad. - - - - - - Current OPC UA value is not falsely reported as newly valid. - - - - - - T2: - - Station continues local historical recording. - - - - - - T3: - - Communication returns. - - - - - - Live acquisition resumes promptly. - - - - - - T4: - - Gateway identifies missing historical interval. - - - - - - Gateway retrieves historical records. - - - - - - T5: - - Historical records retain their original timestamps. - - - - - - They are delivered to all required storage destinations. - - - - - - T6: - - OPC UA current value remains the actual current process value. - - - - - - T7: - - OPC UA HistoryRead for T1–T4 returns recovered historical values. - - - - - - T8: - - Same historical interval is retrieved again. - - - - - - No unintended duplicate historical records are created. - - ``` - - - - - - The system shall not be considered production-capable until this test passes. +--- - - - - - -\\--- - - - - - - - -\\# 55. Security tests - - - - - - +# 55. Security tests Tests shall verify: - - - - - - ```text - - untrusted OPC UA client certificate rejected - - - - - - trusted client accepted - - - - - - insecure OPC UA endpoint disabled by production configuration - - - - - - invalid credentials rejected where user authentication is enabled - - - - - - private keys not logged - - - - - - unauthorized HistoryUpdate rejected - - - - - - configuration containing secret placeholders validated appropriately - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 56. Fault injection - - - - - - +# 56. Fault injection Integration testing shall permit fault injection including: - - - - - - ```text - - network disconnect - - packet loss - - delayed responses - - storage timeout - - storage exception - - disk-full simulation - - OPC UA client disconnect - - process restart - - corrupted local spool entry - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 57. CI requirements - - - - - - +# 57. CI requirements Every change shall run: - - - - - - ```text - - format/lint checks - - static validation - - unit tests - - integration tests suitable for CI - - configuration schema validation - - dependency/security checks where available - - ``` - - - - - - Protocol fixtures shall be testable without external network access. +--- +# 58. Pull request completion criteria - - - - -\\--- - - - - - - - -\\# 58. Pull request completion criteria - - - - - - - -Codex shall consider a development task complete only when: - - - - - - +The implementation agent shall consider a development task complete only when: ```text - - implementation exists - - - - - - tests exist - - - - - - tests pass - - - - - - relevant documentation is updated - - - - - - configuration examples are updated if required - - - - - - no known architectural boundary has been violated - - - - - - new protocol behavior has supporting evidence - - - - - - new failure modes have diagnostics - - - - - - no production secrets have been introduced - - ``` +Implementation-agent task prompts work best when scoped similarly to concrete issues or PRs with explicit files and expected outcomes. +--- +# 59. ExecPlan requirement +The implementation agent shall create an ExecPlan for work involving: +- new AquaCom operations; +- history synchronization; -Codex task prompts work best when scoped similarly to concrete issues or PRs with explicit files and expected outcomes. - - - - - - - -\\--- - - - - - - - -\\# 59. ExecPlan requirement - - - - - - - -Codex shall create an ExecPlan for work involving: - - - - - - - -\\- new AquaCom operations; - - - -\\- history synchronization; - - - -\\- storage architecture; - - - -\\- OPC UA namespace redesign; - - - -\\- security changes; - - - -\\- cross-module refactoring; - - - -\\- persistent-data migration; - - - -\\- changes affecting more than one architectural boundary. - +- storage architecture; +- OPC UA namespace redesign; +- security changes; +- cross-module refactoring; +- persistent-data migration; +- changes affecting more than one architectural boundary. ExecPlan structure: - - - - - - ```text - - Title - - - - - - Purpose - - - - - - Current state - - - - - - Desired state - - - - - - Constraints - - - - - - Files/components affected - - - - - - Implementation sequence - - - - - - Data model changes - - - - - - Failure handling - - - - - - Testing - - - - - - Migration/compatibility - - - - - - Progress - - - - - - Decisions - - - - - - Unexpected findings - - - - - - Acceptance evidence - - ``` - - - - - - The plan shall remain updated while implementation proceeds. +--- - - - - - -\\--- - - - - - - - -\\# 60. Development phases - - - - - - +# 60. Development phases Implementation shall proceed in controlled phases. +--- - - - - - -\\--- - - - - - - - -\\# Phase 0 – Repository and architecture - - - - - - +# Phase 0 – Repository and architecture Deliver: - - - - - - ```text - - AGENTS.md - - README.md - - ARCHITECTURE.md - - docs structure - - CI skeleton - - configuration skeleton - - ``` - - - - - - No AquaCom protocol implementation required. - - - - - - Acceptance: +- repository builds; +- automated test runner works; +- architecture dependencies documented. +--- - - -\\- repository builds; - - - -\\- automated test runner works; - - - -\\- architecture dependencies documented. - - - - - - - -\\--- - - - - - - - -\\# Phase 1 – Canonical core - - - - - - +# Phase 1 – Canonical core Implement: - - - - - - ```text - - StationId - - PointId - - CurrentValue - - HistoricalSample - - DataQuality - - AcquisitionSource - - configuration domain - - ``` - - - - - - Acceptance: +- unit tests; +- serialization tests where applicable; +- stable identifier behavior defined. +--- - - -\\- unit tests; - - - -\\- serialization tests where applicable; - - - -\\- stable identifier behavior defined. - - - - - - - -\\--- - - - - - - - -\\# Phase 2 – AquaCom transport abstraction - - - - - - +# Phase 2 – AquaCom transport abstraction Implement: - - - - - - ```text - - transport interface - - connection lifecycle - - timeouts - - retry primitives - - diagnostics - - ``` - - - - - - Implement only required initial transport. - - - - - - Acceptance: +- mock transport tests; +- connection-loss testing; +- reconnect testing. +--- - - -\\- mock transport tests; - - - -\\- connection-loss testing; - - - -\\- reconnect testing. - - - - - - - -\\--- - - - - - - - -\\# Phase 3 – AquaCom codec framework - - - - - - +# Phase 3 – AquaCom codec framework Implement: - - - - - - ```text - - frame abstraction - - message abstraction - - codec interfaces - - validation framework - - fixture framework - - ``` - - - - - - Only protocol elements with evidence may be implemented. - - - - - - Acceptance: - - - - - - ```text - - captured/documented telegram - - -\ ↓ - - + ↓ decoder - - -\ ↓ - - + ↓ expected structured representation - - ``` +--- - - - - - -\\--- - - - - - - - -\\# Phase 4 – AquaCom simulator - - - - - - +# Phase 4 – AquaCom simulator Implement simulator before depending on field equipment for normal development. - - - - - - Acceptance: +- successful transaction; +- timeout; +- malformed telegram; +- configurable process values. +--- - -\\- successful transaction; - - - -\\- timeout; - - - -\\- malformed telegram; - - - -\\- configurable process values. - - - - - - - -\\--- - - - - - - - -\\# Phase 5 – Station/session manager - - - - - - +# Phase 5 – Station/session manager Implement: - - - - - - ```text - - station lifecycle - - communication state - - poll scheduling - - retry policy - - diagnostics - - ``` - - - - - - Acceptance: +- multiple simulated stations; +- one station failing does not affect others. +--- - - - -\\- multiple simulated stations; - - - -\\- one station failing does not affect others. - - - - - - - -\\--- - - - - - - - -\\# Phase 6 – Live acquisition - - - - - - +# Phase 6 – Live acquisition Implement: - - - - - - ```text - - live AquaCom operation - - point mapping - - scaling - - canonical CurrentValue - - current-state cache - - ``` - - - - - - Acceptance: - - - - - - ```text - - simulated AquaCom - - -\ ↓ - - + ↓ CurrentValue - - ``` - - - - - - with correct value, quality and timestamps. +--- - - - - - -\\--- - - - - - - - -\\# Phase 7 – OPC UA live interface - - - - - - +# Phase 7 – OPC UA live interface Implement: - - - - - - ```text - - server - - namespace - - stable NodeIds - - current reads - - subscriptions - - quality - - timestamps - - security - - ``` - - - - - - Acceptance: - - - - - - An OPC UA test client can browse and subscribe to simulated AquaCom values. +--- - - - - - -\\--- - - - - - - - -\\# Phase 8 – Local historical repository - - - - - - +# Phase 8 – Local historical repository Implement the first queryable history repository. - - - - - - The selected technology may be embedded or external, but shall implement the defined repository abstraction. - - - - - - Acceptance: - - - - - - ```text - - store - - query interval - - query bounds - - restart persistence - - ``` +--- - - - - - -\\--- - - - - - - - -\\# Phase 9 – Historical pipeline - - - - - - +# Phase 9 – Historical pipeline Implement: - - - - - - ```text - - validation - - deduplication - - durable spool - - delivery - - repository integration - - ``` - - - - - - Acceptance: - - - - - - Multiple submissions of the same sample do not produce unintended duplicate records. +--- - - - - - -\\--- - - - - - - - -\\# Phase 10 – AquaCom historical acquisition - - - - - - +# Phase 10 – AquaCom historical acquisition Implement only after sufficient protocol evidence is available. - - - - - - Deliver: - - - - - - ```text - - history retrieval - - watermark - - reconnect synchronization - - backfill - - throttling - - partial recovery - - ``` - - - - - - Acceptance: - - - - - - The simulator's missing time interval is recovered after reconnection. +--- - - - - - -\\--- - - - - - - - -\\# Phase 11 – OPC UA Historical Access - - - - - - +# Phase 11 – OPC UA Historical Access Implement: - - - - - - ```text - - HistoryRead - - raw values - - timestamps - - quality - - continuation - - ``` - - - - - - Acceptance: - - - - - - An OPC UA HA client can retrieve both originally stored and recovered historical values. +--- - - - - - -\\--- - - - - - - - -\\# Phase 12 – Multiple storage destinations - - - - - - +# Phase 12 – Multiple storage destinations Implement storage sink abstraction plus second sink implementation. - - - - - - Acceptance: - - - - - - ```text - - one HistoricalSample + │ + ├── Repository A -\ │ - - - -\ ├── Repository A - - - -\ └── Storage B - - + └── Storage B ``` - - - - - - and failure of Storage B does not stop Repository A. +--- - - - - - -\\--- - - - - - - - -\\# Phase 13 – Store-and-forward resilience - - - - - - +# Phase 13 – Store-and-forward resilience Implement: - - - - - - ```text - - persistent spool - - destination acknowledgements - - retry - - backpressure - - disk limits - - ``` - - - - - - Acceptance: - - - - - - A storage destination may be offline while history accumulates and receive the backlog when restored. +--- - - - - - -\\--- - - - - - - - -\\# Phase 14 – Production hardening - - - - - - +# Phase 14 – Production hardening Implement and validate: - - - - - - ```text - - security - - resource limits - - performance - - operations documentation - - backup/recovery - - deployment - - certificate handling - - structured diagnostics - - long-duration tests - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 61. Codex task sizing - - - - - - +# 61. Implementation agent task sizing Implementation phases shall be divided into bounded tasks rather than requesting the complete gateway in one prompt. - - - - - - Preferred task structure: - - - - - - ```text - - Goal - - - - - - Relevant files - - - - - - Required behavior - - - - - - Constraints - - - - - - Tests - - - - - - Acceptance criteria - - ``` - - - - - - Example: - - - - - - ```text - - Implement the HistoricalSample deduplication service. - - - - - - Relevant: - - src/history - - docs/architecture/historical-data.md - - - - - - Requirements: +- identity based on StationId + PointId + SourceTimestamp +- conflicting value must be detected -\\- identity based on StationId + PointId + SourceTimestamp - - - -\\- conflicting value must be detected - - - -\\- default conflict policy REJECT\\\_CONFLICT - - - - - - +- default conflict policy REJECT_CONFLICT Do not: +- introduce database-specific logic +- modify AquaCom codec -\\- introduce database-specific logic - - - -\\- modify AquaCom codec - - - -\\- modify OPC UA - - - - - - +- modify OPC UA Add unit tests covering: +- first sample +- exact duplicate -\\- first sample - - - -\\- exact duplicate - - - -\\- conflicting duplicate - - - -\\- different point - - - -\\- different timestamp +- conflicting duplicate +- different point +- different timestamp ``` +--- +# 62. Rule for incomplete requirements - - - - -\\--- - - - - - - - -\\# 62. Codex rule for incomplete requirements - - - - - - - -When Codex encounters an implementation ambiguity, it shall classify it as: - - - - - - +When the implementation agent encounters an implementation ambiguity, it shall classify it as: ```text - - A. Local implementation detail - - B. Architectural decision - - C. AquaCom protocol uncertainty - - D. Safety/security uncertainty - - ``` - - - - - - For A: - - - - - - Choose the simplest implementation consistent with architecture. - - - - - - For B: - - - - - - Record the decision in the current ExecPlan. - - - - - - For C: - - - - - - Do not guess. Add a protocol gap. - - - - - - For D: - - - - - - Choose the safer behavior and document the decision. +--- - - - - - -\\--- - - - - - - - -\\# 63. AquaCom source traceability - - - - - - +# 63. AquaCom source traceability Every implemented protocol operation shall have traceability metadata in documentation. - - - - - - Example: - - - - - - ```text - - Operation: HistoricalRead-X - - - - - - Evidence: - - AquaCom specification section 8.3 - - Capture fixture AQ-HIST-004 - - Validated against ControllerType-A - - - - - - Implemented: - - yes - - - - - - Tests: - - -protocol/history\\\_read\\\_004 - - +protocol/history_read_004 ``` - - - - - - This traceability shall make it possible to audit why the implementation behaves as it does. +--- - - - - - -\\--- - - - - - - - -\\# 64. Protocol capture handling - - - - - - +# 64. Protocol capture handling Production captures used as test fixtures shall be: +- sanitized; +- stripped of unnecessary network identifiers; +- documented; - - - -\\- sanitized; - - - -\\- stripped of unnecessary network identifiers; - - - -\\- documented; - - - -\\- immutable once used as reference fixtures. - - - - - - +- immutable once used as reference fixtures. Fixture modification requires justification. - - - - - - Raw captures shall not automatically be committed if they contain sensitive operational information. +--- - - - - - -\\--- - - - - - - - -\\# 65. Storage migration rule - - - - - - +# 65. Storage migration rule Storage schemas shall be versioned. - - - - - - Schema changes shall have explicit migrations. - - - - - - Application startup shall not silently destroy incompatible historical data. - - - - - - Destructive migration requires explicit administrative action. +--- - - - - - -\\--- - - - - - - - -\\# 66. Configuration versioning - - - - - - +# 66. Configuration versioning The configuration model shall include a schema version. - - - - - - Example: - - - - - - ```text - - configVersion: 1 - - ``` - - - - - - Future incompatible configuration changes shall provide validation and, where reasonable, migration tooling. +--- - - - - - -\\--- - - - - - - - -\\# 67. Software version exposure - - - - - - +# 67. Software version exposure Gateway software version shall be available through: +- startup logs; +- diagnostics; - - - - -\\- startup logs; - - - -\\- diagnostics; - - - -\\- OPC UA diagnostics namespace. - - - - - - +- OPC UA diagnostics namespace. Build metadata should include where possible: - - - - - - ```text - - version - - build identifier - - source revision - - configuration schema version - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 68. Compatibility strategy - - - - - - +# 68. Compatibility strategy The gateway shall distinguish: - - - - - - ```text - - protocol compatibility - - configuration compatibility - - storage compatibility - - OPC UA namespace compatibility - - ``` - - - - - - Changes to any of these shall be considered independently. - - - - - - Stable OPC UA NodeIds are particularly important and shall not change unnecessarily between releases. +--- - - - - - -\\--- - - - - - - - -\\# 69. Performance validation - - - - - - +# 69. Performance validation Before production acceptance, test: - - - - - - ```text - - expected station count - - - - - - expected point count - - - - - - normal polling interval - - - - - - normal historical rate - - - - - - large backfill - - - - - - simultaneous OPC UA subscriptions - - - - - - large HistoryRead - - - - - - storage unavailable - - - - - - maximum expected spool backlog - - ``` - - - - - - The purpose is to establish capacity limits rather than merely demonstrate successful execution. +--- - - - - - -\\--- - - - - - - - -\\# 70. Live-priority scheduling - - - - - - +# 70. Live-priority scheduling AquaCom scheduling shall conceptually prioritize: - - - - - - ```text +1. connection/health-critical transactions +2. current/live acquisition -1\\. connection/health-critical transactions +3. alarm/event retrieval when implemented +4. requested configuration operations +5. historical recovery -2\\. current/live acquisition - - - -3\\. alarm/event retrieval when implemented - - - -4\\. requested configuration operations - - - -5\\. historical recovery - - - -6\\. low-priority maintenance - - +6. low-priority maintenance ``` - - - - - - Historical recovery shall be throttled if required to maintain live acquisition responsiveness. +--- - - - - - -\\--- - - - - - - - -\\# 71. Memory and queue limits - - - - - - +# 71. Memory and queue limits Every queue shall be bounded. - - - - - - No component may create an unbounded: - - - - - - ```text - - message queue - - history list - - retry list - - client response - - log buffer - - ``` - - - - - - Queue-full behavior shall be explicitly defined. +--- - - - - - -\\--- - - - - - - - -\\# 72. Graceful shutdown - - - - - - +# 72. Graceful shutdown The application shall support controlled shutdown. - - - - - - Order: - - - - - - ```text - - stop accepting administration changes - - -\ ↓ - - + ↓ stop scheduling new acquisition - - -\ ↓ - - + ↓ complete/cancel active AquaCom transactions - - -\ ↓ - - + ↓ persist watermarks - - -\ ↓ - - + ↓ flush durable state - - -\ ↓ - - + ↓ stop OPC UA - - -\ ↓ - - + ↓ close storage - - -\ ↓ - - + ↓ terminate - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 73. Crash recovery - - - - - - +# 73. Crash recovery Following unclean shutdown: - - - - - - ```text - - load persistent state - - -\ ↓ - - + ↓ validate spool - - -\ ↓ - - + ↓ recover pending deliveries - - -\ ↓ - - + ↓ restore synchronization watermarks - - -\ ↓ - - + ↓ reconnect stations - - -\ ↓ - - + ↓ resume live operation - - -\ ↓ - - + ↓ continue required historical synchronization - - ``` - - - - - - Live communication shall normally resume before large backfill jobs. +--- - - - - - -\\--- - - - - - - - -\\# 74. Initial production scope - - - - - - +# 74. Initial production scope The first production release shall include: - - - - - - ```text - - AquaCom transport - - AquaCom protocol framework - - verified live-value acquisition - - verified historical acquisition - - multiple stations - - canonical data model - - live cache - - historical pipeline - - deduplication - - durable spool - - at least one history repository - - storage adapter framework - - at least two independently configurable storage targets - - OPC UA Data Access - - OPC UA Historical Access - - OPC UA security - - stable NodeIds - - diagnostics - - structured logging - - simulator - - automated tests - - restart recovery - - configuration validation - - ``` +--- - - - - - -\\--- - - - - - - - -\\# 75. Explicitly excluded from first production release - - - - - - +# 75. Explicitly excluded from first production release Unless separately approved: - - - - - - ```text - - AquaCom remote pump control - - AquaCom setpoint write - - OPC UA control writes - - OPC UA Methods for process control - - OPC UA HistoryUpdate - - automatic historian correction - - clustered high availability - - automatic protocol discovery - - ``` - - - - - - Read-only process integration shall be established before remote-control functionality is introduced. +--- - - - - - -\\--- - - - - - - - -\\# 76. Future process-control architecture - - - - - - +# 76. Future process-control architecture If control is later implemented, it shall use a separate command pipeline: - - - - - - ```text - - OPC UA + │ - -\ │ - - - -\ ▼ - - + ▼ authorization + │ - -\ │ - - - -\ ▼ - - + ▼ command validation + │ - -\ │ - - - -\ ▼ - - + ▼ command service + │ - -\ │ - - - -\ ▼ - - + ▼ AquaCom control operation + │ - -\ │ - - - -\ ▼ - - + ▼ confirmation/readback + │ - -\ │ - - - -\ ▼ - - + ▼ audit - - ``` - - - - - - Direct mapping: - - - - - - ```text - - OPC UA write → raw AquaCom telegram - - ``` - - - - - - shall not be permitted. +--- - - - - - -\\--- - - - - - - - -\\# 77. Definition of Done - - - - - - +# 77. Definition of Done The gateway implementation is complete for production evaluation when: +1. All mandatory Phase 0–14 features are implemented. +2. No implemented AquaCom operation depends on undocumented guessed protocol behavior. +3. Current data is available through OPC UA. +4. Historical data is available through OPC UA Historical Access. +5. AquaCom history backfill does not alter current values. +6. Historical source timestamps are preserved. -1\\. All mandatory Phase 0–14 features are implemented. +7. Gateway restart does not lose acknowledged history. +8. Temporary storage outage does not immediately cause historical loss. +9. Multiple storage destinations operate independently. +10. OPC UA NodeIds are stable. +11. OPC UA security is enabled by default. +12. Simulator-based end-to-end acceptance tests pass. +13. Tests against real AquaCom equipment or validated traffic have passed. -2\\. No implemented AquaCom operation depends on undocumented guessed protocol behavior. - - - - - - - -3\\. Current data is available through OPC UA. - - - - - - - -4\\. Historical data is available through OPC UA Historical Access. - - - - - - - -5\\. AquaCom history backfill does not alter current values. - - - - - - - -6\\. Historical source timestamps are preserved. - - - - - - - -7\\. Gateway restart does not lose acknowledged history. - - - - - - - -8\\. Temporary storage outage does not immediately cause historical loss. - - - - - - - -9\\. Multiple storage destinations operate independently. - - - - - - - -10\\. OPC UA NodeIds are stable. - - - - - - - -11\\. OPC UA security is enabled by default. - - - - - - - -12\\. Simulator-based end-to-end acceptance tests pass. - - - - - - - -13\\. Tests against real AquaCom equipment or validated traffic have passed. - - - - - - - -14\\. Operational deployment and recovery documentation exists. - - - - - - - -15\\. Remaining protocol gaps are explicitly documented. - - - - - - - -\\--- - - - - - - - -\\# 78. Implementation objective - - - +14. Operational deployment and recovery documentation exists. +15. Remaining protocol gaps are explicitly documented. +--- +# 78. Implementation objective The resulting software shall behave as an integration platform: - - - - - - ```text + AquaCom + │ -\ AquaCom + ▼ + ┌───────────────────────┐ + │ AquaCom Gateway │ -\ │ + │ │ + │ Protocol │ + │ Acquisition │ -\ ▼ + │ State │ + │ History recovery │ + │ Normalization │ -\ ┌───────────────────────┐ + │ Diagnostics │ + └───────┬───────┬───────┘ + │ │ -\ │ AquaCom Gateway │ + LIVE │ │ HISTORY + │ │ + ▼ ▼ -\ │ │ + OPC UA Historical pipeline + │ │ + │ ├─────────► Repository -\ │ Protocol │ + │ ├─────────► Historian + │ ├─────────► Database + │ └─────────► Archive -\ │ Acquisition │ - - - -\ │ State │ - - - -\ │ History recovery │ - - - -\ │ Normalization │ - - - -\ │ Diagnostics │ - - - -\ └───────┬───────┬───────┘ - - - -\ │ │ - - - -\ LIVE │ │ HISTORY - - - -\ │ │ - - - -\ ▼ ▼ - - - -\ OPC UA Historical pipeline - - - -\ │ │ - - - -\ │ ├─────────► Repository - - - -\ │ ├─────────► Historian - - - -\ │ ├─────────► Database - - - -\ │ └─────────► Archive - - - -\ │ - - - -\ └───────────────► OPC UA HA - + │ + └───────────────► OPC UA HA ``` - - - - - - The implementation shall remain usable even if: +- the SCADA platform changes; +- the historical database changes; +- additional storage destinations are introduced; +- AquaCom equipment is gradually replaced; - - -\\- the SCADA platform changes; - - - -\\- the historical database changes; - - - -\\- additional storage destinations are introduced; - - - -\\- AquaCom equipment is gradually replaced; - - - -\\- additional northbound interfaces are introduced. - - - - - - +- additional northbound interfaces are introduced. The gateway's permanent value shall therefore be the normalized information and integration boundary, not merely its OPC UA endpoint. +--- +# 79. First implementation execution sequence - - - - -\\--- - - - - - - - -\\# 79. First Codex execution sequence - - - - - - - -The recommended first Codex tasks are: - - - - - - +The recommended first implementation tasks are: ```text - - Task 001 - - Create repository skeleton and architecture documents. - - - - - - Task 002 - - Create core domain models and stable identifier rules. - - - - - - Task 003 - - Create configuration model and schema validation. - - - - - - Task 004 - - Create AquaCom transport abstraction and mock transport. - - - - - - Task 005 - - Create AquaCom protocol codec framework without inventing protocol fields. - - - - - - Task 006 - - Create protocol fixture/test-vector framework. - - - - - - Task 007 - - Create AquaCom simulator architecture. - - - - - - Task 008 - - Implement first documented live AquaCom operation. - - - - - - Task 009 - - Implement Station Manager and live acquisition. - - - - - - Task 010 - - Implement current-state service. - - - - - - Task 011 - - Implement OPC UA live-value interface. - - - - - - Task 012 - - Implement historical repository abstraction. - - - - - - Task 013 - - Implement historical pipeline and deduplication. - - - - - - Task 014 - - Implement durable spool. - - - - - - Task 015 - - Implement first verified AquaCom historical operation. - - - - - - Task 016 - - Implement backfill state machine. - - - - - - Task 017 - - Implement OPC UA Historical Access. - - - - - - Task 018 - - Implement second storage destination. - - - - - - Task 019 - - Implement full reconnect/backfill acceptance test. - - - - - - Task 020 - - Production hardening. - - ``` - - - - - - Each task shall result in a working repository state rather than leaving the main branch intentionally uncompilable or untestable. +--- - - - - - -\\--- - - - - - - - -\\# 80. Primary instruction to Codex - - - - - - +# 80. Primary instruction to the implementation agent When implementing this system, prioritize correctness, traceability and recoverability over speed of implementation. - - - - - - In particular: - - - - - - > Never compensate for missing AquaCom documentation by inventing plausible protocol behavior. - - - - - - Build the architecture around the unknown portions first. - - - - - - Add real AquaCom behavior only as verified protocol knowledge becomes available. - - - - - - Every protocol feature shall be backed by evidence and automated tests. - - - - - - Every architectural component shall be replaceable behind its documented interface. - - - - - - Every historical sample shall retain its original process timestamp. - - - - - - And no recovered historical value shall ever masquerade as a new live process value. - - - - diff --git a/docs/product/software-specification.md b/docs/product/software-specification.md index de6eba9..d6c91ef 100644 --- a/docs/product/software-specification.md +++ b/docs/product/software-specification.md @@ -1,451 +1,354 @@ -\# Software Specification – AquaCom Integration Gateway +# Software Specification – AquaCom Integration Gateway +**Document type:** Software Requirements and Architecture Specification +**Status:** Draft -\*\*Document type:\*\* Software Requirements and Architecture Specification +**Version:** 0.1 -\*\*Status:\*\* Draft +**System name:** AquaCom Integration Gateway -\*\*Version:\*\* 0.1 +**Primary interfaces:** AquaCom, OPC UA -\*\*System name:\*\* AquaCom Integration Gateway +**Implementation language:** Not specified -\*\*Primary interfaces:\*\* AquaCom, OPC UA - -\*\*Implementation language:\*\* Not specified - -\*\*Database/storage technology:\*\* Not specified - - - -\--- - - - -\# 1. Purpose +**Database/storage technology:** Not specified +--- +# 1. Purpose The purpose of the AquaCom Integration Gateway is to provide a modern integration layer between legacy AquaCom-compatible field equipment and higher-level OT/IT systems. - - The gateway shall communicate with AquaCom devices and provide: +- current/live process values; +- historical process values retrieved from AquaCom devices; -\- current/live process values; +- normalized timestamps and data quality; -\- historical process values retrieved from AquaCom devices; +- OPC UA access to current values; -\- normalized timestamps and data quality; +- OPC UA Historical Access to historical values; -\- OPC UA access to current values; - -\- OPC UA Historical Access to historical values; - -\- configurable storage of historical data in one or several independent storage systems; - -\- controlled buffering and backfill following communication interruptions; - -\- diagnostics concerning AquaCom communication, history synchronization, storage and OPC UA operation. +- configurable storage of historical data in one or several independent storage systems; +- controlled buffering and backfill following communication interruptions; +- diagnostics concerning AquaCom communication, history synchronization, storage and OPC UA operation. The architecture shall separate AquaCom-specific communication from OPC UA and historical storage so that each component can evolve independently. - - The gateway shall not require an OPC UA client to understand AquaCom. +--- +# 2. Design objectives -\--- +### Requirement keywords +The key words in this document are to be interpreted as follows: +- **shall** / **must** — an absolute requirement of this specification; +- **shall not** — an absolute prohibition; +- **should** — a strong recommendation; deviations require a documented reason (an ExecPlan entry is sufficient); +- **may** / **can** — truly optional behavior. -\# 2. Design objectives - - +Hedged wording such as "where practical" or "where applicable" does not weaken a **shall**; it delegates the boundary decision to the implementation, which must then document where the requirement applies and where it does not. The system shall be designed around the following principles: +1. **Protocol isolation** + AquaCom telegram handling shall be isolated from OPC UA, persistence and application logic. -1\. \*\*Protocol isolation\*\* +2. **Canonical data model** - AquaCom telegram handling shall be isolated from OPC UA, persistence and application logic. + AquaCom information shall be transformed into an internal protocol-independent representation before being exposed or stored. +3. **Separation of live and historical processing** + Historical samples retrieved from an RTU shall not be replayed as current values. -2\. \*\*Canonical data model\*\* +4. **Source timestamp preservation** - AquaCom information shall be transformed into an internal protocol-independent representation before being exposed or stored. + Timestamps originating from the field device shall be retained throughout the system. +5. **Storage independence** + Historical information shall not be coupled to a specific database technology. -3\. \*\*Separation of live and historical processing\*\* +6. **Multiple storage targets** - Historical samples retrieved from an RTU shall not be replayed as current values. + The gateway shall support simultaneous writing to zero, one or several configured historical destinations. +7. **OPC UA as an interface, not the internal data model** + OPC UA shall expose information from the gateway but shall not dictate the internal architecture. -4\. \*\*Source timestamp preservation\*\* +8. **Recoverability** - Timestamps originating from the field device shall be retained throughout the system. + Temporary failure of a communication channel, storage destination or OPC UA client shall not result in unnecessary loss of recoverable historical data. +9. **Deterministic and testable protocol implementation** + AquaCom encoding, decoding and state handling shall be independently testable. -5\. \*\*Storage independence\*\* +10. **Secure-by-default operation** - Historical information shall not be coupled to a specific database technology. + OPC UA security and gateway administration shall use authenticated and protected communication by default. +--- +# 3. Scope -6\. \*\*Multiple storage targets\*\* - - The gateway shall support simultaneous writing to zero, one or several configured historical destinations. - - - -7\. \*\*OPC UA as an interface, not the internal data model\*\* - - OPC UA shall expose information from the gateway but shall not dictate the internal architecture. - - - -8\. \*\*Recoverability\*\* - - Temporary failure of a communication channel, storage destination or OPC UA client shall not result in unnecessary loss of recoverable historical data. - - - -9\. \*\*Deterministic and testable protocol implementation\*\* - - AquaCom encoding, decoding and state handling shall be independently testable. - - - -10\. \*\*Secure-by-default operation\*\* - - OPC UA security and gateway administration shall use authenticated and protected communication by default. - - - -\--- - - - -\# 3. Scope - - - -\## 3.1 Included - - +## 3.1 Included The initial system scope includes: +- AquaCom connection management; +- AquaCom telegram encoding and decoding; -\- AquaCom connection management; +- configuration of AquaCom stations; -\- AquaCom telegram encoding and decoding; +- acquisition of live/current values; -\- configuration of AquaCom stations; +- acquisition of available AquaCom historical information; -\- acquisition of live/current values; +- detection and retrieval of historical data after communication interruptions; -\- acquisition of available AquaCom historical information; +- normalization of AquaCom values into a canonical model; -\- detection and retrieval of historical data after communication interruptions; +- live-value cache; -\- normalization of AquaCom values into a canonical model; +- historical-data processing; -\- live-value cache; +- duplicate detection; -\- historical-data processing; +- local buffering; -\- duplicate detection; +- one or multiple storage adapters; -\- local buffering; +- OPC UA Server; -\- one or multiple storage adapters; +- OPC UA Data Access; -\- OPC UA Server; +- OPC UA Historical Access; -\- OPC UA Data Access; +- communication status; -\- OPC UA Historical Access; +- gateway diagnostics; -\- communication status; +- logging; -\- gateway diagnostics; +- configuration management; -\- logging; - -\- configuration management; - -\- auditability; - -\- health monitoring. - - - -\## 3.2 Not mandatory in the initial implementation +- auditability; +- health monitoring. +## 3.2 Not mandatory in the initial implementation The following shall be considered extensions unless separately specified: +- AquaCom machine control; +- AquaCom setpoint modification; -\- AquaCom machine control; +- OPC UA write access to control process equipment; -\- AquaCom setpoint modification; +- OPC UA Alarms & Conditions; -\- OPC UA write access to control process equipment; +- historian aggregates; -\- OPC UA Alarms \& Conditions; +- report-generation services; -\- historian aggregates; +- HTTP/REST interfaces; -\- report-generation services; - -\- HTTP/REST interfaces; - -\- MQTT publishing; - -\- clustered gateway operation; - -\- active/standby gateway redundancy. +- MQTT publishing; +- clustered gateway operation; +- active/standby gateway redundancy. The architecture shall not prevent these capabilities from being implemented later. +--- - -\--- - - - -\# 4. System context - - +# 4. System context The intended conceptual architecture is: - - ```text - AquaCom field systems + AquaCom field systems - │ + │ - ┌───────────┴───────────┐ + ┌───────────┴───────────┐ - │ │ + │ │ - Current values Stored history + Current values Stored history - │ │ + │ │ - ▼ ▼ + ▼ ▼ - ┌────────────────────────────────────────┐ + ┌────────────────────────────────────────┐ - │ AquaCom Protocol Layer │ + │ AquaCom Protocol Layer │ - │ │ + │ │ - │ Transport │ + │ Transport │ - │ Telegram codec │ + │ Telegram codec │ - │ Session/state management │ + │ Session/state management │ - │ Station capability model │ + │ Station capability model │ - └───────────────────┬────────────────────┘ + └───────────────────┬────────────────────┘ - │ + │ - ▼ + ▼ - ┌────────────────────────────────────────┐ + ┌────────────────────────────────────────┐ - │ Canonical Gateway Model │ + │ Canonical Gateway Model │ - └──────────┬───────────────────┬─────────┘ + └──────────┬───────────────────┬─────────┘ - │ │ + │ │ - Current state Historical pipeline + Current state Historical pipeline - │ │ + │ │ - │ ┌────────┴───────────┐ + │ ┌────────┴───────────┐ - │ │ │ + │ │ │ - │ History store Storage adapters + │ History store Storage adapters - │ │ │ + │ │ │ - │ │ ┌──────┼───────┐ + │ │ ┌──────┼───────┐ - │ │ │ │ │ + │ │ │ │ │ - │ │ DB File Historian + │ │ DB File Historian - │ │ + │ │ - ▼ ▼ + ▼ ▼ - ┌────────────────────┐ + ┌────────────────────┐ - │ OPC UA Server │ + │ OPC UA Server │ - │ │ + │ │ - │ Data Access │ + │ Data Access │ - │ Historical Access │ + │ Historical Access │ - └─────────┬──────────┘ + └─────────┬──────────┘ - │ + │ - ▼ + ▼ - SCADA / clients + SCADA / clients ``` - - The OPC UA server and storage mechanisms shall consume the canonical gateway model rather than AquaCom telegrams directly. +--- +# 5. Logical components -\--- - - - -\# 5. Logical components - - - -\## 5.1 AquaCom Transport Layer - - +## 5.1 AquaCom Transport Layer The transport layer shall provide physical or network communication required by the target AquaCom implementation. - - It shall be responsible for: +- establishing communication; +- maintaining connections; -\- establishing communication; +- reconnecting following failure; -\- maintaining connections; +- sending telegrams; -\- reconnecting following failure; - -\- sending telegrams; - -\- receiving telegrams; - -\- detecting transport timeout; - -\- exposing transport diagnostics. +- receiving telegrams; +- detecting transport timeout; +- exposing transport diagnostics. Transport handling shall not interpret process values. - - The architecture shall allow multiple AquaCom transports to be implemented without changing higher-level application logic. +A transport may be **shared by several stations** (for example a serial multi-drop bus or a modem pool serving multiple RTUs). The transport abstraction shall therefore support arbitration between station sessions using the same physical medium, and the Station Manager shall not assume exclusive one-to-one ownership between a station and a transport instance. For point-to-point transports such as TCP, one transport per station remains the normal case. +Whether AquaCom stations may share one physical medium is a protocol question and shall be answered from verified protocol documentation before multi-drop operation is enabled (see `docs/protocol/`). -\--- - - - -\# 6. AquaCom Protocol Engine - +--- +# 6. AquaCom Protocol Engine The AquaCom Protocol Engine shall implement the AquaCom application protocol. - - Its responsibilities shall include: +- telegram framing; +- address handling; -\- telegram framing; +- checksum validation; -\- address handling; +- telegram validation; -\- checksum validation; +- request generation; -\- telegram validation; +- response decoding; -\- request generation; +- error-response interpretation; -\- response decoding; - -\- error-response interpretation; - -\- mapping protocol operations to logical operations; - -\- protocol timeout handling; - -\- detection of malformed telegrams. +- mapping protocol operations to logical operations; +- protocol timeout handling; +- detection of malformed telegrams. The protocol implementation shall not directly: +- write database records; +- create OPC UA nodes; -\- write database records; - -\- create OPC UA nodes; - -\- modify OPC UA values; - -\- make assumptions regarding graphical presentation. - +- modify OPC UA values; +- make assumptions regarding graphical presentation. The complete AquaCom protocol implementation shall be developed against an authoritative protocol specification and validated using captured communication from actual AquaCom equipment where possible. +--- - -\--- - - - -\# 7. Station Manager - - +# 7. Station Manager Each AquaCom station shall have a logical station instance. - - The Station Manager shall maintain information including: - - | Attribute | Description | |---|---| @@ -470,46 +373,28 @@ The Station Manager shall maintain information including: | Configuration version | Version of station definition | - - Station configuration shall be independent of OPC UA NodeIds. +--- - -\--- - - - -\# 8. Station capability and point definition - - +# 8. Station capability and point definition The system shall maintain a definition of the information available from each station. - - This definition may originate from: +- AquaCom ATF files; +- manually supplied configuration; -\- AquaCom ATF files; - -\- manually supplied configuration; - -\- imported engineering data; - -\- another machine-readable configuration source. - +- imported engineering data; +- another machine-readable configuration source. The point definition shall contain sufficient information to map raw AquaCom information into the canonical gateway model. - - A point definition should support at least: - - | Field | Purpose | |---|---| @@ -542,160 +427,114 @@ A point definition should support at least: | Description | Engineering description | - - Changes to display names shall not automatically change stable identifiers. +--- - -\--- - - - -\# 9. Canonical data model - - +# 9. Canonical data model AquaCom data shall be converted into a canonical representation before further processing. - - -\## 9.1 Current value - - +## 9.1 Current value A current process value shall conceptually contain: - - ```text CurrentValue - StationId + StationId - PointId + PointId - Value + Value - DataType + DataType - Quality + Quality - SourceTimestamp + SourceTimestamp - ReceivedTimestamp + ReceivedTimestamp - UpdatedTimestamp + UpdatedTimestamp - Source + Source ``` - - -\### SourceTimestamp - - +### SourceTimestamp `SourceTimestamp` represents the time associated with the process value at its source when such information is available. - - -\### ReceivedTimestamp - - +### ReceivedTimestamp `ReceivedTimestamp` represents the time when the gateway received the value. - - -\### UpdatedTimestamp - - +### UpdatedTimestamp `UpdatedTimestamp` represents the time when the current gateway state was updated. - - The distinction between these timestamps shall be maintained internally. - - OPC UA itself distinguishes source and server timestamps in its `DataValue` model, making this mapping appropriate. +--- - -\--- - - - -\# 10. Historical sample model - - +# 10. Historical sample model Historical process data shall use a distinct model: - - ```text HistoricalSample - StationId + StationId - PointId + PointId - SourceTimestamp + SourceTimestamp - ReceivedTimestamp + ReceivedTimestamp - Value + Value - DataType + DataType - Quality + Quality - AcquisitionSource + AcquisitionSource - AcquisitionId + AcquisitionId ``` - - `AcquisitionSource` shall identify how the value entered the gateway, for example: - - ```text LIVE -AQUACOM\_HISTORY +AQUACOM_HISTORY -AQUACOM\_REPORT +AQUACOM_REPORT IMPORT RECOVERY +SIMULATOR + ``` - - The exact values are implementation-defined. - - Historical samples shall never become current values solely because they have just been received. - +A record that arrives through a historical acquisition path but carries no usable `SourceTimestamp` shall be **quarantined with diagnostics**: rejected from normal process history, recorded distinctly for investigation, and never silently stored with a fabricated or gateway-generated timestamp. Rejected history acquisition may then be retried once the timestamp problem is resolved. Fabricating a `SourceTimestamp` solely to make storage convenient is prohibited. Example: - - ```text Gateway time: 18:31 @@ -706,208 +545,150 @@ Value: 2.31 m ``` - - The sample shall be stored as a value from 14:05 and shall not cause a normal live OPC UA subscription to report that the current process value is 2.31 m. +--- - -\--- - - - -\# 11. Live-value processing - - +# 11. Live-value processing The live-value subsystem shall maintain the latest known state of every configured current point. - - A live update shall perform the following logical sequence: - - ```text AquaCom telegram - │ + │ - ▼ + ▼ Protocol decode - │ + │ - ▼ + ▼ Point mapping - │ + │ - ▼ + ▼ Type/scaling validation - │ + │ - ▼ + ▼ Canonical CurrentValue - │ + │ - ├─────────► Current-state cache + ├─────────► Current-state cache - │ + │ - ├─────────► OPC UA + ├─────────► OPC UA - │ + │ - └─────────► Historical pipeline + └─────────► Historical pipeline - if configured + if configured ``` - - Acquisition of live values and historization of those values shall be separate functions. - - A point may therefore be: +- exposed live but not stored; +- stored but not exposed through OPC UA; -\- exposed live but not stored; +- both exposed and stored; -\- stored but not exposed through OPC UA; - -\- both exposed and stored; - -\- disabled completely. - - - -\--- - - - -\# 12. Historical acquisition +- disabled completely. +--- +# 12. Historical acquisition The gateway shall contain a History Acquisition Manager responsible for obtaining historical information available in AquaCom field equipment. - - The component shall support: +- normal periodic history retrieval; +- history retrieval following reconnect; -\- normal periodic history retrieval; +- configurable historical synchronization; -\- history retrieval following reconnect; +- tracking the last successfully synchronized time; -\- configurable historical synchronization; +- detection of gaps where possible; -\- tracking the last successfully synchronized time; +- recovery/backfill; -\- detection of gaps where possible; - -\- recovery/backfill; - -\- duplicate suppression; - -\- partial transfer recovery; - -\- rate limitation so history retrieval does not prevent live communication. +- duplicate suppression; +- partial transfer recovery; +- rate limitation so history retrieval does not prevent live communication. A station shall maintain a persistent synchronization watermark where applicable. - - Conceptually: - - ```text Last complete history: 2026-10-05 12:00 - - Communication lost: 12:03 - - Communication restored: 17:24 - - Gateway determines missing interval: 12:00 → 17:24 - - Gateway requests AquaCom history. - - Recovered historical values - │ + │ - ▼ + ▼ Historical pipeline - │ + │ - ▼ + ▼ Historian/storage ``` - - Live polling shall normally take priority over large backfill operations. +--- - -\--- - - - -\# 13. Duplicate handling - - +# 13. Duplicate handling Historical processing shall be idempotent. - - Receiving the same historical sample multiple times shall not normally result in multiple logical records. - - A logical historical identity should be derivable from information equivalent to: - - ```text StationId @@ -918,50 +699,32 @@ SourceTimestamp ``` - - Where AquaCom provides additional sequence or sample identifiers, these may form part of the identity. - - The implementation shall define behavior for the unusual situation where two different values are supplied for the same logical historical sample. - - Supported strategies should include: +- retain first value; +- retain latest value; -\- retain first value; +- version the sample; -\- retain latest value; +- flag conflict for diagnostics. -\- version the sample; +The selected strategy shall be configurable, and the default shall be **flag conflict for diagnostics** (`REJECT_CONFLICT`). The default deliberately favors explicit conflict detection over silent overwrite: a conflicting historical value usually indicates an equipment, clock, or retrieval problem, and overwriting would hide it. -\- flag conflict for diagnostics. - - - -The selected strategy shall be configurable. - - - -\--- - - - -\# 14. History repository abstraction +Historical processing shall remain idempotent across restart. A gateway restart during backfill or spool replay shall not create duplicate logical records for samples received before the restart (see §12 and §17). +--- +# 14. History repository abstraction OPC UA Historical Access shall not directly depend on a specific database. - - The gateway shall define a logical History Repository interface capable of operations equivalent to: - - ```text StoreSample @@ -980,114 +743,78 @@ CheckAvailability ``` - +> Note: this specification uses requirement-style names (`StoreSample`, `ReadRaw`, ...). The concrete method signatures — `store(sample)`, `storeBatch(samples)`, `readRaw(...)`, `readAtTime(...)`, `getBounds(...)` — are defined in `ARCHITECTURE.md` §25 and shall be used consistently by all implementations. The implementation technology is intentionally unspecified. - - An installation may use: +- an internal gateway historian; +- an external relational database; -\- an internal gateway historian; - -\- an external relational database; - -\- a time-series database; - -\- an industrial historian; - -\- another storage service capable of fulfilling the required history interface. +- a time-series database; +- an industrial historian; +- another storage service capable of fulfilling the required history interface. One configured history provider shall be designated as the authoritative source for OPC UA Historical Access. +--- - -\--- - - - -\# 15. Multiple historical storage destinations - - +# 15. Multiple historical storage destinations The gateway shall permit historical information to be delivered to one or several destinations simultaneously. - - Example: - - ```text - Historical pipeline + Historical pipeline - │ + │ - ┌─────────────┼──────────────┐ + ┌─────────────┼──────────────┐ - │ │ │ + │ │ │ - ▼ ▼ ▼ + ▼ ▼ ▼ - Local DB Enterprise Archive + Local DB Enterprise Archive - historian file + historian file ``` - - Storage destinations shall be implemented as independent adapters. - - Failure of one destination shall not normally prevent delivery to another destination. - - Each storage adapter shall independently maintain: +- enabled state; +- connection status; -\- enabled state; +- last successful write; -\- connection status; +- error count; -\- last successful write; +- pending record count; -\- error count; - -\- pending record count; - -\- retry state. - - - -\--- - - - -\# 16. Storage adapter interface +- retry state. +--- +# 16. Storage adapter interface A storage adapter shall consume canonical historical records. - - The adapter shall not receive raw AquaCom telegrams. - - A logical interface shall provide functionality equivalent to: - - ```text Initialize @@ -1104,152 +831,104 @@ Shutdown ``` - - Optional storage adapters may support queries, but writing and querying shall be considered separate capabilities. - - Possible adapters include, without limiting implementations to: +- relational database; +- time-series database; -\- relational database; +- external historian; -\- time-series database; +- append-only file; -\- external historian; +- structured CSV; -\- append-only file; +- JSON/NDJSON; -\- structured CSV; +- message broker; -\- JSON/NDJSON; +- data-lake/object storage; -\- message broker; - -\- data-lake/object storage; - -\- another application through an API. - - - -\--- - - - -\# 17. Store-and-forward +- another application through an API. +--- +# 17. Store-and-forward The gateway shall provide a durable buffering mechanism when a configured storage destination is temporarily unavailable. - - Example: - - ```text AquaCom - │ + │ - ▼ + ▼ History pipeline - │ + │ - ▼ + ▼ Durable spool - │ + │ - ├──► Storage A + ├──► Storage A - ├──► Storage B + ├──► Storage B - └──► Storage C + └──► Storage C ``` - - Each destination shall maintain its own delivery acknowledgement. - - A record shall therefore not be deleted from the spool merely because one of several destinations has accepted it. - - The implementation should use at-least-once delivery semantics combined with idempotent storage where practical. - - Configuration shall include: +- maximum spool size; +- maximum storage duration; -\- maximum spool size; - -\- maximum storage duration; - -\- disk-space thresholds; - -\- retry behavior; - -\- overflow policy. +- disk-space thresholds; +- retry behavior; +- overflow policy. Loss of buffered information shall generate a diagnostic event. +--- - -\--- - - - -\# 18. OPC UA Server - - +# 18. OPC UA Server The gateway shall contain an OPC UA server compliant with applicable OPC UA specifications. - - OPC UA shall expose: +1. current values through normal OPC UA Data Access; +2. historical values through OPC UA Historical Access; -1\. current values through normal OPC UA Data Access; - -2\. historical values through OPC UA Historical Access; - -3\. gateway communication and synchronization diagnostics. - - +3. gateway communication and synchronization diagnostics. OPC UA Historical Access defines standardized mechanisms for exposing historical information, including `HistoryRead`; it does not require the historical backend itself to be a particular database technology. +--- - -\--- - - - -\# 19. OPC UA namespace - - +# 19. OPC UA namespace A logical namespace should use a structure comparable to: - - ```text AquaCom @@ -1294,143 +973,91 @@ AquaCom └── Gateway - ├── Status + ├── Status - ├── StartTime + ├── StartTime - ├── Version + ├── Version - ├── Storage + ├── Storage - └── Diagnostics + └── Diagnostics ``` - - The precise hierarchy shall be configurable without changing the AquaCom protocol implementation. +--- - -\--- - - - -\# 20. Stable OPC UA NodeIds - - +# 20. Stable OPC UA NodeIds NodeIds used for engineering data shall remain stable across: +- gateway restart; +- OPC UA server restart; -\- gateway restart; +- database restart; -\- OPC UA server restart; - -\- database restart; - -\- software upgrade; - -\- temporary station removal; - -\- changes to point display names. +- software upgrade; +- temporary station removal; +- changes to point display names. NodeIds shall therefore derive from stable engineering identifiers rather than runtime object allocation. - - Changing the human-readable name of: - - ```text Level ``` - - to: - - ```text WetWellLevel ``` - - shall not automatically create a different logical OPC UA point. +--- - -\--- - - - -\# 21. Current OPC UA values - - +# 21. Current OPC UA values Each configured point exposed as current data shall support normal OPC UA reads. - - Where appropriate it shall also support OPC UA subscriptions. - - The OPC UA value shall contain: +- process value; +- appropriate StatusCode; -\- process value; - -\- appropriate StatusCode; - -\- source timestamp where available; - -\- server timestamp. - +- source timestamp where available; +- server timestamp. The gateway shall translate internal quality into appropriate OPC UA quality/status semantics. - - Communication loss shall not silently retain an indefinitely "Good" value. +--- - -\--- - - - -\# 22. OPC UA Historical Access - - +# 22. OPC UA Historical Access Points configured for historical access shall advertise historical capability according to OPC UA. - - OPC UA distinguishes whether history is available through the Variable's access attributes and whether the server is actively collecting history through its `Historizing` attribute. - - The gateway shall support, at minimum: - - -\*\*Raw historical reads\*\* - - +**Raw historical reads** ```text @@ -1446,226 +1073,152 @@ MaximumValues ``` - - The implementation shall support continuation where the requested history exceeds the configured response size. - - Historical responses shall maintain: +- process value; +- source timestamp; -\- process value; - -\- source timestamp; - -\- applicable quality/status information. - - +- applicable quality/status information. The gateway shall not require historical data to have originally been received while an OPC UA client was connected. +--- - -\--- - - - -\# 23. HistoryRead data flow - - +# 23. HistoryRead data flow A Historical Access request shall follow the logical path: - - ```text OPC UA Client - │ + │ - │ HistoryRead + │ HistoryRead - ▼ + ▼ OPC UA Server - │ + │ - ▼ + ▼ History Provider - │ + │ - ▼ + ▼ Configured authoritative repository - │ + │ - ▼ + ▼ -Canonical HistoricalSample\[] +Canonical HistoricalSample[] - │ + │ - ▼ + ▼ -OPC UA DataValue\[] +OPC UA DataValue[] - │ + │ - ▼ + ▼ OPC UA Client ``` - - The OPC UA layer shall not perform AquaCom history polling synchronously in response to a client `HistoryRead`. - - Historical retrieval from field equipment and historical access from clients shall therefore be decoupled. - - This prevents a long-running AquaCom operation from blocking an OPC UA request. +--- - -\--- - - - -\# 24. OPC UA HistoryUpdate - - +# 24. OPC UA HistoryUpdate Modification of historical records through OPC UA shall be disabled by default. - - If later enabled, separate authorization shall be required. - - Normal SCADA clients shall not be permitted to alter gateway history solely because they are allowed to read it. +--- - -\--- - - - -\# 25. Current state versus historical state - - +# 25. Current state versus historical state The gateway shall maintain a strict separation: - - ```text Current State - │ - - └── latest known process value - + │ + └── latest known process value Historical State - │ + │ - └── timestamped sequence of previous process values + └── timestamped sequence of previous process values ``` - - Historical backfill shall never cause an old value to overwrite a newer current value unless a separately defined reconciliation rule determines that the current state itself is invalid. +--- - -\--- - - - -\# 26. Time handling - - +# 26. Time handling All internal timestamps shall have an unambiguous time basis. - - UTC shall be preferred internally. - - The system shall preserve information necessary to correctly interpret AquaCom timestamps supplied in local time. - - Configuration shall include: +- station timezone; +- daylight-saving-time handling; -\- station timezone; - -\- daylight-saving-time handling; - -\- assumed timezone for timestamps lacking zone information; - -\- permitted clock deviation; - -\- invalid-time handling. +- assumed timezone for timestamps lacking zone information; +- permitted clock deviation; +- invalid-time handling. The system shall detect or flag where practical: +- future timestamps; +- implausibly old timestamps; -\- future timestamps; - -\- implausibly old timestamps; - -\- backward-moving device clocks; - -\- duplicate timestamps; - -\- ambiguous daylight-saving-time timestamps. +- backward-moving device clocks; +- duplicate timestamps; +- ambiguous daylight-saving-time timestamps. Original source timestamps shall not silently be replaced by gateway receipt time. +--- - -\--- - - - -\# 27. Quality handling - - +# 27. Quality handling The canonical data model shall contain protocol-independent data quality. - - At minimum the internal model shall distinguish: - - ```text GOOD @@ -1678,220 +1231,148 @@ UNKNOWN ``` - - Additional sub-status information may identify: +- communication failure; +- stale value; -\- communication failure; +- configuration error; -\- stale value; +- conversion error; -\- configuration error; - -\- conversion error; - -\- invalid source value; - -\- historical gap; - -\- timestamp uncertainty. +- invalid source value; +- historical gap; +- timestamp uncertainty. The OPC UA layer shall map internal quality to suitable OPC UA `StatusCode` values. - - Storage adapters shall retain quality information where the destination format supports it. +--- - -\--- - - - -\# 28. Configuration - - +# 28. Configuration Configuration shall be external to the executable software and shall be machine-readable. - - Configuration shall cover at least: +- gateway identity; +- AquaCom stations; -\- gateway identity; +- transports; -\- AquaCom stations; +- point definitions; -\- transports; +- polling; -\- point definitions; +- historical acquisition; -\- polling; +- OPC UA namespace; -\- historical acquisition; +- OPC UA endpoints; -\- OPC UA namespace; +- certificates; -\- OPC UA endpoints; +- authorization; -\- certificates; +- storage destinations; -\- authorization; +- historian configuration; -\- storage destinations; - -\- historian configuration; - -\- buffering; - -\- logging; - -\- diagnostics. +- buffering; +- logging; +- diagnostics. Configuration shall be versionable using normal source-control mechanisms. - - Secrets shall not be stored as unprotected ordinary configuration values. +--- - -\--- - - - -\# 29. Dynamic configuration - - +# 29. Dynamic configuration Where technically reasonable, non-structural configuration should be reloadable without restarting the entire gateway. - - Structural changes that affect point identity or protocol configuration may require controlled restart. - - The gateway shall validate configuration before activation. - - Invalid configuration shall not replace the last known valid configuration. +--- - -\--- - - - -\# 30. Security - - +# 30. Security The OPC UA endpoint shall use OPC UA security mechanisms supporting: +- server authentication; +- client/application authentication; -\- server authentication; - -\- client/application authentication; - -\- confidentiality; - -\- integrity; - -\- user authorization where required. +- confidentiality; +- integrity; +- user authorization where required. The OPC UA security model provides application authentication using certificates as well as user authentication and authorization mechanisms. - - Security mode `None` shall be disabled by default. Current OPC UA guidance specifies that the None profile should be disabled by default. - - The gateway shall support certificate trust management. - - Administrative functions shall be restricted separately from normal OPC UA data access. +--- - -\--- - - - -\# 31. AquaCom security boundary - - +# 31. AquaCom security boundary AquaCom communication shall be treated as communication with a legacy field protocol. - - Security controls around AquaCom shall therefore be provided by the surrounding OT architecture, for example: - - ```text Enterprise/SCADA network - │ + │ - Firewall + Firewall - │ + │ - AquaCom Gateway + AquaCom Gateway - │ + │ - OT network + OT network - │ + │ - VPN/router/firewall + VPN/router/firewall - │ + │ - AquaCom RTU + AquaCom RTU ``` - - The gateway shall not assume that authenticity, confidentiality or authorization are inherently provided by AquaCom. +--- - -\--- - - - -\# 32. Diagnostics - - +# 32. Diagnostics The system shall expose diagnostic information for the gateway and every station. - - Station diagnostics shall include at least: - - ```text Enabled @@ -1918,12 +1399,8 @@ HistoricalBacklog ``` - - Gateway diagnostics shall include: - - ```text GatewayState @@ -1948,12 +1425,8 @@ OPCUAState ``` - - Storage diagnostics shall be available for each configured destination: - - ```text State @@ -1968,98 +1441,64 @@ RetryCount ``` +--- - -\--- - - - -\# 33. Logging - - +# 33. Logging Logging shall distinguish at minimum: +- operational information; +- communication errors; -\- operational information; +- protocol errors; -\- communication errors; +- configuration errors; -\- protocol errors; +- storage errors; -\- configuration errors; - -\- storage errors; - -\- security events; - -\- historical synchronization events; - -\- software faults. +- security events; +- historical synchronization events; +- software faults. Raw AquaCom telegram logging shall be separately configurable because it may generate large volumes of information. - - Sensitive information shall not be unnecessarily written to normal logs. +--- - -\--- - - - -\# 34. Audit logging - - +# 34. Audit logging Security-relevant and administrative changes shall be auditable. - - Examples include: +- configuration changed; +- station enabled/disabled; -\- configuration changed; +- certificate trusted/rejected; -\- station enabled/disabled; +- storage adapter changed; -\- certificate trusted/rejected; - -\- storage adapter changed; - -\- history synchronization manually initiated; - -\- historical records administratively modified; - -\- software started/stopped. +- history synchronization manually initiated; +- historical records administratively modified; +- software started/stopped. Future control functionality shall additionally audit all process commands. +--- - -\--- - - - -\# 35. Failure handling - - +# 35. Failure handling The gateway shall continue providing unaffected functionality where practical. - - Examples: - - | Failure | Expected behavior | |---|---| @@ -2080,158 +1519,110 @@ Examples: | Historical duplicate received | Deduplication policy applied | - - A failure in a secondary data consumer shall not stop AquaCom communication. +--- - -\--- - - - -\# 36. Startup sequence - - +# 36. Startup sequence A logical startup shall perform: - - ```text Load configuration - ↓ + ↓ Validate configuration - ↓ + ↓ Initialize persistent state - ↓ + ↓ Initialize storage adapters - ↓ + ↓ Initialize historical subsystem - ↓ + ↓ Initialize OPC UA namespace - ↓ + ↓ Initialize AquaCom stations - ↓ + ↓ Start acquisition - ↓ + ↓ Start history synchronization ``` - - A storage destination that is temporarily unavailable during startup shall not necessarily prevent the complete gateway from entering service. +--- - -\--- - - - -\# 37. Shutdown - - +# 37. Shutdown Shutdown shall be controlled. - - The gateway shall: +1. stop accepting new administrative work; +2. stop initiating new AquaCom transactions; -1\. stop accepting new administrative work; +3. finish or cancel active transactions safely; -2\. stop initiating new AquaCom transactions; +4. persist synchronization watermarks; -3\. finish or cancel active transactions safely; +5. flush historical queues where possible; -4\. persist synchronization watermarks; - -5\. flush historical queues where possible; - -6\. close storage adapters; - -7\. close OPC UA services; - -8\. terminate. +6. close storage adapters; +7. close OPC UA services; +8. terminate. Unclean shutdown shall be detectable at the next startup. +--- - -\--- - - - -\# 38. Persistence of gateway state - - +# 38. Persistence of gateway state State required for reliable recovery shall survive restart. - - This includes, where applicable: +- history synchronization watermarks; +- spool contents; -\- history synchronization watermarks; - -\- spool contents; - -\- delivery acknowledgements; - -\- configuration version; - -\- station state needed for recovery. +- delivery acknowledgements; +- configuration version; +- station state needed for recovery. Transient live values need not be permanently persisted unless required by installation policy. +--- - -\--- - - - -\# 39. Historical retention - - +# 39. Historical retention Retention shall be configurable independently for each storage destination. - - The gateway shall not assume that all historical destinations have the same retention period. - - For example: - - ```text Local operational cache 30 days @@ -2242,336 +1633,226 @@ Archive indefinite ``` - - These values are examples only and are not system defaults. +--- - -\--- - - - -\# 40. Data export extensibility - - +# 40. Data export extensibility The internal historical pipeline shall permit future exporters without modification to the AquaCom protocol implementation. - - Examples might later include: - - ```text AquaCom - ↓ + ↓ Canonical data - ├── OPC UA + ├── OPC UA - ├── SQL + ├── SQL - ├── time-series DB + ├── time-series DB - ├── enterprise historian + ├── enterprise historian - ├── MQTT + ├── MQTT - └── data lake + └── data lake ``` - - This shall be achieved through explicit interfaces rather than direct dependencies between components. +--- - -\--- - - - -\# 41. Performance requirements - - +# 41. Performance requirements Performance targets shall be installation-specific and configurable. - - The gateway architecture shall support measurement of: +- AquaCom request latency; +- point update latency; -\- AquaCom request latency; +- OPC UA response latency; -\- point update latency; +- historical ingestion rate; -\- OPC UA response latency; +- history query time; -\- historical ingestion rate; - -\- history query time; - -\- pending history size; - -\- storage write throughput; - -\- spool utilization. +- pending history size; +- storage write throughput; +- spool utilization. Historical backfill shall have configurable throttling. - - Large history synchronization jobs shall not starve current-value acquisition. +--- - -\--- - - - -\# 42. Resource limits - - +# 42. Resource limits The gateway shall protect itself against unbounded resource consumption. - - Configurable limits shall include: +- maximum AquaCom telegram size; +- maximum queued requests; -\- maximum AquaCom telegram size; +- maximum OPC UA HistoryRead result size; -\- maximum queued requests; +- maximum number of simultaneous historical queries; -\- maximum OPC UA HistoryRead result size; - -\- maximum number of simultaneous historical queries; - -\- maximum spool storage; - -\- maximum storage batch size; - -\- maximum diagnostic-log size. +- maximum spool storage; +- maximum storage batch size; +- maximum diagnostic-log size. OPC UA continuation mechanisms shall be used when a historical query exceeds the configured response size. +--- - -\--- - - - -\# 43. Deployment - - +# 43. Deployment The specification shall not assume a particular operating system. - - The gateway should be capable of deployment as a long-running service. - - The implementation shall allow unattended startup following host restart. - - Runtime dependencies shall be documented and minimized. - - The gateway should support deployment in either: - - ```text Physical/virtual OT server ``` - - or - - ```text Containerized environment ``` - - provided that the selected deployment method satisfies communication and security requirements. +--- - -\--- - - - -\# 44. Internal component boundaries - - +# 44. Internal component boundaries The following logical modules shall remain separable: - - ```text AquaCom Transport - │ + │ AquaCom Protocol - │ + │ Station/session handling - │ + │ Canonical Data Model - │ + │ - ┌──────┴────────┐ + ┌──────┴────────┐ - │ │ + │ │ Live State History Engine - │ │ + │ │ - │ ┌─────┴───────┐ + │ ┌─────┴───────┐ - │ │ │ + │ │ │ OPC UA Repository Storage adapters ``` - - A change of OPC UA library shall not require reimplementation of AquaCom. - - A change of database shall not require reimplementation of AquaCom. - - A new storage adapter shall not require modification to the AquaCom parser. +--- - -\--- - - - -\# 45. Testing requirements - - +# 45. Testing requirements The system shall support automated testing at multiple levels. - - -\## 45.1 Protocol unit tests - - +## 45.1 Protocol unit tests Tests shall verify: +- telegram framing; +- checksum calculation; -\- telegram framing; +- request encoding; -\- checksum calculation; +- response decoding; -\- request encoding; - -\- response decoding; - -\- invalid telegram detection; - -\- timeout behavior; - -\- error response handling. +- invalid telegram detection; +- timeout behavior; +- error response handling. Known production captures should be converted into reusable test vectors where legally and operationally appropriate. +--- - -\--- - - - -\## 45.2 AquaCom simulator - - +## 45.2 AquaCom simulator A simulator or mock endpoint shall be provided for development and testing. - - It shall support: +- normal responses; +- configurable live values; -\- normal responses; +- historical data; -\- configurable live values; +- delayed responses; -\- historical data; +- malformed telegrams; -\- delayed responses; - -\- malformed telegrams; - -\- communication loss; - -\- duplicate historical samples; - -\- clock anomalies. +- communication loss; +- duplicate historical samples; +- clock anomalies. Development shall not require connection to operational wastewater infrastructure. +--- - -\--- - - - -\## 45.3 Historical tests - - +## 45.3 Historical tests Tests shall verify: - - ```text Historical backfill after outage @@ -2592,146 +1873,96 @@ Spool replay Multi-destination delivery +Out-of-order arrival (backfill delivered after newer live samples) + +Queries return results ordered by SourceTimestamp, not arrival order + +Records rejected or quarantined when the source timestamp is missing or unusable + ``` +--- - -\--- - - - -\## 45.4 OPC UA tests - - +## 45.4 OPC UA tests Tests shall verify: +- namespace creation; +- stable NodeIds; -\- namespace creation; +- current reads; -\- stable NodeIds; +- subscriptions; -\- current reads; +- source timestamps; -\- subscriptions; +- quality; -\- source timestamps; +- HistoryRead; -\- quality; +- continuation; -\- HistoryRead; +- empty historical intervals; -\- continuation; +- large historical intervals; -\- empty historical intervals; +- station communication loss; -\- large historical intervals; +- certificate validation; -\- station communication loss; - -\- certificate validation; - -\- authorization. - - - -\--- - - - -\# 46. End-to-end acceptance scenario +- authorization. +--- +# 46. End-to-end acceptance scenario A mandatory acceptance scenario shall simulate: - - ```text -1\. Station communicating normally. +1. Station communicating normally. +2. Live values available through OPC UA. +3. Values stored historically. -2\. Live values available through OPC UA. +4. AquaCom connection interrupted. +5. OPC UA indicates communication degradation. +6. Station continues recording locally. -3\. Values stored historically. +7. AquaCom connection restored. +8. Gateway resumes live acquisition. +9. Gateway determines missing historical interval. -4\. AquaCom connection interrupted. +10. Gateway retrieves missing samples. +11. Retrieved samples retain their original timestamps. +12. Historical samples are persisted. -5\. OPC UA indicates communication degradation. +13. Old historical samples do not appear as new current values. +14. OPC UA HistoryRead returns the complete time interval. +15. Configured secondary historical destinations receive the recovered data. -6\. Station continues recording locally. - - - -7\. AquaCom connection restored. - - - -8\. Gateway resumes live acquisition. - - - -9\. Gateway determines missing historical interval. - - - -10\. Gateway retrieves missing samples. - - - -11\. Retrieved samples retain their original timestamps. - - - -12\. Historical samples are persisted. - - - -13\. Old historical samples do not appear as new current values. - - - -14\. OPC UA HistoryRead returns the complete time interval. - - - -15\. Configured secondary historical destinations receive the recovered data. - - - -16\. Duplicate acquisition does not produce unintended duplicate records. +16. Duplicate acquisition does not produce unintended duplicate records. ``` - - Successful execution of this scenario shall be a principal acceptance criterion for the gateway. +--- - -\--- - - - -\# 47. Recommended initial functional baseline - - +# 47. Recommended initial functional baseline The first production-capable release should implement: - - ```text AquaCom communication @@ -2780,27 +2011,17 @@ Automated history tests ``` - - This baseline deliberately excludes remote process control. +--- - -\--- - - - -\# 48. Future extensions - - +# 48. Future extensions The architecture shall permit later addition of: - - ```text -OPC UA Alarms \& Conditions +OPC UA Alarms & Conditions AquaCom alarm/event retrieval @@ -2832,275 +2053,203 @@ automated station discovery ``` +OPC UA Alarms & Conditions is defined separately in OPC UA Part 9 and can therefore be introduced later without treating alarm events as ordinary process-value history. +--- -OPC UA Alarms \& Conditions is defined separately in OPC UA Part 9 and can therefore be introduced later without treating alarm events as ordinary process-value history. - - - -\--- - - - -\# 49. Key architectural decision - - +# 49. Key architectural decision The system shall be considered an: - - -\*\*AquaCom Integration Gateway\*\* - - +**AquaCom Integration Gateway** rather than merely an: - - -\*\*AquaCom OPC UA Server\*\*. - - +**AquaCom OPC UA Server**. The distinction is intentional. - - The internal architecture shall be: - - ```text - AQUACOM + AQUACOM - │ + │ - ▼ + ▼ - Protocol abstraction + Protocol abstraction - │ + │ - ▼ + ▼ - Canonical data + Canonical data - │ + │ - ┌───────────┼────────────┐ + ┌───────────┼────────────┐ - │ │ │ + │ │ │ - ▼ ▼ ▼ + ▼ ▼ ▼ - Live state History Diagnostics + Live state History Diagnostics - │ │ + │ │ - │ ┌─────┼───────────────┐ + │ ┌─────┼───────────────┐ - │ │ │ │ + │ │ │ │ - ▼ ▼ ▼ ▼ + ▼ ▼ ▼ ▼ - OPC UA DB Historian Archive + OPC UA DB Historian Archive - │ + │ - ├── Data Access + ├── Data Access - └── Historical Access + └── Historical Access ``` - - OPC UA is consequently one standardized northbound interface to the gateway rather than the mechanism upon which acquisition and historical storage depend. +--- - -\--- - - - -\# 50. Architectural rule for historical information - - +# 50. Architectural rule for historical information The following rule shall be treated as fundamental: - - -> \*\*A historical sample represents the state of the process at its SourceTimestamp, not the state of the process at the time the gateway received the sample.\*\* - - +> **A historical sample represents the state of the process at its SourceTimestamp, not the state of the process at the time the gateway received the sample.** Therefore: - - ```text Historical recovery - ≠ + ≠ replay of live values ``` - - and: - - ```text AquaCom historical timestamp - │ + │ - ▼ + ▼ Canonical SourceTimestamp - │ + │ - ├──► Historical repository + ├──► Historical repository - ├──► External storage + ├──► External storage - └──► OPC UA Historical Access + └──► OPC UA Historical Access ``` - - This rule shall apply consistently throughout implementation, testing and validation. +--- - -\--- - - - -\# 51. Architectural rule for storage - - +# 51. Architectural rule for storage Historical acquisition and historical storage shall be decoupled. - - The gateway shall be capable of configurations conceptually equivalent to: - - ```text Configuration A - - AquaCom - ↓ + ↓ Gateway - ↓ + ↓ OPC UA live only ``` - - ```text Configuration B - - AquaCom - ↓ + ↓ Gateway - ├── OPC UA live + ├── OPC UA live - └── Internal history + └── Internal history - ↓ + ↓ - OPC UA HA + OPC UA HA ``` - - ```text Configuration C - - AquaCom - ↓ + ↓ Gateway - ├── OPC UA + ├── OPC UA - ├── SQL + ├── SQL - └── Historian + └── Historian ``` - - ```text Configuration D - - AquaCom - ↓ + ↓ Gateway - ├── OPC UA + ├── OPC UA - ├── Time-series DB + ├── Time-series DB - ├── Enterprise historian + ├── Enterprise historian - ├── Archive + ├── Archive - └── Future integration adapter + └── Future integration adapter ``` - - The protocol implementation shall be identical in all cases. +--- - -\--- - - - -\# 52. Requirement summary - - +# 52. Requirement summary | ID | Requirement | @@ -3166,123 +2315,115 @@ The protocol implementation shall be identical in all cases. | AQG-030 | Administrative and security-sensitive operations shall be auditable. | +--- - -\--- - - - -\# 53. Final target architecture - - +# 53. Final target architecture ```text - FIELD LAYER + FIELD LAYER - │ + │ - AquaCom-compatible + AquaCom-compatible - RTU / PLC + RTU / PLC - │ + │ - ▼ + ▼ - ┌───────────────────┐ + ┌───────────────────┐ - │ AquaCom Transport │ + │ AquaCom Transport │ - └─────────┬─────────┘ + └─────────┬─────────┘ - │ + │ - ┌─────────▼─────────┐ + ┌─────────▼─────────┐ - │ AquaCom Protocol │ + │ AquaCom Protocol │ - │ Engine │ + │ Engine │ - └─────────┬─────────┘ + └─────────┬─────────┘ - │ + │ - ┌─────────▼─────────┐ + ┌─────────▼─────────┐ - │ Station Manager │ + │ Station Manager │ - └─────────┬─────────┘ + └─────────┬─────────┘ - │ + │ - ┌─────────▼─────────┐ + ┌─────────▼─────────┐ - │ Canonical Model │ + │ Canonical Model │ - └────┬─────────┬────┘ + └────┬─────────┬────┘ - │ │ + │ │ - LIVE │ │ HISTORY + LIVE │ │ HISTORY - │ │ + │ │ - ┌──────────▼───┐ ┌───▼────────────────┐ + ┌──────────▼───┐ ┌───▼────────────────┐ - │ Current State│ │ History Acquisition │ + │ Current State│ │ History Acquisition │ - │ Cache │ │ / Backfill Manager │ + │ Cache │ │ / Backfill Manager │ - └──────┬───────┘ └─────────┬──────────┘ + └──────┬───────┘ └─────────┬──────────┘ - │ │ + │ │ - │ ┌─────▼─────┐ + │ ┌─────▼─────┐ - │ │ Historical│ + │ │ Historical│ - │ │ Pipeline │ + │ │ Pipeline │ - │ └─────┬─────┘ + │ └─────┬─────┘ - │ │ + │ │ - │ ┌───────────┼───────────┐ + │ ┌───────────┼───────────┐ - │ │ │ │ + │ │ │ │ - │ ▼ ▼ ▼ + │ ▼ ▼ ▼ - │ Repository Storage A Storage B + │ Repository Storage A Storage B - │ │ + │ │ - │ │ + │ │ - ┌──────▼───────▼──────┐ + ┌──────▼───────▼──────┐ - │ OPC UA Server │ + │ OPC UA Server │ - │ │ + │ │ - │ Data Access │ + │ Data Access │ - │ Historical Access │ + │ Historical Access │ - └──────────┬──────────┘ + └──────────┬──────────┘ - │ + │ - ▼ + ▼ - SCADA / historian / + SCADA / historian / - analytics clients + analytics clients ``` - - This architecture shall be the baseline for detailed design and implementation. diff --git a/missing_docs.md b/missing_docs.md new file mode 100644 index 0000000..d25a615 --- /dev/null +++ b/missing_docs.md @@ -0,0 +1,76 @@ +# Missing documentation + +Documents referenced by this repository's guidance that do **not** yet exist. They are planned content, not stale references — do not delete the references; create the documents as their subject matter becomes established. + +Referenced from: `AGENTS.md`, `ARCHITECTURE.md`, `docs/product/codex-implementation-specification.md`, `docs/product/software-specification.md`, `docs/configuration/configuration-reference.md`. + +## Root + +| Document | Referenced for | Source of reference | +|---|---|---| +| `README.md` | Repository overview; required by the repository structure (§5) | codex-implementation-specification.md §5 | +| `CHANGELOG.md` | Version history; required by the repository structure (§5) | codex-implementation-specification.md §5 | + +## docs/architecture/ + +Referenced by `ARCHITECTURE.md` (§1) and the repository structure (§5). + +| Document | Referenced for | +|---|---| +| `docs/architecture/overview.md` | Detailed architecture overview | +| `docs/architecture/data-flow.md` | Detailed data-flow description | +| `docs/architecture/live-data.md` | Detailed live-data architecture | +| `docs/architecture/historical-data.md` | Detailed historical-data architecture | +| `docs/architecture/opcua.md` | Detailed OPC UA architecture | +| `docs/architecture/storage.md` | Detailed storage architecture | +| `docs/architecture/security.md` | Detailed security architecture | + +## docs/protocol/ + +Referenced by `AGENTS.md` (protocol-evidence and uncertainty rules), `ARCHITECTURE.md` (§54), `software-specification.md` (§5.1), and the repository structure (§5). + +| Document | Referenced for | Priority | +|---|---|---| +| `docs/protocol/aquacom.md` | Authoritative AquaCom protocol knowledge base | High — blocks Phase 3+ implementation | +| `docs/protocol/known-operations.md` | Registry of verified AquaCom operations | High | +| `docs/protocol/protocol-gaps.md` | **Protocol-gap registry.** Every unresolved AquaCom protocol question shall be recorded here; the evidence rules in `AGENTS.md` and the implementation spec are not executable without it. | **Highest — create first** | +| `docs/protocol/test-vectors.md` | Approved protocol test vectors for implemented telegram types | High — required by every implemented telegram type | + +## docs/operations/ + +Referenced by the repository structure (§5) and Phase 14 (production hardening). + +| Document | Referenced for | +|---|---| +| `docs/operations/deployment.md` | Deployment and operations documentation (Phase 14; DoD item 14) | +| `docs/operations/diagnostics.md` | Operator diagnostics guide | +| `docs/operations/backup-recovery.md` | Backup and recovery procedures (Phase 14; DoD item 14) | +| `docs/operations/troubleshooting.md` | Troubleshooting guide | + +## docs/exec-plans/ + +Referenced by `AGENTS.md` and codex-implementation-specification.md §59. + +| Path | Referenced for | +|---|---| +| `docs/exec-plans/active/` | Active ExecPlans for substantial work | +| `docs/exec-plans/completed/` | Completed ExecPlans | + +## Schemas + +Referenced by `configuration-reference.md` §1 and the repository structure (§5). Extensions depend on the configuration format chosen during technology selection. + +| Path | Referenced for | +|---|---| +| `schemas/gateway-config.schema.*` | Machine-readable configuration schema | +| `schemas/station-config.schema.*` | Station configuration schema | +| `schemas/point-config.schema.*` | Point configuration schema | +| `schemas/storage-config.schema.*` | Storage configuration schema (configuration-reference.md §84) | + +--- + +## Notes + +- The directories `docs/architecture/` and `docs/protocol/` already exist but are empty; `docs/operations/`, `docs/exec-plans/` and `schemas/` do not exist yet. +- `protocol-gaps.md` deserves priority: the architecture's evidence rules direct unresolved protocol questions into it, so its absence would leave the first such question with nowhere to go. Creating an empty registry with the entry template from codex-implementation-specification.md §17 is sufficient to start. +- ExecPlan directory contents are created on demand as substantial work begins; an empty directory is the normal state. \ No newline at end of file