20 KiB
# AquaCom Integration Gateway – Codex Instructions
This repository implements the **AquaCom Integration Gateway**.
The gateway communicates with AquaCom-compatible field equipment, normalizes live and historical process data, exposes data through OPC UA, and supports one or more historical storage destinations.
Before making architectural or behavioral changes, read:
- ARCHITECTURE.md
- docs/product/software-specification.md
- docs/architecture/
- docs/protocol/
- docs/configuration/configuration-reference.md
For substantial work, also read or create the applicable ExecPlan under:
docs/exec-plans/
---
## 1. Primary architecture
The intended architecture is:
AquaCom
  │
  ▼
Transport
  │
  ▼
AquaCom Protocol Engine
  │
  ▼
Station / Session Manager
  │
  ▼
Canonical Data Model
  │
  ├──────────────► Current-State Service
  │ │
  │ ▼
  │ OPC UA
  │ Data Access
  │
  └──────────────► Historical Pipeline
  │
  ┌──────────┼──────────┐
  │ │ │
  ▼ ▼ ▼
  Repository Sink A Sink B
  │
  ▼
  OPC UA
  Historical Access
Maintain these boundaries.
---
## 2. Non-negotiable architecture rules
The following rules are mandatory.
### 2.1 AquaCom isolation
All AquaCom-specific parsing, encoding, framing, checksum handling and protocol behavior belongs in the AquaCom modules.
No OPC UA, history repository or storage adapter shall parse AquaCom telegrams.
### 2.2 OPC UA isolation
The OPC UA implementation shall consume canonical gateway data.
The OPC UA implementation shall not:
- construct AquaCom requests;
- decode AquaCom responses;
- perform direct AquaCom history retrieval;
- depend upon AquaCom telegram structures.
### 2.3 Storage isolation
Storage implementations shall consume canonical historical records.
Storage adapters shall not depend upon:
- AquaCom telegrams;
- AquaCom addressing;
- AquaCom command structures;
- OPC UA node implementations.
### 2.4 Canonical model
Data shall cross subsystem boundaries through the canonical gateway model.
Do not bypass the canonical model for convenience.
### 2.5 Live and historical data are different
Historical samples shall never be replayed as current values merely because they were recently received.
For example:
Gateway receives at: 18:30
Historical sample timestamp: 14:05
The value belongs to history at 14:05.
It is not a new live value at 18:30.
### 2.6 Preserve timestamps
Keep at least the following concepts separate:
SourceTimestamp
ReceivedTimestamp
Where applicable also maintain:
ServerTimestamp
UpdatedTimestamp
Never silently replace a valid AquaCom source timestamp with gateway receipt time.
### 2.7 Stable identifiers
StationId, PointId and OPC UA NodeIds must remain stable across:
- process restart;
- gateway restart;
- software upgrade;
- display-name changes;
- transport-address changes.
Display names are not stable identifiers.
### 2.8 Live acquisition priority
Normal live acquisition shall take priority over bulk historical backfill where both compete for limited AquaCom communication capacity.
---
## 3. Critical AquaCom rule
**Never invent AquaCom protocol behavior.**
Do not infer undocumented protocol fields merely because an implementation appears plausible.
Do not invent:
- telegram fields;
- byte positions;
- ASCII fields;
- framing rules;
- checksum algorithms;
- addressing behavior;
- command codes;
- response codes;
- value scaling;
- timestamp encoding;
- alarm semantics;
- historical-data semantics.
AquaCom behavior may only be implemented when supported by evidence.
Acceptable evidence includes:
1. authoritative AquaCom documentation;
2. validated traffic captures;
3. validated behavior from an existing known-good implementation;
4. approved protocol test vectors.
If required protocol information is unavailable:
1. do not guess;
2. leave the feature unimplemented;
3. add an entry to:
docs/protocol/protocol-gaps.md
4. add interfaces, fixtures or simulation support around the missing implementation where useful.
Compilation success is not evidence that an AquaCom implementation is correct.
---
## 4. Protocol traceability
Every implemented AquaCom operation shall be traceable to its evidence.
Document protocol implementation under:
docs/protocol/
Where practical record:
Operation
Evidence source
Specification section
Capture/test-vector reference
Supported controller types
Implementation status
Associated tests
Known limitations
Every implemented telegram type shall have automated test vectors.
---
## 5. Protocol fixtures
Protocol fixtures belong under:
fixtures/aquacom/
Fixtures derived from operational traffic shall be sanitized before being committed.
Do not commit unnecessary:
- production IP addresses;
- phone numbers;
- station identifiers;
- credentials;
- private network information;
- other operationally sensitive information.
Reference fixtures used for protocol validation should remain immutable unless there is a documented reason to change them.
---
## 6. Core data model
The core domain shall remain independent of AquaCom, OPC UA and storage technology.
Core concepts include:
StationId
PointId
CurrentValue
HistoricalSample
DataQuality
AcquisitionSource
CommunicationState
StationState
HistorySynchronizationState
The core layer must not reference an OPC UA library or database-specific API.
---
## 7. CurrentValue requirements
A canonical current value shall conceptually contain:
StationId
PointId
Value
DataType
Quality
SourceTimestamp?
ReceivedTimestamp
UpdatedTimestamp
AcquisitionSource
A missing source timestamp must remain distinguishable from a gateway-generated timestamp.
---
## 8. HistoricalSample requirements
A canonical historical sample shall conceptually contain:
StationId
PointId
Value
DataType
Quality
SourceTimestamp
ReceivedTimestamp
AcquisitionSource
AcquisitionId?
Metadata?
A normal historical process sample requires a meaningful SourceTimestamp.
Do not fabricate one solely to make storage easier.
---
## 9. Historical processing
The historical pipeline shall logically follow:
HistoricalSample
  │
  ▼
Validation
  │
  ▼
Normalization
  │
  ▼
Deduplication
  │
  ▼
Durable Spool
  │
  ├────► Repository
  ├────► Storage Sink A
  └────► Storage Sink B
Historical delivery shall be restart-safe.
---
## 10. Deduplication
Default historical identity is conceptually:
StationId
\+
PointId
\+
SourceTimestamp
Use a stronger device-provided identity if verified and available.
Duplicate handling must be deterministic.
Do not silently overwrite conflicting historical values.
The preferred default for conflicting values is:
REJECT\_CONFLICT
and generate diagnostics.
---
## 11. History synchronization
Historical synchronization state shall survive restart.
Do not advance a synchronization watermark until the implementation has sufficient evidence that the corresponding data interval has been successfully processed.
Historical synchronization shall support interruption and continuation.
Large backfills shall be throttleable.
---
## 12. OPC UA rules
OPC UA is a northbound interface to gateway information.
### Current data
Map canonical current information approximately as:
Value
  → OPC UA Value
SourceTimestamp
  → OPC UA SourceTimestamp
Gateway processing timestamp
  → OPC UA ServerTimestamp
DataQuality
  → OPC UA StatusCode
### Historical data
OPC UA Historical Access shall query a History Repository abstraction.
Never implement:
OPC UA HistoryRead
  ↓
synchronous AquaCom request
  ↓
wait for RTU
Implement:
AquaCom history synchronization
  ↓
Historical Repository
OPC UA HistoryRead
  ↓
Historical Repository
The client-visible OPC UA history interface must remain decoupled from field-device communication latency.
### NodeIds
OPC UA NodeIds shall be deterministic and stable.
Changing a point's display name shall not normally change its NodeId.
### Writes
Process-control writes are outside the initial production scope unless explicitly approved.
OPC UA Historical Update shall be disabled by default.
---
## 13. Storage rules
Distinguish:
Repository
from:
Storage Sink
A **Repository** supports historical queries.
A **Storage Sink** accepts historical delivery but does not necessarily support queries.
One component may implement both.
Do not require every storage destination to become an OPC UA Historical Access source.
---
## 14. Multiple storage destinations
Historical data may be delivered to multiple destinations.
Each destination shall maintain independent:
- health;
- acknowledgement;
- retry state;
- diagnostics.
Failure of one storage destination shall not unnecessarily stop other destinations.
A destination may be configured as:
REQUIRED
or:
OPTIONAL
Required delivery behavior shall be explicit.
---
## 15. Durable spool
Historical data pending storage shall use persistent buffering.
The spool must not be an unbounded in-memory queue.
Spool state shall survive process restart.
Each destination shall acknowledge delivery independently.
Do not delete a record solely because one of several required destinations accepted it.
---
## 16. Configuration
Configuration shall be external, machine-readable and schema-validated.
Validate configuration before activating it.
Invalid configuration shall not silently replace the last known valid configuration.
Configuration shall support concepts including:
gateway
stations
transports
points
live acquisition
historical acquisition
OPC UA
history repository
storage destinations
security
logging
diagnostics
Do not hard-code station-specific process definitions into protocol code.
---
## 17. Secrets
Never commit production:
- passwords;
- API tokens;
- private keys;
- OPC UA private keys;
- database credentials;
- secret connection strings.
Example configuration shall use placeholders.
---
## 18. Security
Production OPC UA configuration shall use secure endpoints by default.
Do not make insecure OPC UA operation the default merely to simplify development.
Keep development-only insecure behavior clearly separated and documented.
Administrative capabilities shall be authorized separately from ordinary data access.
If uncertain between a permissive and restrictive security behavior, prefer the restrictive behavior and document the decision.
---
## 19. Diagnostics
New failure modes shall include meaningful diagnostics where practical.
Diagnostics should allow operators to distinguish:
transport failure
protocol failure
station offline
invalid response
history synchronization failure
storage failure
spool backlog
OPC UA failure
configuration failure
Avoid generic errors where useful context is available.
---
## 20. Logging
Prefer structured logging.
Include contextual information where available:
component
stationId
pointId
operation
correlationId
error
Raw AquaCom telegram logging shall be separately configurable.
Do not enable full telegram logging as the normal production default.
---
## 21. Correlation
AquaCom request/response transactions should have correlation identifiers.
Where practical, propagate the correlation identifier into:
- transport diagnostics;
- protocol logs;
- station diagnostics;
- processing errors.
---
## 22. Simulator
The AquaCom simulator is a first-class project component.
Development and CI shall not require operational field equipment.
The simulator shall eventually support:
- normal communication;
- configurable live values;
- historical values;
- delayed response;
- timeout;
- disconnect;
- malformed telegram;
- invalid checksum;
- duplicate history;
- conflicting history;
- clock anomalies;
- reconnect/backfill.
Do not implement the simulator so that it merely calls the same decoder being tested to construct all expected protocol behavior. Avoid self-validating tests.
---
## 23. Testing rules
Every functional change shall have applicable automated tests.
Every bug fix should include a regression test where practical.
Do not weaken or remove tests merely to make a change pass.
Tests shall include appropriate combinations of:
unit tests
protocol fixture tests
integration tests
restart/recovery tests
acceptance tests
Protocol tests must run without external network access.
---
## 24. Mandatory architecture tests
Maintain tests that demonstrate at least:
AquaCom simulator
  → AquaCom decoder
AquaCom live acquisition
  → Current-State Service
Current-State Service
  → OPC UA current value
AquaCom history
  → Historical Pipeline
Historical Pipeline
  → Repository
Historical Pipeline
  → multiple storage destinations
Repository
  → OPC UA HistoryRead
---
## 25. Critical historical acceptance behavior
The following behavior must always remain valid:
1\. Station communicates normally.
2\. Current value is exposed through OPC UA.
3\. Communication is lost.
4\. The device retains historical information.
5\. Communication returns.
6\. Live acquisition resumes.
7\. Missing history is retrieved.
8\. Historical samples retain original timestamps.
9\. Recovered historical samples are persisted.
10\. Recovered historical samples do NOT overwrite
  current state simply because they were just received.
11\. OPC UA HistoryRead returns the recovered values
  at their original timestamps.
12\. Re-retrieving the same historical interval does
  not produce unintended duplicates.
Any change that breaks this scenario is an architectural regression.
---
## 26. Bounded resources
Do not introduce unbounded:
- queues;
- history buffers;
- retry lists;
- logs;
- response payloads;
- continuation state;
- caches.
Resource limits and overflow behavior shall be explicit.
---
## 27. Graceful shutdown
Shutdown shall preserve consistency.
Conceptual order:
stop new administrative work
  ↓
stop scheduling new acquisition
  ↓
complete or cancel active transactions safely
  ↓
persist synchronization state
  ↓
flush durable state where possible
  ↓
stop OPC UA
  ↓
close storage destinations
  ↓
terminate
---
## 28. Restart behavior
Following restart:
load configuration
  ↓
validate persistent state
  ↓
recover spool
  ↓
restore history watermarks
  ↓
resume AquaCom communication
  ↓
resume live acquisition
  ↓
resume historical synchronization
Prioritize restoration of live operation before large history recovery jobs.
---
## 29. Documentation requirements
Update documentation when observable behavior changes.
Relevant locations include:
ARCHITECTURE.md
docs/architecture/
docs/protocol/
docs/configuration/
docs/operations/
Do not leave major architectural decisions only in source-code comments.
---
## 30. ExecPlan requirement
Create or update an ExecPlan for substantial work involving:
- new AquaCom operations;
- historical synchronization;
- persistence;
- storage architecture;
- OPC UA namespace changes;
- security;
- configuration format changes;
- cross-module refactoring;
- data migration;
- significant recovery behavior.
Store active plans under:
docs/exec-plans/active/
Move completed plans to:
docs/exec-plans/completed/
An ExecPlan should contain:
Purpose
Current state
Desired state
Constraints
Affected components
Implementation sequence
Data-model changes
Failure handling
Testing
Compatibility/migration
Progress
Decisions
Unexpected findings
Acceptance evidence
Keep the plan current as implementation progresses.
---
## 31. Handling uncertainty
Classify uncertainties as:
### A. Local implementation detail
Choose the simplest solution consistent with existing architecture.
### B. Architectural decision
Record the decision and reasoning in the current ExecPlan.
### C. AquaCom protocol uncertainty
Do not guess.
Record it in:
docs/protocol/protocol-gaps.md
### D. Security or safety uncertainty
Prefer the safer behavior and document the decision.
---
## 32. Task completion
Do not consider a task complete until applicable conditions are satisfied:
- implementation is complete;
- relevant tests exist;
- tests pass;
- lint/static checks pass;
- configuration validation passes;
- relevant documentation is updated;
- architecture boundaries remain intact;
- new AquaCom behavior has evidence;
- new failure modes have diagnostics;
- no secrets were introduced.
---
## 33. Initial scope
The initial production goal is primarily read-oriented integration.
Expected scope includes:
AquaCom live acquisition
AquaCom historical acquisition
multiple stations
canonical data model
current-state cache
historical pipeline
deduplication
persistent synchronization state
durable spool
historical repository
multiple storage destinations
OPC UA Data Access
OPC UA Historical Access
security
diagnostics
simulation
automated testing
restart recovery
Unless explicitly approved, do not implement:
remote pump control
setpoint changes
generic OPC UA process writes
OPC UA Methods for process control
OPC UA HistoryUpdate
---
## 34. Future control functionality
If process control is later introduced, use a separate command architecture:
OPC UA/API
  │
  ▼
Authorization
  │
  ▼
Command validation
  │
  ▼
Command service
  │
  ▼
AquaCom control operation
  │
  ▼
Confirmation / readback
  │
  ▼
Audit
Never implement process control as:
OPC UA write
  ↓
raw AquaCom telegram
without an explicit command-validation and authorization layer.
---
## 35. Guiding implementation principle
Optimize for:
1. correctness;
2. traceability;
3. recoverability;
4. testability;
5. maintainability;
6. operational diagnostics.
Do not optimize for implementation speed at the expense of protocol correctness.
When AquaCom knowledge is incomplete, build the interfaces, simulation, data model and tests around the missing area first.
Then implement real protocol behavior only when adequate evidence exists.
The central architectural rule remains:
A historical value describes the process at its source timestamp, not at the time the gateway happened to receive it.
And the central protocol rule remains:
Never invent undocumented AquaCom behavior.