jobe/aquacom-gateway-suite
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
08d44babbbd60d4772fdc70471539810cb419005
aquacom-gateway-suite/docs/product/codex-implementation-specification.md
T
Jonas Bengtsson 08d44babbb 📝 Update documents
2026-10-05 21:27:47 +02:00

3753 lines
48 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Implementation Specification
## AquaCom Integration Gateway
**Document type:** Implementation specification
**Target implementation agent:** any AI coding agent (Codex, Copilot, or equivalent)
**System:** AquaCom Integration Gateway
**Primary southbound interface:** AquaCom
**Primary northbound interface:** OPC UA
**Additional capability:** Pluggable historical storage
**Programming language:** To be selected during implementation planning
**Version:** 0.1
**Status:** Draft
---
# 1. Purpose
This document defines how the implementation agent shall implement the AquaCom Integration Gateway.
The gateway shall:
- communicate with AquaCom-compatible devices;
- acquire current process information;
- acquire historical process information;
- maintain a protocol-independent internal data model;
- expose current values through OPC UA;
- expose historical values through OPC UA Historical Access;
- persist historical information through one or several configurable storage providers;
- recover historical data after communication interruptions where AquaCom equipment supports this;
- maintain diagnostic information;
- provide a simulation environment for development and automated testing.
This document describes:
- repository organization;
- architectural boundaries;
- required modules;
- internal interfaces;
- data contracts;
- state machines;
- configuration;
- persistence;
- testing;
- milestones;
- implementation-agent working rules;
- acceptance criteria.
It does not prescribe the implementation programming language.
---
# 2. Fundamental implementation rule
The implementation shall be based on the following processing architecture:
```text
AquaCom
│
▼
Transport
│
▼
AquaCom protocol engine
│
▼
Station/session manager
│
▼
Canonical data model
│
├──────────────► Current-state service
│ │
│ ▼
│ OPC UA DA
│
└──────────────► Historical pipeline
│
┌──────────┼──────────────┐
│ │ │
▼ ▼ ▼
Repository Storage A Storage B
│
▼
OPC UA HA
```
No component above the AquaCom protocol layer shall parse or construct AquaCom telegrams.
No AquaCom component shall depend upon OPC UA.
No AquaCom component shall depend upon a particular database.
No storage adapter shall depend upon AquaCom telegram structure.
---
# 3. Implementation agent principles
The implementation agent shall follow these principles throughout development.
## 3.1 Do not invent AquaCom protocol details
The implementation agent shall never invent:
- byte positions;
- telegram structure;
- field identifiers;
- checksums;
- addressing rules;
- commands;
- response codes;
- scaling;
- timestamp encoding;
- historical-data behavior.
Protocol behavior may only be implemented when supported by at least one of:
1. authoritative AquaCom documentation;
2. validated packet/serial captures;
3. existing known-good implementation behavior;
4. approved test vectors.
Unknown protocol details shall be represented as:
```text
NOT_IMPLEMENTED
```
or documented protocol gaps.
A compilation-successful guess is not an acceptable implementation.
---
# 4. Repository documentation strategy
The repository shall contain a short root `AGENTS.md`.
Detailed requirements shall live under `docs/`.
AGENTS.md shall primarily direct the implementation agent to the authoritative project documentation rather than attempt to contain the complete project specification.
This follows current agent guidance favoring concise repository instructions with deeper documentation maintained separately.
Complex implementation work shall use an execution plan maintained in the repository. Current agent guidance recommends plan documents for significant features or refactors.
---
# 5. Repository structure
The initial repository shall use the following logical organization:
```text
/
├── AGENTS.md
├── README.md
├── ARCHITECTURE.md
├── CHANGELOG.md
├── LICENSE
│
├── docs/
│ ├── product/
│ │ └── software-specification.md
│ │
│ ├── architecture/
│ │ ├── overview.md
│ │ ├── data-flow.md
│ │ ├── live-data.md
│ │ ├── historical-data.md
│ │ ├── opcua.md
│ │ ├── storage.md
│ │ └── security.md
│ │
│ ├── protocol/
│ │ ├── aquacom.md
│ │ ├── known-operations.md
│ │ ├── protocol-gaps.md
│ │ └── test-vectors.md
│ │
│ ├── configuration/
│ │ └── configuration-reference.md
│ │
│ ├── operations/
│ │ ├── deployment.md
│ │ ├── diagnostics.md
│ │ ├── backup-recovery.md
│ │ └── troubleshooting.md
│ │
│ └── exec-plans/
│ ├── active/
│ └── completed/
│
├── schemas/
│ ├── gateway-config.schema.*
│ ├── station-config.schema.*
│ └── point-config.schema.*
│
├── config/
│ └── examples/
│
├── src/
│ ├── core/
│ ├── aquacom/
│ ├── history/
│ ├── storage/
│ ├── opcua/
│ ├── diagnostics/
│ └── host/
│
├── tests/
│ ├── unit/
│ ├── integration/
│ ├── protocol/
│ ├── opcua/
│ ├── history/
│ └── acceptance/
│
├── simulator/
│
├── fixtures/
│ ├── aquacom/
│ ├── history/
│ └── configuration/
│
├── tools/
│
└── deployment/
```
File extensions shall be selected once the implementation language and configuration formats are chosen.
---
# 6. Root AGENTS.md
The root `AGENTS.md` implements this strategy: it is the short entry point that directs the implementation agent to the authoritative documents, lists the non-negotiable architecture and evidence rules, and never restates full specifications. The current root `AGENTS.md` in this repository is the reference implementation of that form and takes precedence over any example restated here.
Nested `AGENTS.md` files may later be added where a specific module requires additional constraints. Current coding agents support hierarchical `AGENTS.md` instructions scoped to directory trees, making this suitable for stricter protocol or storage-specific rules if the repository grows.
---
# 7. Architecture document
`ARCHITECTURE.md` shall act as the primary architecture map.
It shall contain:
- system purpose;
- module diagram;
- dependency direction;
- canonical data model;
- live-data flow;
- historical-data flow;
- security boundaries;
- links to detailed documents.
The following dependency direction is mandatory:
```text
┌──────────────┐
│ host │
└──────┬───────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
OPC UA history diagnostics
│ │
│ storage
│ │
└──────┬─────┘
▼
core
▲
│
aquacom
```
`core` shall contain no dependencies on AquaCom, OPC UA or a particular storage implementation.
---
# 8. Core domain module
The `core` module shall contain protocol-neutral data models.
Required logical entities include:
```text
StationId
PointId
CurrentValue
HistoricalSample
DataQuality
AcquisitionSource
StationState
CommunicationState
HistorySynchronizationState
```
---
# 9. Stable identifiers
## 9.1 StationId
`StationId` shall be an immutable logical gateway identifier.
It shall not depend solely upon:
- IP address;
- hostname;
- serial port;
- display name.
Example:
```text
station:PS001
```
---
## 9.2 PointId
`PointId` shall be an immutable logical identifier.
Example:
```text
station:PS001/point:LEVEL
```
Changing:
```text
Wet well level
```
to:
```text
Inlet sump level
```
shall not change `PointId`.
---
# 10. CurrentValue contract
The canonical current-value model shall conceptually contain:
```text
CurrentValue
StationId
PointId
Value
DataType
Quality
SourceTimestamp?
ReceivedTimestamp
UpdatedTimestamp
AcquisitionSource
```
`SourceTimestamp` may be absent when AquaCom provides no trustworthy source timestamp.
`ReceivedTimestamp` shall always be gateway-generated.
---
# 11. HistoricalSample contract
Historical samples shall contain:
```text
HistoricalSample
StationId
PointId
Value
DataType
Quality
SourceTimestamp
ReceivedTimestamp
AcquisitionSource
AcquisitionId?
Metadata?
```
`SourceTimestamp` is mandatory for normal process history.
A record without a reliable historical timestamp shall not silently be interpreted as historical process data. Such a record shall be **quarantined with diagnostics**: rejected from normal history, recorded distinctly for investigation, and never stored with a fabricated or gateway-generated timestamp (see `software-specification.md` §10).
---
# 12. DataQuality
Internal quality shall be independent of OPC UA.
At minimum:
```text
GOOD
UNCERTAIN
BAD
UNKNOWN
```
Optional reason/substatus shall support values equivalent to:
```text
COMMUNICATION_FAILURE
STALE
INVALID_VALUE
INVALID_TIMESTAMP
CONFIGURATION_ERROR
CONVERSION_ERROR
SOURCE_UNCERTAIN
HISTORY_GAP
```
The OPC UA module shall map this model to OPC UA StatusCodes.
---
# 13. AcquisitionSource
Historical and current information shall identify origin.
Examples:
```text
LIVE
AQUACOM_HISTORY
AQUACOM_REPORT
RECOVERY
IMPORT
SIMULATOR
```
Implementation shall permit additional values.
---
# 14. AquaCom transport interface
The AquaCom transport abstraction shall provide functions equivalent to:
```text
open()
close()
send(bytes)
receive()
isConnected()
getDiagnostics()
```
The interface shall not interpret AquaCom fields.
Potential transport implementations may include:
- TCP;
- serial;
- modem abstraction;
- serial-over-IP.
The initial implementation only needs transports required by the target environment.
---
# 15. AquaCom protocol codec
The protocol codec shall provide deterministic transformation between:
```text
raw transport bytes
```
and:
```text
AquaComMessage
```
Logical interfaces:
```text
encode(request) -> bytes
decode(bytes) -> message
validate(bytes) -> validation_result
```
The codec shall have no network dependency.
This is required so protocol behavior can be exhaustively unit tested.
---
# 16. Protocol test vectors
Every implemented AquaCom telegram type shall have test fixtures.
Fixture format shall contain conceptually:
```text
name
source
direction
raw-data
decoded-operation
decoded-fields
expected-validation
notes
```
Example source classifications:
```text
SPECIFICATION
CAPTURE
KNOWN_GOOD_IMPLEMENTATION
SYNTHETIC_FROM_SPECIFICATION
```
The test suite shall prove both:
```text
raw → decoded
```
and where applicable:
```text
object → encoded raw
```
---
# 17. Protocol gap registry
`docs/protocol/protocol-gaps.md` shall maintain unresolved protocol issues.
Each entry shall contain:
```text
Gap ID
Description
Affected operation
Evidence available
Evidence missing
Impact
Implementation status
```
Example:
```text
AQ-GAP-007
Exact encoding of history timestamp field for controller type X
is not yet verified.
Impact:
History retrieval cannot be enabled for this controller type.
Required evidence:
Protocol specification or validated capture.
```
The implementation agent shall not close a protocol gap without evidence.
### Evidence acquisition plan
Protocol evidence does not currently exist in this repository. The evidence rule above is therefore not executable until evidence is acquired. Until then, Phase 3 (codec framework) may proceed with the framework, fixture and simulation scaffolding, but no telegram behavior may be implemented.
Evidence shall be acquired through at least one of:
1. obtaining the authoritative AquaCom protocol specification from the equipment vendor or system owner;
2. capturing traffic from a known-good AquaCom deployment (sanitized per §64 before committing);
3. deriving behavior from an existing known-good integration.
The responsible party for obtaining the authoritative documentation shall be identified before Phase 3 starts, and the acquisition status shall be tracked in `docs/protocol/protocol-gaps.md` until resolved. Implementation milestones that depend on specific operations (live polling, history retrieval) are blocked by missing evidence for those operations and shall be sequenced after the evidence exists.
---
# 18. AquaCom session manager
Each configured station shall use an independent session/state abstraction.
Conceptual states:
```text
DISABLED
INITIALIZING
CONNECTING
ONLINE
DEGRADED
OFFLINE
RECOVERING
```
Basic transition:
```text
successful communication
OFFLINE ──────────────────────────────► ONLINE
ONLINE
│
│ repeated communication failures
▼
DEGRADED
│
│ threshold exceeded
▼
OFFLINE
│
│ transport available
▼
RECOVERING
│
│ successful synchronization
▼
ONLINE
```
Thresholds shall be configurable.
---
# 19. Live acquisition scheduler
The live acquisition scheduler shall:
- schedule AquaCom live requests;
- apply per-station polling configuration;
- prevent overlapping invalid station transactions;
- prioritize live communication over history backfill;
- implement retry policy;
- update communication diagnostics.
The scheduler shall not write directly to OPC UA.
It shall publish canonical `CurrentValue` updates.
---
# 20. Current-state service
The Current-State Service shall maintain the latest accepted current value for each PointId.
Interface equivalent:
```text
update(CurrentValue)
get(PointId)
getAll(StationId)
subscribe(PointId)
```
The service shall reject obsolete updates where appropriate.
A recovered historical sample shall not invoke:
```text
CurrentState.update()
```
unless it has independently been determined to represent current state.
---
# 21. Historical acquisition manager
The Historical Acquisition Manager shall manage history retrieval from AquaCom stations.
Responsibilities:
- determine retrieval windows;
- initiate historical requests;
- maintain synchronization watermark;
- process partial responses;
- resume interrupted synchronization;
- throttle retrieval;
- deduplicate;
- send canonical historical samples to the historical pipeline.
Conceptual interface:
```text
synchronize(station)
synchronize(station, start, end)
getSynchronizationState(station)
pause(station)
resume(station)
```
---
# 22. Synchronization watermark
Each station shall maintain persistent history synchronization state.
Example:
```text
StationId
HistoryStreamId
LastCompleteTimestamp
LastAttemptTimestamp
LastSuccessfulTimestamp
SynchronizationState
```
The watermark shall only advance when the implementation has sufficient evidence that the interval has been processed successfully. The precise advancement condition — durably spooled, not finally delivered — is defined in `ARCHITECTURE.md` §21; final delivery is tracked independently through per-destination spool acknowledgements.
---
# 23. Historical recovery state machine
Conceptual history states:
```text
IDLE
CHECK_REQUIRED_RANGE
REQUESTING
RECEIVING
PERSISTING
VERIFYING
COMPLETE
RETRY_WAIT
FAILED
```
Typical sequence:
```text
IDLE
│
▼
CHECK_REQUIRED_RANGE
│
▼
REQUESTING
│
▼
RECEIVING
│
▼
PERSISTING
│
▼
VERIFYING
│
├── incomplete ──► REQUESTING
│
└── complete ───► COMPLETE
```
A restart during synchronization shall resume from persistent state without requiring complete duplication of the historical interval.
---
# 24. Historical pipeline
The historical pipeline shall receive:
```text
HistoricalSample
```
and perform:
```text
validation
↓
normalization
↓
deduplication
↓
durable enqueue
↓
distribution
```
Conceptually:
```text
HistoricalSample
│
▼
Validator
│
▼
Deduplicator
│
▼
Durable spool
│
┌─────┼─────────┐
▼ ▼ ▼
Repo Store A Store B
```
---
# 25. Deduplication key
Default logical identity:
```text
StationId
+
PointId
+
SourceTimestamp
```
If AquaCom supplies a stronger identity such as a sequence number, the implementation may include it.
The deduplication algorithm shall be deterministic.
Collision behavior shall be configurable:
```text
KEEP_FIRST
KEEP_LAST
REJECT_CONFLICT
VERSION_CONFLICT
```
The configuration reference (`docs/configuration/configuration-reference.md` §24.2) names these policies `keep-first`, `keep-last`, `reject`, and `version`; the semantic mapping between the two namings shall be kept in the configuration model so the same policy cannot drift apart between documents.
Recommended default:
```text
REJECT_CONFLICT
```
where conflicting values are stored diagnostically rather than silently overwritten.
Deduplication state shall remain correct across restart (see `ARCHITECTURE.md` §23): a restart during synchronization shall not introduce duplicate records, either through persisted deduplication state or through idempotent storage.
The deduplication window shall be bounded.
---
# 26. History repository interface
A History Repository is a queryable source used by OPC UA Historical Access.
Required logical interface:
```text
store(sample)
storeBatch(samples)
readRaw(
pointId,
startTime,
endTime,
limit,
continuation
)
readAtTime(
pointId,
timestamps
)
getBounds(pointId)
health()
```
The repository may be:
- local;
- external;
- relational;
- time-series;
- industrial historian.
OPC UA shall interact only through this abstraction.
---
# 27. Storage sink interface
A storage sink is an independent history destination.
Required interface:
```text
initialize()
write(sample)
writeBatch(samples)
flush()
health()
shutdown()
```
A storage sink does not need to support queries.
Examples:
```text
SQL sink
time-series sink
industrial historian sink
file/archive sink
message-stream sink
```
---
# 28. Repository versus sink
These concepts shall remain distinct.
```text
Repository
= historical data can be queried
Sink
= historical data can be delivered
```
A component may implement both interfaces.
Example:
```text
Local historian:
Repository = yes
Sink = yes
Archive file:
Repository = no
Sink = yes
```
---
# 29. Durable spool
Historical delivery shall pass through persistent buffering.
Conceptual record:
```text
SpoolRecord
RecordId
HistoricalSample
CreatedTimestamp
Destinations[]
Acknowledgements[]
RetryState
```
Example:
```text
record 123
StorageLocal ACK
CentralHistorian PENDING
Archive ACK
```
Record 123 shall remain available until required delivery policy has been fulfilled.
---
# 30. Delivery policy
Configuration shall define whether a destination is:
```text
REQUIRED
OPTIONAL
```
Example:
```text
LocalHistory:
REQUIRED
CentralHistorian:
REQUIRED
ArchiveExport:
OPTIONAL
```
A failed optional destination shall generate diagnostics but shall not indefinitely block spool cleanup.
---
# 31. Storage failure handling
Each storage destination shall implement independent retry.
Recommended conceptual retry progression:
```text
immediate
short delay
increasing delay
maximum delay
periodic retry
```
Exact timing shall be configurable.
Retries shall not block live acquisition.
---
# 32. OPC UA module
The OPC UA module shall contain:
```text
Server lifecycle
Namespace manager
Node mapping
Current-value provider
Historical provider
StatusCode mapper
Security configuration
Diagnostics exposure
```
It shall depend upon:
```text
core
current-state service
history repository
diagnostics
```
It shall not depend directly upon AquaCom transport or AquaCom codec.
---
# 33. OPC UA namespace mapping
Default structure:
```text
AquaCom
│
├── Gateway
│ ├── Status
│ ├── Version
│ ├── StartTime
│ ├── ConfigurationVersion
│ ├── HistoricalQueueDepth
│ └── Storage
│
└── Stations
│
├── PS001
│ ├── Process
│ │ ├── Level
│ │ ├── Pump01Running
│ │ └── Pump02Running
│ │
│ └── Diagnostics
│ ├── CommunicationState
│ ├── LastCommunication
│ ├── HistoryState
│ └── HistoryWatermark
│
└── PS002
```
Namespace generation shall be deterministic.
---
# 34. OPC UA NodeId generation
OPC UA NodeIds shall derive from stable internal identifiers.
Example:
```text
ns=<gateway namespace>;s=station/PS001/point/LEVEL
```
The exact syntax may differ.
The critical requirement is:
```text
same logical point
=
same NodeId
```
across:
- restart;
- upgrade;
- rename;
- transport address change.
---
# 35. OPC UA current-value mapping
For each configured live point:
```text
CurrentValue.Value
→ OPC UA Value
CurrentValue.SourceTimestamp
→ OPC UA SourceTimestamp
gateway processing time
→ OPC UA ServerTimestamp
CurrentValue.Quality
→ OPC UA StatusCode
```
If communication fails, the node shall transition to a suitable non-Good status according to configuration.
---
# 36. OPC UA history provider
When an OPC UA client sends:
```text
HistoryRead
```
the OPC UA layer shall call:
```text
HistoryRepository.readRaw()
```
It shall never directly cause:
```text
AquaCom history request
```
The following is prohibited:
```text
OPC UA HistoryRead
│
▼
AquaCom WGH request
│
▼
wait for RTU
```
Correct behavior:
```text
AquaCom synchronization
│
▼
Repository
OPC UA HistoryRead
│
▼
Repository
```
---
# 37. OPC UA historical continuation
The implementation shall support bounded historical reads.
Configuration:
```text
opcua.historicalAccess.maxValuesPerRead
```
(This is the field defined in `docs/configuration/configuration-reference.md` §19.3. Earlier revisions of this document called it `maximumValuesPerHistoryRead`; the configuration-reference name is authoritative.)
If more values exist than can be returned:
```text
HistoryRead
│
▼
batch 1 + continuation token
│
▼
HistoryRead continuation
│
▼
batch 2
```
Continuation state shall be resource-limited.
---
# 38. Time handling
Internal canonical timestamps shall be timezone-unambiguous.
Preferred internal representation:
```text
UTC timestamp
```
Station configuration shall define AquaCom timezone interpretation where required.
Configuration fields shall include concepts equivalent to:
```text
timezone
timestampSource
maximumFutureDeviation
maximumPastDeviation
dstPolicy
```
The raw source timestamp may optionally also be retained for diagnostics.
---
# 39. Configuration model
A gateway configuration shall contain approximately:
```text
gateway
logging
opcua
history
storage
stations
security
diagnostics
```
---
# 40. Gateway configuration
Conceptual example:
```text
gateway:
id: aquacom-gateway-01
site: central-ot
timezone: Europe/Stockholm
```
---
# 41. Station configuration
Conceptual configuration:
```text
station:
id: PS001
enabled: true
transport:
type: tcp
...
aquacom:
address: ...
profile: ...
acquisition:
live:
enabled: true
interval: ...
history:
enabled: true
synchronization: ...
points:
source: ...
```
Exact syntax shall be selected during technology selection.
---
# 42. Point configuration
Conceptual:
```text
point:
id: LEVEL
name: Wet well level
source:
aquacomReference: ...
datatype: float
engineeringUnit: m
live:
enabled: true
history:
enabled: true
opcua:
enabled: true
historizing: true
storage:
enabled: true
```
---
# 43. Configuration schema validation
Machine-readable schemas shall validate configuration before startup.
Validation shall detect:
- duplicate station IDs;
- duplicate point IDs;
- invalid references;
- impossible polling intervals;
- unsupported transports;
- missing required storage;
- invalid timezone;
- duplicate OPC UA NodeIds;
- invalid security configuration.
Invalid configuration shall fail safely.
---
# 44. Secrets
Secrets shall be supplied through an external secret mechanism appropriate to the selected deployment environment.
The repository shall never contain production:
```text
passwords
private keys
API tokens
certificate private keys
database credentials
```
Example files shall use placeholders.
---
# 45. Diagnostics model
Gateway-level diagnostics:
```text
GatewayState
Version
ConfigurationVersion
StartTime
Uptime
StationCount
OnlineStationCount
HistoricalQueueDepth
SpoolDiskUsage
OPCUAState
```
Per-station:
```text
CommunicationState
LastRequest
LastResponse
LastSuccessfulCommunication
ConsecutiveFailures
ProtocolErrors
HistorySynchronizationState
HistoryWatermark
HistoricalBacklog
```
Per-storage destination:
```text
State
LastSuccessfulWrite
LastFailure
PendingRecords
Retries
Latency
```
---
# 46. Logging requirements
Structured logging shall be preferred.
Every log record should support context equivalent to:
```text
timestamp
severity
component
stationId?
pointId?
operation?
correlationId?
message
error?
```
Raw telegram logging shall be a separate debug facility.
Production default shall not log every telegram.
---
# 47. Correlation IDs
AquaCom transactions shall have a transaction/correlation identifier.
Example flow:
```text
AQ transaction 81245
│
├── send request
├── receive response
├── decode
├── update values
└── diagnostics
```
This ID should be propagated through relevant diagnostics.
---
# 48. Metrics
The implementation shall expose or internally maintain metrics equivalent to:
```text
aquacom_requests_total
aquacom_errors_total
aquacom_request_duration
station_online
live_values_processed
history_samples_received
history_samples_deduplicated
history_samples_pending
storage_writes_total
storage_write_errors
opcua_clients_connected
opcua_history_reads
spool_bytes
```
Specific metrics technology shall not be prescribed.
---
# 49. Simulator
A simulator shall be treated as a first-class component.
It shall support scripted station behavior.
Required simulation scenarios:
```text
normal live operation
communication timeout
connection loss
malformed response
bad checksum
delayed response
historical values
historical gap
duplicate history
conflicting duplicate
station clock error
reconnect and backfill
```
The simulator shall not share parsing code with the production AquaCom decoder in ways that make protocol tests self-validating.
---
# 50. Test pyramid
The project shall implement:
```text
Acceptance
/\
/ \
Integration
/ \
/ \
Unit tests
```
The majority of tests should be deterministic unit and component tests.
---
# 51. Required unit tests
## AquaCom codec
```text
framing
checksum
address encoding
telegram decoding
invalid telegram rejection
unexpected field handling
```
## Canonical model
```text
quality mapping
timestamp validation
stable identifier handling
```
## History
```text
deduplication
watermark progression
gap handling
conflict handling
spool acknowledgement
```
## OPC UA
```text
NodeId mapping
quality mapping
timestamp mapping
history result conversion
```
---
# 52. Required integration tests
At minimum:
```text
Simulator → AquaCom engine
AquaCom engine → Current State
AquaCom history → Historical pipeline
Historical pipeline → Repository
Historical pipeline → multiple sinks
Repository → OPC UA HistoryRead
Current state → OPC UA subscription
```
---
# 53. Restart recovery tests
Automated tests shall cover gateway restart during:
```text
normal operation
history retrieval
database outage
spool backlog
partial delivery
reconnect recovery
```
Expected result:
No acknowledged historical data is silently lost.
---
# 54. End-to-end mandatory acceptance test
Scenario:
```text
T0:
Station online.
Gateway receives live level values.
OPC UA client receives current values.
Values are persisted historically.
T1:
Station communication is interrupted.
Gateway marks communication degraded/bad.
Current OPC UA value is not falsely reported as newly valid.
T2:
Station continues local historical recording.
T3:
Communication returns.
Live acquisition resumes promptly.
T4:
Gateway identifies missing historical interval.
Gateway retrieves historical records.
T5:
Historical records retain their original timestamps.
They are delivered to all required storage destinations.
T6:
OPC UA current value remains the actual current process value.
T7:
OPC UA HistoryRead for T1–T4 returns recovered historical values.
T8:
Same historical interval is retrieved again.
No unintended duplicate historical records are created.
```
The system shall not be considered production-capable until this test passes.
---
# 55. Security tests
Tests shall verify:
```text
untrusted OPC UA client certificate rejected
trusted client accepted
insecure OPC UA endpoint disabled by production configuration
invalid credentials rejected where user authentication is enabled
private keys not logged
unauthorized HistoryUpdate rejected
configuration containing secret placeholders validated appropriately
```
---
# 56. Fault injection
Integration testing shall permit fault injection including:
```text
network disconnect
packet loss
delayed responses
storage timeout
storage exception
disk-full simulation
OPC UA client disconnect
process restart
corrupted local spool entry
```
---
# 57. CI requirements
Every change shall run:
```text
format/lint checks
static validation
unit tests
integration tests suitable for CI
configuration schema validation
dependency/security checks where available
```
Protocol fixtures shall be testable without external network access.
---
# 58. Pull request completion criteria
The implementation agent shall consider a development task complete only when:
```text
implementation exists
tests exist
tests pass
relevant documentation is updated
configuration examples are updated if required
no known architectural boundary has been violated
new protocol behavior has supporting evidence
new failure modes have diagnostics
no production secrets have been introduced
```
Implementation-agent task prompts work best when scoped similarly to concrete issues or PRs with explicit files and expected outcomes.
---
# 59. ExecPlan requirement
The implementation agent shall create an ExecPlan for work involving:
- new AquaCom operations;
- history synchronization;
- storage architecture;
- OPC UA namespace redesign;
- security changes;
- cross-module refactoring;
- persistent-data migration;
- changes affecting more than one architectural boundary.
ExecPlan structure:
```text
Title
Purpose
Current state
Desired state
Constraints
Files/components affected
Implementation sequence
Data model changes
Failure handling
Testing
Migration/compatibility
Progress
Decisions
Unexpected findings
Acceptance evidence
```
The plan shall remain updated while implementation proceeds.
---
# 60. Development phases
Implementation shall proceed in controlled phases.
---
# Phase 0 – Repository and architecture
Deliver:
```text
AGENTS.md
README.md
ARCHITECTURE.md
docs structure
CI skeleton
configuration skeleton
```
No AquaCom protocol implementation required.
Acceptance:
- repository builds;
- automated test runner works;
- architecture dependencies documented.
---
# Phase 1 – Canonical core
Implement:
```text
StationId
PointId
CurrentValue
HistoricalSample
DataQuality
AcquisitionSource
configuration domain
```
Acceptance:
- unit tests;
- serialization tests where applicable;
- stable identifier behavior defined.
---
# Phase 2 – AquaCom transport abstraction
Implement:
```text
transport interface
connection lifecycle
timeouts
retry primitives
diagnostics
```
Implement only required initial transport.
Acceptance:
- mock transport tests;
- connection-loss testing;
- reconnect testing.
---
# Phase 3 – AquaCom codec framework
Implement:
```text
frame abstraction
message abstraction
codec interfaces
validation framework
fixture framework
```
Only protocol elements with evidence may be implemented.
Acceptance:
```text
captured/documented telegram
↓
decoder
↓
expected structured representation
```
---
# Phase 4 – AquaCom simulator
Implement simulator before depending on field equipment for normal development.
Acceptance:
- successful transaction;
- timeout;
- malformed telegram;
- configurable process values.
---
# Phase 5 – Station/session manager
Implement:
```text
station lifecycle
communication state
poll scheduling
retry policy
diagnostics
```
Acceptance:
- multiple simulated stations;
- one station failing does not affect others.
---
# Phase 6 – Live acquisition
Implement:
```text
live AquaCom operation
point mapping
scaling
canonical CurrentValue
current-state cache
```
Acceptance:
```text
simulated AquaCom
↓
CurrentValue
```
with correct value, quality and timestamps.
---
# Phase 7 – OPC UA live interface
Implement:
```text
server
namespace
stable NodeIds
current reads
subscriptions
quality
timestamps
security
```
Acceptance:
An OPC UA test client can browse and subscribe to simulated AquaCom values.
---
# Phase 8 – Local historical repository
Implement the first queryable history repository.
The selected technology may be embedded or external, but shall implement the defined repository abstraction.
Acceptance:
```text
store
query interval
query bounds
restart persistence
```
---
# Phase 9 – Historical pipeline
Implement:
```text
validation
deduplication
durable spool
delivery
repository integration
```
Acceptance:
Multiple submissions of the same sample do not produce unintended duplicate records.
---
# Phase 10 – AquaCom historical acquisition
Implement only after sufficient protocol evidence is available.
Deliver:
```text
history retrieval
watermark
reconnect synchronization
backfill
throttling
partial recovery
```
Acceptance:
The simulator's missing time interval is recovered after reconnection.
---
# Phase 11 – OPC UA Historical Access
Implement:
```text
HistoryRead
raw values
timestamps
quality
continuation
```
Acceptance:
An OPC UA HA client can retrieve both originally stored and recovered historical values.
---
# Phase 12 – Multiple storage destinations
Implement storage sink abstraction plus second sink implementation.
Acceptance:
```text
one HistoricalSample
│
├── Repository A
└── Storage B
```
and failure of Storage B does not stop Repository A.
---
# Phase 13 – Store-and-forward resilience
Implement:
```text
persistent spool
destination acknowledgements
retry
backpressure
disk limits
```
Acceptance:
A storage destination may be offline while history accumulates and receive the backlog when restored.
---
# Phase 14 – Production hardening
Implement and validate:
```text
security
resource limits
performance
operations documentation
backup/recovery
deployment
certificate handling
structured diagnostics
long-duration tests
```
---
# 61. Implementation agent task sizing
Implementation phases shall be divided into bounded tasks rather than requesting the complete gateway in one prompt.
Preferred task structure:
```text
Goal
Relevant files
Required behavior
Constraints
Tests
Acceptance criteria
```
Example:
```text
Implement the HistoricalSample deduplication service.
Relevant:
src/history
docs/architecture/historical-data.md
Requirements:
- identity based on StationId + PointId + SourceTimestamp
- conflicting value must be detected
- default conflict policy REJECT_CONFLICT
Do not:
- introduce database-specific logic
- modify AquaCom codec
- modify OPC UA
Add unit tests covering:
- first sample
- exact duplicate
- conflicting duplicate
- different point
- different timestamp
```
---
# 62. Rule for incomplete requirements
When the implementation agent encounters an implementation ambiguity, it shall classify it as:
```text
A. Local implementation detail
B. Architectural decision
C. AquaCom protocol uncertainty
D. Safety/security uncertainty
```
For A:
Choose the simplest implementation consistent with architecture.
For B:
Record the decision in the current ExecPlan.
For C:
Do not guess. Add a protocol gap.
For D:
Choose the safer behavior and document the decision.
---
# 63. AquaCom source traceability
Every implemented protocol operation shall have traceability metadata in documentation.
Example:
```text
Operation: HistoricalRead-X
Evidence:
AquaCom specification section 8.3
Capture fixture AQ-HIST-004
Validated against ControllerType-A
Implemented:
yes
Tests:
protocol/history_read_004
```
This traceability shall make it possible to audit why the implementation behaves as it does.
---
# 64. Protocol capture handling
Production captures used as test fixtures shall be:
- sanitized;
- stripped of unnecessary network identifiers;
- documented;
- immutable once used as reference fixtures.
Fixture modification requires justification.
Raw captures shall not automatically be committed if they contain sensitive operational information.
---
# 65. Storage migration rule
Storage schemas shall be versioned.
Schema changes shall have explicit migrations.
Application startup shall not silently destroy incompatible historical data.
Destructive migration requires explicit administrative action.
---
# 66. Configuration versioning
The configuration model shall include a schema version.
Example:
```text
configVersion: 1
```
Future incompatible configuration changes shall provide validation and, where reasonable, migration tooling.
---
# 67. Software version exposure
Gateway software version shall be available through:
- startup logs;
- diagnostics;
- OPC UA diagnostics namespace.
Build metadata should include where possible:
```text
version
build identifier
source revision
configuration schema version
```
---
# 68. Compatibility strategy
The gateway shall distinguish:
```text
protocol compatibility
configuration compatibility
storage compatibility
OPC UA namespace compatibility
```
Changes to any of these shall be considered independently.
Stable OPC UA NodeIds are particularly important and shall not change unnecessarily between releases.
---
# 69. Performance validation
Before production acceptance, test:
```text
expected station count
expected point count
normal polling interval
normal historical rate
large backfill
simultaneous OPC UA subscriptions
large HistoryRead
storage unavailable
maximum expected spool backlog
```
The purpose is to establish capacity limits rather than merely demonstrate successful execution.
---
# 70. Live-priority scheduling
AquaCom scheduling shall conceptually prioritize:
```text
1. connection/health-critical transactions
2. current/live acquisition
3. alarm/event retrieval when implemented
4. requested configuration operations
5. historical recovery
6. low-priority maintenance
```
Historical recovery shall be throttled if required to maintain live acquisition responsiveness.
---
# 71. Memory and queue limits
Every queue shall be bounded.
No component may create an unbounded:
```text
message queue
history list
retry list
client response
log buffer
```
Queue-full behavior shall be explicitly defined.
---
# 72. Graceful shutdown
The application shall support controlled shutdown.
Order:
```text
stop accepting administration changes
↓
stop scheduling new acquisition
↓
complete/cancel active AquaCom transactions
↓
persist watermarks
↓
flush durable state
↓
stop OPC UA
↓
close storage
↓
terminate
```
---
# 73. Crash recovery
Following unclean shutdown:
```text
load persistent state
↓
validate spool
↓
recover pending deliveries
↓
restore synchronization watermarks
↓
reconnect stations
↓
resume live operation
↓
continue required historical synchronization
```
Live communication shall normally resume before large backfill jobs.
---
# 74. Initial production scope
The first production release shall include:
```text
AquaCom transport
AquaCom protocol framework
verified live-value acquisition
verified historical acquisition
multiple stations
canonical data model
live cache
historical pipeline
deduplication
durable spool
at least one history repository
storage adapter framework
at least two independently configurable storage targets
OPC UA Data Access
OPC UA Historical Access
OPC UA security
stable NodeIds
diagnostics
structured logging
simulator
automated tests
restart recovery
configuration validation
```
---
# 75. Explicitly excluded from first production release
Unless separately approved:
```text
AquaCom remote pump control
AquaCom setpoint write
OPC UA control writes
OPC UA Methods for process control
OPC UA HistoryUpdate
automatic historian correction
clustered high availability
automatic protocol discovery
```
Read-only process integration shall be established before remote-control functionality is introduced.
---
# 76. Future process-control architecture
If control is later implemented, it shall use a separate command pipeline:
```text
OPC UA
│
▼
authorization
│
▼
command validation
│
▼
command service
│
▼
AquaCom control operation
│
▼
confirmation/readback
│
▼
audit
```
Direct mapping:
```text
OPC UA write → raw AquaCom telegram
```
shall not be permitted.
---
# 77. Definition of Done
The gateway implementation is complete for production evaluation when:
1. All mandatory Phase 0–14 features are implemented.
2. No implemented AquaCom operation depends on undocumented guessed protocol behavior.
3. Current data is available through OPC UA.
4. Historical data is available through OPC UA Historical Access.
5. AquaCom history backfill does not alter current values.
6. Historical source timestamps are preserved.
7. Gateway restart does not lose acknowledged history.
8. Temporary storage outage does not immediately cause historical loss.
9. Multiple storage destinations operate independently.
10. OPC UA NodeIds are stable.
11. OPC UA security is enabled by default.
12. Simulator-based end-to-end acceptance tests pass.
13. Tests against real AquaCom equipment or validated traffic have passed.
14. Operational deployment and recovery documentation exists.
15. Remaining protocol gaps are explicitly documented.
---
# 78. Implementation objective
The resulting software shall behave as an integration platform:
```text
AquaCom
│
▼
┌───────────────────────┐
│ AquaCom Gateway │
│ │
│ Protocol │
│ Acquisition │
│ State │
│ History recovery │
│ Normalization │
│ Diagnostics │
└───────┬───────┬───────┘
│ │
LIVE │ │ HISTORY
│ │
▼ ▼
OPC UA Historical pipeline
│ │
│ ├─────────► Repository
│ ├─────────► Historian
│ ├─────────► Database
│ └─────────► Archive
│
└───────────────► OPC UA HA
```
The implementation shall remain usable even if:
- the SCADA platform changes;
- the historical database changes;
- additional storage destinations are introduced;
- AquaCom equipment is gradually replaced;
- additional northbound interfaces are introduced.
The gateway's permanent value shall therefore be the normalized information and integration boundary, not merely its OPC UA endpoint.
---
# 79. First implementation execution sequence
The recommended first implementation tasks are:
```text
Task 001
Create repository skeleton and architecture documents.
Task 002
Create core domain models and stable identifier rules.
Task 003
Create configuration model and schema validation.
Task 004
Create AquaCom transport abstraction and mock transport.
Task 005
Create AquaCom protocol codec framework without inventing protocol fields.
Task 006
Create protocol fixture/test-vector framework.
Task 007
Create AquaCom simulator architecture.
Task 008
Implement first documented live AquaCom operation.
Task 009
Implement Station Manager and live acquisition.
Task 010
Implement current-state service.
Task 011
Implement OPC UA live-value interface.
Task 012
Implement historical repository abstraction.
Task 013
Implement historical pipeline and deduplication.
Task 014
Implement durable spool.
Task 015
Implement first verified AquaCom historical operation.
Task 016
Implement backfill state machine.
Task 017
Implement OPC UA Historical Access.
Task 018
Implement second storage destination.
Task 019
Implement full reconnect/backfill acceptance test.
Task 020
Production hardening.
```
Each task shall result in a working repository state rather than leaving the main branch intentionally uncompilable or untestable.
---
# 80. Primary instruction to the implementation agent
When implementing this system, prioritize correctness, traceability and recoverability over speed of implementation.
In particular:
> Never compensate for missing AquaCom documentation by inventing plausible protocol behavior.
Build the architecture around the unknown portions first.
Add real AquaCom behavior only as verified protocol knowledge becomes available.
Every protocol feature shall be backed by evidence and automated tests.
Every architectural component shall be replaceable behind its documented interface.
Every historical sample shall retain its original process timestamp.
And no recovered historical value shall ever masquerade as a new live process value.
Reference in New Issue View Git Blame Copy Permalink