\\# Codex Implementation Specification \\## AquaCom Integration Gateway \\\*\\\*Document type:\\\*\\\* Implementation specification \\\*\\\*Target implementation agent:\\\*\\\* Codex \\\*\\\*System:\\\*\\\* AquaCom Integration Gateway \\\*\\\*Primary southbound interface:\\\*\\\* AquaCom \\\*\\\*Primary northbound interface:\\\*\\\* OPC UA \\\*\\\*Additional capability:\\\*\\\* Pluggable historical storage \\\*\\\*Programming language:\\\*\\\* To be selected during implementation planning \\\*\\\*Version:\\\*\\\* 0.1 \\\*\\\*Status:\\\*\\\* Draft \\--- \\# 1. Purpose This document defines how Codex shall implement the AquaCom Integration Gateway. The gateway shall: \\- communicate with AquaCom-compatible devices; \\- acquire current process information; \\- acquire historical process information; \\- maintain a protocol-independent internal data model; \\- expose current values through OPC UA; \\- expose historical values through OPC UA Historical Access; \\- persist historical information through one or several configurable storage providers; \\- recover historical data after communication interruptions where AquaCom equipment supports this; \\- maintain diagnostic information; \\- provide a simulation environment for development and automated testing. This document describes: \\- repository organization; \\- architectural boundaries; \\- required modules; \\- internal interfaces; \\- data contracts; \\- state machines; \\- configuration; \\- persistence; \\- testing; \\- milestones; \\- Codex working rules; \\- acceptance criteria. It does not prescribe the implementation programming language. \\--- \\# 2. Fundamental implementation rule The implementation shall be based on the following processing architecture: ```text AquaCom \ │ \ ▼ Transport \ │ \ ▼ AquaCom protocol engine \ │ \ ▼ Station/session manager \ │ \ ▼ Canonical data model \ │ \ ├──────────────► Current-state service \ │ │ \ │ ▼ \ │ OPC UA DA \ │ \ └──────────────► Historical pipeline \ │ \ ┌──────────┼──────────────┐ \ │ │ │ \ ▼ ▼ ▼ \ Repository Storage A Storage B \ │ \ ▼ \ OPC UA HA ``` No component above the AquaCom protocol layer shall parse or construct AquaCom telegrams. No AquaCom component shall depend upon OPC UA. No AquaCom component shall depend upon a particular database. No storage adapter shall depend upon AquaCom telegram structure. \\--- \\# 3. Codex implementation principles Codex shall follow these principles throughout development. \\## 3.1 Do not invent AquaCom protocol details Codex shall never invent: \\- byte positions; \\- telegram structure; \\- field identifiers; \\- checksums; \\- addressing rules; \\- commands; \\- response codes; \\- scaling; \\- timestamp encoding; \\- historical-data behavior. Protocol behavior may only be implemented when supported by at least one of: 1\\. authoritative AquaCom documentation; 2\\. validated packet/serial captures; 3\\. existing known-good implementation behavior; 4\\. approved test vectors. Unknown protocol details shall be represented as: ```text NOT\\\_IMPLEMENTED ``` or documented protocol gaps. A compilation-successful guess is not an acceptable implementation. \\--- \\# 4. Repository documentation strategy The repository shall contain a short root `AGENTS.md`. Detailed requirements shall live under `docs/`. `AGENTS.md` shall primarily direct Codex to the authoritative project documentation rather than attempt to contain the complete project specification. This follows current Codex guidance favoring concise repository instructions with deeper documentation maintained separately. Complex implementation work shall use an execution plan maintained in the repository. OpenAI's current Codex guidance explicitly recommends plan documents for significant features or refactors. \\--- \\# 5. Repository structure The initial repository shall use the following logical organization: ```text / ├── AGENTS.md ├── README.md ├── ARCHITECTURE.md ├── CHANGELOG.md ├── LICENSE │ ├── docs/ │ ├── product/ │ │ └── software-specification.md │ │ │ ├── architecture/ │ │ ├── overview.md │ │ ├── data-flow.md │ │ ├── live-data.md │ │ ├── historical-data.md │ │ ├── opcua.md │ │ ├── storage.md │ │ └── security.md │ │ │ ├── protocol/ │ │ ├── aquacom.md │ │ ├── known-operations.md │ │ ├── protocol-gaps.md │ │ └── test-vectors.md │ │ │ ├── configuration/ │ │ └── configuration-reference.md │ │ │ ├── operations/ │ │ ├── deployment.md │ │ ├── diagnostics.md │ │ ├── backup-recovery.md │ │ └── troubleshooting.md │ │ │ └── exec-plans/ │ ├── active/ │ └── completed/ │ ├── schemas/ │ ├── gateway-config.schema.\\\* │ ├── station-config.schema.\\\* │ └── point-config.schema.\\\* │ ├── config/ │ └── examples/ │ ├── src/ │ ├── core/ │ ├── aquacom/ │ ├── history/ │ ├── storage/ │ ├── opcua/ │ ├── diagnostics/ │ └── host/ │ ├── tests/ │ ├── unit/ │ ├── integration/ │ ├── protocol/ │ ├── opcua/ │ ├── history/ │ └── acceptance/ │ ├── simulator/ │ ├── fixtures/ │ ├── aquacom/ │ ├── history/ │ └── configuration/ │ ├── tools/ │ └── deployment/ ``` File extensions shall be selected once the implementation language and configuration formats are chosen. \\--- \\# 6. Root AGENTS.md The root `AGENTS.md` shall contain approximately the following instructions. ```text \\# AquaCom Integration Gateway Read ARCHITECTURE.md and the relevant documents under docs/ before changing implementation behavior. For substantial features, protocol changes, persistence changes, or cross-module refactors, create or update an ExecPlan under docs/exec-plans/active/. \\## Non-negotiable architecture rules 1\\. AquaCom protocol parsing belongs only in src/aquacom. 2\\. OPC UA must never parse AquaCom telegrams. 3\\. Storage adapters consume canonical historical records only. 4\\. Historical samples must never be replayed as current process values. 5\\. Preserve source timestamps separately from gateway receipt timestamps. 6\\. Stable point IDs and OPC UA NodeIds must survive restart and renaming. 7\\. Do not invent undocumented AquaCom protocol behavior. 8\\. Every protocol implementation requires automated test vectors. 9\\. Every bug fix requires a regression test where practical. 10\\. Live acquisition takes priority over historical backfill. \\## AquaCom protocol evidence Protocol behavior may only be implemented using documented or validated evidence stored or referenced from docs/protocol/. If information is insufficient, document the gap in docs/protocol/protocol-gaps.md instead of guessing. \\## Testing Run applicable unit, integration and acceptance tests before considering a task complete. Do not weaken or remove tests merely to make a change pass. \\## Security Never commit secrets, private keys, production certificates, passwords or production network addresses. OPC UA insecure modes must not become the production default. \\## Documentation Update relevant architecture or configuration documentation whenever observable behavior changes. ``` Nested `AGENTS.md` files may later be added where a specific module requires additional constraints. Codex supports hierarchical `AGENTS.md` instructions scoped to directory trees, making this suitable for stricter protocol or storage-specific rules if the repository grows. \\--- \\# 7. Architecture document `ARCHITECTURE.md` shall act as the primary architecture map. It shall contain: \\- system purpose; \\- module diagram; \\- dependency direction; \\- canonical data model; \\- live-data flow; \\- historical-data flow; \\- security boundaries; \\- links to detailed documents. The following dependency direction is mandatory: ```text \ ┌──────────────┐ \ │ host │ \ └──────┬───────┘ \ │ \ ┌────────────┼────────────┐ \ ▼ ▼ ▼ \ OPC UA history diagnostics \ │ │ \ │ storage \ │ │ \ └──────┬─────┘ \ ▼ \ core \ ▲ \ │ \ aquacom ``` `core` shall contain no dependencies on AquaCom, OPC UA or a particular storage implementation. \\--- \\# 8. Core domain module The `core` module shall contain protocol-neutral data models. Required logical entities include: ```text StationId PointId CurrentValue HistoricalSample DataQuality AcquisitionSource StationState CommunicationState HistorySynchronizationState ``` \\--- \\# 9. Stable identifiers \\## 9.1 StationId `StationId` shall be an immutable logical gateway identifier. It shall not depend solely upon: \\- IP address; \\- hostname; \\- serial port; \\- display name. Example: ```text station:PS001 ``` \\--- \\## 9.2 PointId `PointId` shall be an immutable logical identifier. Example: ```text station:PS001/point:LEVEL ``` Changing: ```text Wet well level ``` to: ```text Inlet sump level ``` shall not change `PointId`. \\--- \\# 10. CurrentValue contract The canonical current-value model shall conceptually contain: ```text CurrentValue StationId PointId Value DataType Quality SourceTimestamp? ReceivedTimestamp UpdatedTimestamp AcquisitionSource ``` `SourceTimestamp` may be absent when AquaCom provides no trustworthy source timestamp. `ReceivedTimestamp` shall always be gateway-generated. \\--- \\# 11. HistoricalSample contract Historical samples shall contain: ```text HistoricalSample StationId PointId Value DataType Quality SourceTimestamp ReceivedTimestamp AcquisitionSource AcquisitionId? Metadata? ``` `SourceTimestamp` is mandatory for normal process history. A record without a reliable historical timestamp shall not silently be interpreted as historical process data. \\--- \\# 12. DataQuality Internal quality shall be independent of OPC UA. At minimum: ```text GOOD UNCERTAIN BAD UNKNOWN ``` Optional reason/substatus shall support values equivalent to: ```text COMMUNICATION\\\_FAILURE STALE INVALID\\\_VALUE INVALID\\\_TIMESTAMP CONFIGURATION\\\_ERROR CONVERSION\\\_ERROR SOURCE\\\_UNCERTAIN HISTORY\\\_GAP ``` The OPC UA module shall map this model to OPC UA StatusCodes. \\--- \\# 13. AcquisitionSource Historical and current information shall identify origin. Examples: ```text LIVE AQUACOM\\\_HISTORY AQUACOM\\\_REPORT RECOVERY IMPORT SIMULATOR ``` Implementation shall permit additional values. \\--- \\# 14. AquaCom transport interface The AquaCom transport abstraction shall provide functions equivalent to: ```text open() close() send(bytes) receive() isConnected() getDiagnostics() ``` The interface shall not interpret AquaCom fields. Potential transport implementations may include: \\- TCP; \\- serial; \\- modem abstraction; \\- serial-over-IP. The initial implementation only needs transports required by the target environment. \\--- \\# 15. AquaCom protocol codec The protocol codec shall provide deterministic transformation between: ```text raw transport bytes ``` and: ```text AquaComMessage ``` Logical interfaces: ```text encode(request) -> bytes decode(bytes) -> message validate(bytes) -> validation\\\_result ``` The codec shall have no network dependency. This is required so protocol behavior can be exhaustively unit tested. \\--- \\# 16. Protocol test vectors Every implemented AquaCom telegram type shall have test fixtures. Fixture format shall contain conceptually: ```text name source direction raw-data decoded-operation decoded-fields expected-validation notes ``` Example source classifications: ```text SPECIFICATION CAPTURE KNOWN\\\_GOOD\\\_IMPLEMENTATION SYNTHETIC\\\_FROM\\\_SPECIFICATION ``` The test suite shall prove both: ```text raw → decoded ``` and where applicable: ```text object → encoded raw ``` \\--- \\# 17. Protocol gap registry `docs/protocol/protocol-gaps.md` shall maintain unresolved protocol issues. Each entry shall contain: ```text Gap ID Description Affected operation Evidence available Evidence missing Impact Implementation status ``` Example: ```text AQ-GAP-007 Exact encoding of history timestamp field for controller type X is not yet verified. Impact: History retrieval cannot be enabled for this controller type. Required evidence: Protocol specification or validated capture. ``` Codex shall not close a protocol gap without evidence. \\--- \\# 18. AquaCom session manager Each configured station shall use an independent session/state abstraction. Conceptual states: ```text DISABLED INITIALIZING CONNECTING ONLINE DEGRADED OFFLINE RECOVERING ``` Basic transition: ```text \ successful communication OFFLINE ──────────────────────────────► ONLINE ONLINE \ │ \ │ repeated communication failures \ ▼ DEGRADED \ │ \ │ threshold exceeded \ ▼ OFFLINE \ │ \ │ transport available \ ▼ RECOVERING \ │ \ │ successful synchronization \ ▼ ONLINE ``` Thresholds shall be configurable. \\--- \\# 19. Live acquisition scheduler The live acquisition scheduler shall: \\- schedule AquaCom live requests; \\- apply per-station polling configuration; \\- prevent overlapping invalid station transactions; \\- prioritize live communication over history backfill; \\- implement retry policy; \\- update communication diagnostics. The scheduler shall not write directly to OPC UA. It shall publish canonical `CurrentValue` updates. \\--- \\# 20. Current-state service The Current-State Service shall maintain the latest accepted current value for each PointId. Interface equivalent: ```text update(CurrentValue) get(PointId) getAll(StationId) subscribe(PointId) ``` The service shall reject obsolete updates where appropriate. A recovered historical sample shall not invoke: ```text CurrentState.update() ``` unless it has independently been determined to represent current state. \\--- \\# 21. Historical acquisition manager The Historical Acquisition Manager shall manage history retrieval from AquaCom stations. Responsibilities: \\- determine retrieval windows; \\- initiate historical requests; \\- maintain synchronization watermark; \\- process partial responses; \\- resume interrupted synchronization; \\- throttle retrieval; \\- deduplicate; \\- send canonical historical samples to the historical pipeline. Conceptual interface: ```text synchronize(station) synchronize(station, start, end) getSynchronizationState(station) pause(station) resume(station) ``` \\--- \\# 22. Synchronization watermark Each station shall maintain persistent history synchronization state. Example: ```text StationId HistoryStreamId LastCompleteTimestamp LastAttemptTimestamp LastSuccessfulTimestamp SynchronizationState ``` The watermark shall only advance when the implementation has sufficient evidence that the interval has been processed successfully. \\--- \\# 23. Historical recovery state machine Conceptual history states: ```text IDLE CHECK\\\_REQUIRED\\\_RANGE REQUESTING RECEIVING PERSISTING VERIFYING COMPLETE RETRY\\\_WAIT FAILED ``` Typical sequence: ```text IDLE \ │ \ ▼ CHECK\\\_REQUIRED\\\_RANGE \ │ \ ▼ REQUESTING \ │ \ ▼ RECEIVING \ │ \ ▼ PERSISTING \ │ \ ▼ VERIFYING \ │ \ ├── incomplete ──► REQUESTING \ │ \ └── complete ───► COMPLETE ``` A restart during synchronization shall resume from persistent state without requiring complete duplication of the historical interval. \\--- \\# 24. Historical pipeline The historical pipeline shall receive: ```text HistoricalSample ``` and perform: ```text validation \ ↓ normalization \ ↓ deduplication \ ↓ durable enqueue \ ↓ distribution ``` Conceptually: ```text HistoricalSample \ │ \ ▼ Validator \ │ \ ▼ Deduplicator \ │ \ ▼ Durable spool \ │ \ ┌─────┼─────────┐ \ ▼ ▼ ▼ Repo Store A Store B ``` \\--- \\# 25. Deduplication key Default logical identity: ```text StationId \\+ PointId \\+ SourceTimestamp ``` If AquaCom supplies a stronger identity such as a sequence number, the implementation may include it. The deduplication algorithm shall be deterministic. Collision behavior shall be configurable: ```text KEEP\\\_FIRST KEEP\\\_LAST REJECT\\\_CONFLICT VERSION\\\_CONFLICT ``` Recommended default: ```text REJECT\\\_CONFLICT ``` where conflicting values are stored diagnostically rather than silently overwritten. \\--- \\# 26. History repository interface A History Repository is a queryable source used by OPC UA Historical Access. Required logical interface: ```text store(sample) storeBatch(samples) readRaw( \ pointId, \ startTime, \ endTime, \ limit, \ continuation ) readAtTime( \ pointId, \ timestamps ) getBounds(pointId) health() ``` The repository may be: \\- local; \\- external; \\- relational; \\- time-series; \\- industrial historian. OPC UA shall interact only through this abstraction. \\--- \\# 27. Storage sink interface A storage sink is an independent history destination. Required interface: ```text initialize() write(sample) writeBatch(samples) flush() health() shutdown() ``` A storage sink does not need to support queries. Examples: ```text SQL sink time-series sink industrial historian sink file/archive sink message-stream sink ``` \\--- \\# 28. Repository versus sink These concepts shall remain distinct. ```text Repository \ = historical data can be queried Sink \ = historical data can be delivered ``` A component may implement both interfaces. Example: ```text Local historian: \ Repository = yes \ Sink = yes Archive file: \ Repository = no \ Sink = yes ``` \\--- \\# 29. Durable spool Historical delivery shall pass through persistent buffering. Conceptual record: ```text SpoolRecord RecordId HistoricalSample CreatedTimestamp Destinations\\\[] Acknowledgements\\\[] RetryState ``` Example: ```text record 123 StorageLocal ACK CentralHistorian PENDING Archive ACK ``` Record 123 shall remain available until required delivery policy has been fulfilled. \\--- \\# 30. Delivery policy Configuration shall define whether a destination is: ```text REQUIRED OPTIONAL ``` Example: ```text LocalHistory: \ REQUIRED CentralHistorian: \ REQUIRED ArchiveExport: \ OPTIONAL ``` A failed optional destination shall generate diagnostics but shall not indefinitely block spool cleanup. \\--- \\# 31. Storage failure handling Each storage destination shall implement independent retry. Recommended conceptual retry progression: ```text immediate short delay increasing delay maximum delay periodic retry ``` Exact timing shall be configurable. Retries shall not block live acquisition. \\--- \\# 32. OPC UA module The OPC UA module shall contain: ```text Server lifecycle Namespace manager Node mapping Current-value provider Historical provider StatusCode mapper Security configuration Diagnostics exposure ``` It shall depend upon: ```text core current-state service history repository diagnostics ``` It shall not depend directly upon AquaCom transport or AquaCom codec. \\--- \\# 33. OPC UA namespace mapping Default structure: ```text AquaCom │ ├── Gateway │ ├── Status │ ├── Version │ ├── StartTime │ ├── ConfigurationVersion │ ├── HistoricalQueueDepth │ └── Storage │ └── Stations \ │ \ ├── PS001 \ │ ├── Process \ │ │ ├── Level \ │ │ ├── Pump01Running \ │ │ └── Pump02Running \ │ │ \ │ └── Diagnostics \ │ ├── CommunicationState \ │ ├── LastCommunication \ │ ├── HistoryState \ │ └── HistoryWatermark \ │ \ └── PS002 ``` Namespace generation shall be deterministic. \\--- \\# 34. OPC UA NodeId generation OPC UA NodeIds shall derive from stable internal identifiers. Example: ```text ns=;s=station/PS001/point/LEVEL ``` The exact syntax may differ. The critical requirement is: ```text same logical point \ = same NodeId ``` across: \\- restart; \\- upgrade; \\- rename; \\- transport address change. \\--- \\# 35. OPC UA current-value mapping For each configured live point: ```text CurrentValue.Value \ → OPC UA Value CurrentValue.SourceTimestamp \ → OPC UA SourceTimestamp gateway processing time \ → OPC UA ServerTimestamp CurrentValue.Quality \ → OPC UA StatusCode ``` If communication fails, the node shall transition to a suitable non-Good status according to configuration. \\--- \\# 36. OPC UA history provider When an OPC UA client sends: ```text HistoryRead ``` the OPC UA layer shall call: ```text HistoryRepository.readRaw() ``` It shall never directly cause: ```text AquaCom history request ``` The following is prohibited: ```text OPC UA HistoryRead \ │ \ ▼ AquaCom WGH request \ │ \ ▼ wait for RTU ``` Correct behavior: ```text AquaCom synchronization \ │ \ ▼ Repository OPC UA HistoryRead \ │ \ ▼ Repository ``` \\--- \\# 37. OPC UA historical continuation The implementation shall support bounded historical reads. Configuration: ```text maximumValuesPerHistoryRead ``` If more values exist than can be returned: ```text HistoryRead \ │ \ ▼ batch 1 + continuation token \ │ \ ▼ HistoryRead continuation \ │ \ ▼ batch 2 ``` Continuation state shall be resource-limited. \\--- \\# 38. Time handling Internal canonical timestamps shall be timezone-unambiguous. Preferred internal representation: ```text UTC timestamp ``` Station configuration shall define AquaCom timezone interpretation where required. Configuration fields shall include concepts equivalent to: ```text timezone timestampSource maximumFutureDeviation maximumPastDeviation dstPolicy ``` The raw source timestamp may optionally also be retained for diagnostics. \\--- \\# 39. Configuration model A gateway configuration shall contain approximately: ```text gateway logging opcua history storage stations security diagnostics ``` \\--- \\# 40. Gateway configuration Conceptual example: ```text gateway: \ id: aquacom-gateway-01 \ site: central-ot \ timezone: Europe/Stockholm ``` \\--- \\# 41. Station configuration Conceptual configuration: ```text station: \ id: PS001 \ enabled: true \ transport: \ type: tcp \ ... \ aquacom: \ address: ... \ profile: ... \ acquisition: \ live: \ enabled: true \ interval: ... \ history: \ enabled: true \ synchronization: ... \ points: \ source: ... ``` Exact syntax shall be selected during technology selection. \\--- \\# 42. Point configuration Conceptual: ```text point: \ id: LEVEL \ name: Wet well level \ source: \ aquacomReference: ... \ datatype: float \ engineeringUnit: m \ live: \ enabled: true \ history: \ enabled: true \ opcua: \ enabled: true \ historizing: true \ storage: \ enabled: true ``` \\--- \\# 43. Configuration schema validation Machine-readable schemas shall validate configuration before startup. Validation shall detect: \\- duplicate station IDs; \\- duplicate point IDs; \\- invalid references; \\- impossible polling intervals; \\- unsupported transports; \\- missing required storage; \\- invalid timezone; \\- duplicate OPC UA NodeIds; \\- invalid security configuration. Invalid configuration shall fail safely. \\--- \\# 44. Secrets Secrets shall be supplied through an external secret mechanism appropriate to the selected deployment environment. The repository shall never contain production: ```text passwords private keys API tokens certificate private keys database credentials ``` Example files shall use placeholders. \\--- \\# 45. Diagnostics model Gateway-level diagnostics: ```text GatewayState Version ConfigurationVersion StartTime Uptime StationCount OnlineStationCount HistoricalQueueDepth SpoolDiskUsage OPCUAState ``` Per-station: ```text CommunicationState LastRequest LastResponse LastSuccessfulCommunication ConsecutiveFailures ProtocolErrors HistorySynchronizationState HistoryWatermark HistoricalBacklog ``` Per-storage destination: ```text State LastSuccessfulWrite LastFailure PendingRecords Retries Latency ``` \\--- \\# 46. Logging requirements Structured logging shall be preferred. Every log record should support context equivalent to: ```text timestamp severity component stationId? pointId? operation? correlationId? message error? ``` Raw telegram logging shall be a separate debug facility. Production default shall not log every telegram. \\--- \\# 47. Correlation IDs AquaCom transactions shall have a transaction/correlation identifier. Example flow: ```text AQ transaction 81245 \ │ \ ├── send request \ ├── receive response \ ├── decode \ ├── update values \ └── diagnostics ``` This ID should be propagated through relevant diagnostics. \\--- \\# 48. Metrics The implementation shall expose or internally maintain metrics equivalent to: ```text aquacom\\\_requests\\\_total aquacom\\\_errors\\\_total aquacom\\\_request\\\_duration station\\\_online live\\\_values\\\_processed history\\\_samples\\\_received history\\\_samples\\\_deduplicated history\\\_samples\\\_pending storage\\\_writes\\\_total storage\\\_write\\\_errors opcua\\\_clients\\\_connected opcua\\\_history\\\_reads spool\\\_bytes ``` Specific metrics technology shall not be prescribed. \\--- \\# 49. Simulator A simulator shall be treated as a first-class component. It shall support scripted station behavior. Required simulation scenarios: ```text normal live operation communication timeout connection loss malformed response bad checksum delayed response historical values historical gap duplicate history conflicting duplicate station clock error reconnect and backfill ``` The simulator shall not share parsing code with the production AquaCom decoder in ways that make protocol tests self-validating. \\--- \\# 50. Test pyramid The project shall implement: ```text \ Acceptance \ /\\\\ \ / \\\\ \ Integration \ / \\\\ \ / \\\\ \ Unit tests ``` The majority of tests should be deterministic unit and component tests. \\--- \\# 51. Required unit tests \\## AquaCom codec ```text framing checksum address encoding telegram decoding invalid telegram rejection unexpected field handling ``` \\## Canonical model ```text quality mapping timestamp validation stable identifier handling ``` \\## History ```text deduplication watermark progression gap handling conflict handling spool acknowledgement ``` \\## OPC UA ```text NodeId mapping quality mapping timestamp mapping history result conversion ``` \\--- \\# 52. Required integration tests At minimum: ```text Simulator → AquaCom engine AquaCom engine → Current State AquaCom history → Historical pipeline Historical pipeline → Repository Historical pipeline → multiple sinks Repository → OPC UA HistoryRead Current state → OPC UA subscription ``` \\--- \\# 53. Restart recovery tests Automated tests shall cover gateway restart during: ```text normal operation history retrieval database outage spool backlog partial delivery reconnect recovery ``` Expected result: No acknowledged historical data is silently lost. \\--- \\# 54. End-to-end mandatory acceptance test Scenario: ```text T0: Station online. Gateway receives live level values. OPC UA client receives current values. Values are persisted historically. T1: Station communication is interrupted. Gateway marks communication degraded/bad. Current OPC UA value is not falsely reported as newly valid. T2: Station continues local historical recording. T3: Communication returns. Live acquisition resumes promptly. T4: Gateway identifies missing historical interval. Gateway retrieves historical records. T5: Historical records retain their original timestamps. They are delivered to all required storage destinations. T6: OPC UA current value remains the actual current process value. T7: OPC UA HistoryRead for T1–T4 returns recovered historical values. T8: Same historical interval is retrieved again. No unintended duplicate historical records are created. ``` The system shall not be considered production-capable until this test passes. \\--- \\# 55. Security tests Tests shall verify: ```text untrusted OPC UA client certificate rejected trusted client accepted insecure OPC UA endpoint disabled by production configuration invalid credentials rejected where user authentication is enabled private keys not logged unauthorized HistoryUpdate rejected configuration containing secret placeholders validated appropriately ``` \\--- \\# 56. Fault injection Integration testing shall permit fault injection including: ```text network disconnect packet loss delayed responses storage timeout storage exception disk-full simulation OPC UA client disconnect process restart corrupted local spool entry ``` \\--- \\# 57. CI requirements Every change shall run: ```text format/lint checks static validation unit tests integration tests suitable for CI configuration schema validation dependency/security checks where available ``` Protocol fixtures shall be testable without external network access. \\--- \\# 58. Pull request completion criteria Codex shall consider a development task complete only when: ```text implementation exists tests exist tests pass relevant documentation is updated configuration examples are updated if required no known architectural boundary has been violated new protocol behavior has supporting evidence new failure modes have diagnostics no production secrets have been introduced ``` Codex task prompts work best when scoped similarly to concrete issues or PRs with explicit files and expected outcomes. \\--- \\# 59. ExecPlan requirement Codex shall create an ExecPlan for work involving: \\- new AquaCom operations; \\- history synchronization; \\- storage architecture; \\- OPC UA namespace redesign; \\- security changes; \\- cross-module refactoring; \\- persistent-data migration; \\- changes affecting more than one architectural boundary. ExecPlan structure: ```text Title Purpose Current state Desired state Constraints Files/components affected Implementation sequence Data model changes Failure handling Testing Migration/compatibility Progress Decisions Unexpected findings Acceptance evidence ``` The plan shall remain updated while implementation proceeds. \\--- \\# 60. Development phases Implementation shall proceed in controlled phases. \\--- \\# Phase 0 – Repository and architecture Deliver: ```text AGENTS.md README.md ARCHITECTURE.md docs structure CI skeleton configuration skeleton ``` No AquaCom protocol implementation required. Acceptance: \\- repository builds; \\- automated test runner works; \\- architecture dependencies documented. \\--- \\# Phase 1 – Canonical core Implement: ```text StationId PointId CurrentValue HistoricalSample DataQuality AcquisitionSource configuration domain ``` Acceptance: \\- unit tests; \\- serialization tests where applicable; \\- stable identifier behavior defined. \\--- \\# Phase 2 – AquaCom transport abstraction Implement: ```text transport interface connection lifecycle timeouts retry primitives diagnostics ``` Implement only required initial transport. Acceptance: \\- mock transport tests; \\- connection-loss testing; \\- reconnect testing. \\--- \\# Phase 3 – AquaCom codec framework Implement: ```text frame abstraction message abstraction codec interfaces validation framework fixture framework ``` Only protocol elements with evidence may be implemented. Acceptance: ```text captured/documented telegram \ ↓ decoder \ ↓ expected structured representation ``` \\--- \\# Phase 4 – AquaCom simulator Implement simulator before depending on field equipment for normal development. Acceptance: \\- successful transaction; \\- timeout; \\- malformed telegram; \\- configurable process values. \\--- \\# Phase 5 – Station/session manager Implement: ```text station lifecycle communication state poll scheduling retry policy diagnostics ``` Acceptance: \\- multiple simulated stations; \\- one station failing does not affect others. \\--- \\# Phase 6 – Live acquisition Implement: ```text live AquaCom operation point mapping scaling canonical CurrentValue current-state cache ``` Acceptance: ```text simulated AquaCom \ ↓ CurrentValue ``` with correct value, quality and timestamps. \\--- \\# Phase 7 – OPC UA live interface Implement: ```text server namespace stable NodeIds current reads subscriptions quality timestamps security ``` Acceptance: An OPC UA test client can browse and subscribe to simulated AquaCom values. \\--- \\# Phase 8 – Local historical repository Implement the first queryable history repository. The selected technology may be embedded or external, but shall implement the defined repository abstraction. Acceptance: ```text store query interval query bounds restart persistence ``` \\--- \\# Phase 9 – Historical pipeline Implement: ```text validation deduplication durable spool delivery repository integration ``` Acceptance: Multiple submissions of the same sample do not produce unintended duplicate records. \\--- \\# Phase 10 – AquaCom historical acquisition Implement only after sufficient protocol evidence is available. Deliver: ```text history retrieval watermark reconnect synchronization backfill throttling partial recovery ``` Acceptance: The simulator's missing time interval is recovered after reconnection. \\--- \\# Phase 11 – OPC UA Historical Access Implement: ```text HistoryRead raw values timestamps quality continuation ``` Acceptance: An OPC UA HA client can retrieve both originally stored and recovered historical values. \\--- \\# Phase 12 – Multiple storage destinations Implement storage sink abstraction plus second sink implementation. Acceptance: ```text one HistoricalSample \ │ \ ├── Repository A \ └── Storage B ``` and failure of Storage B does not stop Repository A. \\--- \\# Phase 13 – Store-and-forward resilience Implement: ```text persistent spool destination acknowledgements retry backpressure disk limits ``` Acceptance: A storage destination may be offline while history accumulates and receive the backlog when restored. \\--- \\# Phase 14 – Production hardening Implement and validate: ```text security resource limits performance operations documentation backup/recovery deployment certificate handling structured diagnostics long-duration tests ``` \\--- \\# 61. Codex task sizing Implementation phases shall be divided into bounded tasks rather than requesting the complete gateway in one prompt. Preferred task structure: ```text Goal Relevant files Required behavior Constraints Tests Acceptance criteria ``` Example: ```text Implement the HistoricalSample deduplication service. Relevant: src/history docs/architecture/historical-data.md Requirements: \\- identity based on StationId + PointId + SourceTimestamp \\- conflicting value must be detected \\- default conflict policy REJECT\\\_CONFLICT Do not: \\- introduce database-specific logic \\- modify AquaCom codec \\- modify OPC UA Add unit tests covering: \\- first sample \\- exact duplicate \\- conflicting duplicate \\- different point \\- different timestamp ``` \\--- \\# 62. Codex rule for incomplete requirements When Codex encounters an implementation ambiguity, it shall classify it as: ```text A. Local implementation detail B. Architectural decision C. AquaCom protocol uncertainty D. Safety/security uncertainty ``` For A: Choose the simplest implementation consistent with architecture. For B: Record the decision in the current ExecPlan. For C: Do not guess. Add a protocol gap. For D: Choose the safer behavior and document the decision. \\--- \\# 63. AquaCom source traceability Every implemented protocol operation shall have traceability metadata in documentation. Example: ```text Operation: HistoricalRead-X Evidence: AquaCom specification section 8.3 Capture fixture AQ-HIST-004 Validated against ControllerType-A Implemented: yes Tests: protocol/history\\\_read\\\_004 ``` This traceability shall make it possible to audit why the implementation behaves as it does. \\--- \\# 64. Protocol capture handling Production captures used as test fixtures shall be: \\- sanitized; \\- stripped of unnecessary network identifiers; \\- documented; \\- immutable once used as reference fixtures. Fixture modification requires justification. Raw captures shall not automatically be committed if they contain sensitive operational information. \\--- \\# 65. Storage migration rule Storage schemas shall be versioned. Schema changes shall have explicit migrations. Application startup shall not silently destroy incompatible historical data. Destructive migration requires explicit administrative action. \\--- \\# 66. Configuration versioning The configuration model shall include a schema version. Example: ```text configVersion: 1 ``` Future incompatible configuration changes shall provide validation and, where reasonable, migration tooling. \\--- \\# 67. Software version exposure Gateway software version shall be available through: \\- startup logs; \\- diagnostics; \\- OPC UA diagnostics namespace. Build metadata should include where possible: ```text version build identifier source revision configuration schema version ``` \\--- \\# 68. Compatibility strategy The gateway shall distinguish: ```text protocol compatibility configuration compatibility storage compatibility OPC UA namespace compatibility ``` Changes to any of these shall be considered independently. Stable OPC UA NodeIds are particularly important and shall not change unnecessarily between releases. \\--- \\# 69. Performance validation Before production acceptance, test: ```text expected station count expected point count normal polling interval normal historical rate large backfill simultaneous OPC UA subscriptions large HistoryRead storage unavailable maximum expected spool backlog ``` The purpose is to establish capacity limits rather than merely demonstrate successful execution. \\--- \\# 70. Live-priority scheduling AquaCom scheduling shall conceptually prioritize: ```text 1\\. connection/health-critical transactions 2\\. current/live acquisition 3\\. alarm/event retrieval when implemented 4\\. requested configuration operations 5\\. historical recovery 6\\. low-priority maintenance ``` Historical recovery shall be throttled if required to maintain live acquisition responsiveness. \\--- \\# 71. Memory and queue limits Every queue shall be bounded. No component may create an unbounded: ```text message queue history list retry list client response log buffer ``` Queue-full behavior shall be explicitly defined. \\--- \\# 72. Graceful shutdown The application shall support controlled shutdown. Order: ```text stop accepting administration changes \ ↓ stop scheduling new acquisition \ ↓ complete/cancel active AquaCom transactions \ ↓ persist watermarks \ ↓ flush durable state \ ↓ stop OPC UA \ ↓ close storage \ ↓ terminate ``` \\--- \\# 73. Crash recovery Following unclean shutdown: ```text load persistent state \ ↓ validate spool \ ↓ recover pending deliveries \ ↓ restore synchronization watermarks \ ↓ reconnect stations \ ↓ resume live operation \ ↓ continue required historical synchronization ``` Live communication shall normally resume before large backfill jobs. \\--- \\# 74. Initial production scope The first production release shall include: ```text AquaCom transport AquaCom protocol framework verified live-value acquisition verified historical acquisition multiple stations canonical data model live cache historical pipeline deduplication durable spool at least one history repository storage adapter framework at least two independently configurable storage targets OPC UA Data Access OPC UA Historical Access OPC UA security stable NodeIds diagnostics structured logging simulator automated tests restart recovery configuration validation ``` \\--- \\# 75. Explicitly excluded from first production release Unless separately approved: ```text AquaCom remote pump control AquaCom setpoint write OPC UA control writes OPC UA Methods for process control OPC UA HistoryUpdate automatic historian correction clustered high availability automatic protocol discovery ``` Read-only process integration shall be established before remote-control functionality is introduced. \\--- \\# 76. Future process-control architecture If control is later implemented, it shall use a separate command pipeline: ```text OPC UA \ │ \ ▼ authorization \ │ \ ▼ command validation \ │ \ ▼ command service \ │ \ ▼ AquaCom control operation \ │ \ ▼ confirmation/readback \ │ \ ▼ audit ``` Direct mapping: ```text OPC UA write → raw AquaCom telegram ``` shall not be permitted. \\--- \\# 77. Definition of Done The gateway implementation is complete for production evaluation when: 1\\. All mandatory Phase 0–14 features are implemented. 2\\. No implemented AquaCom operation depends on undocumented guessed protocol behavior. 3\\. Current data is available through OPC UA. 4\\. Historical data is available through OPC UA Historical Access. 5\\. AquaCom history backfill does not alter current values. 6\\. Historical source timestamps are preserved. 7\\. Gateway restart does not lose acknowledged history. 8\\. Temporary storage outage does not immediately cause historical loss. 9\\. Multiple storage destinations operate independently. 10\\. OPC UA NodeIds are stable. 11\\. OPC UA security is enabled by default. 12\\. Simulator-based end-to-end acceptance tests pass. 13\\. Tests against real AquaCom equipment or validated traffic have passed. 14\\. Operational deployment and recovery documentation exists. 15\\. Remaining protocol gaps are explicitly documented. \\--- \\# 78. Implementation objective The resulting software shall behave as an integration platform: ```text \ AquaCom \ │ \ ▼ \ ┌───────────────────────┐ \ │ AquaCom Gateway │ \ │ │ \ │ Protocol │ \ │ Acquisition │ \ │ State │ \ │ History recovery │ \ │ Normalization │ \ │ Diagnostics │ \ └───────┬───────┬───────┘ \ │ │ \ LIVE │ │ HISTORY \ │ │ \ ▼ ▼ \ OPC UA Historical pipeline \ │ │ \ │ ├─────────► Repository \ │ ├─────────► Historian \ │ ├─────────► Database \ │ └─────────► Archive \ │ \ └───────────────► OPC UA HA ``` The implementation shall remain usable even if: \\- the SCADA platform changes; \\- the historical database changes; \\- additional storage destinations are introduced; \\- AquaCom equipment is gradually replaced; \\- additional northbound interfaces are introduced. The gateway's permanent value shall therefore be the normalized information and integration boundary, not merely its OPC UA endpoint. \\--- \\# 79. First Codex execution sequence The recommended first Codex tasks are: ```text Task 001 Create repository skeleton and architecture documents. Task 002 Create core domain models and stable identifier rules. Task 003 Create configuration model and schema validation. Task 004 Create AquaCom transport abstraction and mock transport. Task 005 Create AquaCom protocol codec framework without inventing protocol fields. Task 006 Create protocol fixture/test-vector framework. Task 007 Create AquaCom simulator architecture. Task 008 Implement first documented live AquaCom operation. Task 009 Implement Station Manager and live acquisition. Task 010 Implement current-state service. Task 011 Implement OPC UA live-value interface. Task 012 Implement historical repository abstraction. Task 013 Implement historical pipeline and deduplication. Task 014 Implement durable spool. Task 015 Implement first verified AquaCom historical operation. Task 016 Implement backfill state machine. Task 017 Implement OPC UA Historical Access. Task 018 Implement second storage destination. Task 019 Implement full reconnect/backfill acceptance test. Task 020 Production hardening. ``` Each task shall result in a working repository state rather than leaving the main branch intentionally uncompilable or untestable. \\--- \\# 80. Primary instruction to Codex When implementing this system, prioritize correctness, traceability and recoverability over speed of implementation. In particular: > Never compensate for missing AquaCom documentation by inventing plausible protocol behavior. Build the architecture around the unknown portions first. Add real AquaCom behavior only as verified protocol knowledge becomes available. Every protocol feature shall be backed by evidence and automated tests. Every architectural component shall be replaceable behind its documented interface. Every historical sample shall retain its original process timestamp. And no recovered historical value shall ever masquerade as a new live process value.