🎉 Initial commit

2026-10-07 10:38:21 +02:00
commit 1c64aeb071
38 changed files with 3391 additions and 0 deletions
@@ -0,0 +1,211 @@
+# OPC DA Diagnozer — Implementation Plan
+
+Companion to [`docs/SPECIFICATION.md`](SPECIFICATION.md). Targets **v1.0** as the first
+production-deployable release.
+
+---
+
+## 1. Milestones
+
+| Milestone | Content | Exit condition |
+|-----------|---------|----------------|
+| **v0.1 — Foreground proof** | CLI reads configured tags once per second in a console, prints events | Successfully reads from a real (or simulation) OPC DA server |
+| **v0.2 — Detection engine** | All detectors (D1–D15), event model, JSONL logging | Unit tests green; chaos scenarios produce correct events |
+| **v0.5 — Windows service** | pywin32 service, watchdog/hang recovery, Event Log, CLI install/run | Runs unattended through reboot + server-kill test |
+| **v1.0 — Production** | Packaging, runbook, 24 h soak, optional webhook | Acceptance criteria AC1–AC10 (spec §13) met |
+
+## 2. Project layout
+
+```
+opcda-diagnozer/
+├── README.md
+├── pyproject.toml
+├── config.example.yaml
+├── docs/
+│   ├── SPECIFICATION.md
+│   └── IMPLEMENTATION_PLAN.md
+├── src/opcda_diagnozer/
+│   ├── __init__.py
+│   ├── __main__.py            # python -m opcda_diagnozer
+│   ├── main.py                # CLI: run / install / uninstall / check-config / probe / list-devices / list-tags
+│   ├── config.py              # pydantic-free typed config (dataclasses) + YAML load + validation
+│   ├── events.py              # DiagEvent, Severity, EventKind
+│   ├── quality.py             # OPC DA quality decode tables
+│   ├── opc/
+│   │   ├── base.py            # OpcClient + Browser protocols, ReadResult, OpcError(hresult)
+│   │   ├── discovery.py       # list-servers: Automation GetOPCServers / registry scan
+│   │   ├── browse.py          # list-devices/list-tags: address-space walk (branches/leaves)
+│   │   ├── automation.py      # OPC DA Automation Wrapper adapter (OPCAutomation.dll)
+│   │   └── fake.py            # in-process fake client + fake address space (tests, demo)
+│   ├── monitor/
+│   │   ├── state.py           # connection state machine + backoff
+│   │   ├── poller.py          # COM thread: connect → poll loop → feed detectors
+│   │   └── detectors.py       # D1–D15 rule implementations
+│   ├── report/
+│   │   ├── base.py            # Reporter protocol
+│   │   ├── jsonl.py           # rotating JSONL file
+│   │   ├── eventlog.py        # Windows Event Log (WARN+)
+│   │   └── http.py            # optional webhook
+│   └── service.py             # pywin32 ServiceFramework wrapper
+└── tests/
+    ├── test_quality.py
+    ├── test_detectors.py
+    ├── test_state.py
+    ├── test_config.py
+    └── test_poller_fake.py    # end-to-end against opc.fake, no COM needed
+```
+
+## 3. Phase 0 — Feasibility & environment (≈0.5–1 day)
+
+**Goal:** validate the COM access strategy against reality before building anything.
+
+Tasks:
+1. Identify target OPC DA server: ProgID, host, DA version, bitness (32/64-bit EXE?).
+2. Choose wrapper backend:
+   - **A. OPC DA Automation Wrapper** (`OPCAutomation.dll`) — ubiquitous in industry,
+     ships with many SCADA installs; likely default.
+   - **B. OpenOPC / Graybox wrapper** (`GBDaCtl.dll`) — fallback.
+   - Note: both are typically **32-bit in-proc DLLs** → client Python must be 32-bit
+     (see spec §11). Decide interpreter bitness here.
+3. Stand up a **test server** (Matrikon OPC Simulation Server, KepServerEX demo, or the
+     actual control-system server in a maintenance window) and successfully read one tag
+     from Python (`win32com.client.Dispatch`).
+4. Verify DCOM configuration for the service account (`dcomcnfg`: Launch/Activation →
+   Remote Activation; Access → Local+Remote). Record required firewall ports
+   (TCP 135 + RPC dynamic range).
+5. Record baseline behavior: normal read RTT, quality values, what error the wrapper
+   surfaces when the server is stopped (feeds D1 implementation).
+
+**Exit:** a throwaway script reads ≥ 3 tags from the remote server with measured RTT.
+
+## 4. Phase 1 — Core engine (≈2–3 days)
+
+1. `config.py` — YAML config (see `config.example.yaml` sketch in README), dataclass
+   model, strict validation, `check-config` messages that explain exactly what is wrong.
+2. `quality.py` — decode tables (spec §7), `decode_quality(raw: int) -> QualityInfo`.
+3. `opc/base.py` — client abstraction:
+   ```python
+   class OpcClient(Protocol):
+       def connect(self) -> None: ...
+       def read(self, tags: Sequence[str]) -> list[ReadResult]: ...
+       def close(self) -> None: ...
+   ```
+   `ReadResult`: `tag, value, quality_raw, device_time, error` — plus per-call RTT
+   measured by the poller. COM errors are normalized to `OpcError` carrying the raw
+   `HRESULT` + decoded name.
+4. `opc/automation.py` — Automation Wrapper adapter (single group, static items,
+   `SyncRead`); every COM exception mapped to `OpcError` with `win32com` scode.
+5. `monitor/state.py` — state machine (spec §10.2), backoff, reconnect accounting.
+6. `monitor/detectors.py` — D1–D15 as small pure functions/classes over
+   `PollOutcome` (list[ReadResult] + rtt_ms + error), using an injected clock for
+   testability. Rolling window (deque, size `window_polls`) for p95/streak logic.
+7. `monitor/poller.py` — the single COM thread; watchdog logic for hang detection
+   (spec §10.4): a second thread timestamps poll starts/ends; on `hard_hang_s` the COM
+   thread is abandoned and rebuilt.
+8. `events.py` — event dataclass + JSON serialization.
+9. `opc/fake.py` — scriptable fake client (sequences of outcomes: good reads, errors,
+   hangs, quality flips) → powers the entire unit-test suite without COM/hardware.
+
+**Exit:** `opcda-diagnozer run` (foreground console) produces correct events against the
+fake client; unit tests for all detectors.
+
+## 5. Phase 2 — Reporters (≈1 day)
+
+1. `report/jsonl.py` — rotation (size + backups), atomic-ish append, header line with
+   config digest for provenance.
+2. `report/eventlog.py` — pywin32 `win32evtlog` usage; map severities; source
+   registration via `install` step.
+3. `report/http.py` — optional webhook with timeout + simple retry/drop on failure
+   (never blocks the poll loop; queue with bound).
+4. Heartbeat assembly (D15): uptime, reconnects, RTT percentiles, tags-bad counts.
+
+**Exit:** events visible in file + Event Log; heartbeat cadence verified.
+
+## 6. Phase 3 — Windows service (≈1 day)
+
+1. `service.py` — `win32serviceutil.ServiceFramework`; service host thread spawns the
+   COM worker; clean `stop` handling (`SetServiceStatus STOP_PENDING`, join with
+   timeout, abandon hung COM thread on shutdown).
+2. CLI verbs: `install /uninstall /start /stop` (wraps `sc.exe` semantics),
+   `run` (foreground), `check-config`, `probe` (connect once, print one read),
+   `list-items` (browse validation tool).
+3. SCM recovery config (`sc failure ... restart`), auto-start, service account
+   documentation.
+4. Config/log default dir: `%ProgramData%\OPCDA-Diagnozer\`.
+
+**Exit:** survives reboot; survives OPC-server kill (produces D1/D2 events, reconnects);
+`probe` and `check-config` give usable diagnostics to an ops engineer.
+
+## 7. Phase 4 — Integration & chaos testing (≈1–2 days)
+
+Scenario matrix against a simulation server (and, when available, the real server):
+
+| Scenario | How to inject | Expected events |
+|----------|---------------|-----------------|
+| Server process killed | `taskkill` | D1 (HRESULT), D2 with outage duration |
+| Server service stopped/restarted | `net stop/start` | D1, D2 |
+| Firewall drop (silent) | block outbound RPC ports / pull NIC | D7 hang, then D1 |
+| Latency injection | Clumsy / `netsh` delay | D4 spikes, D5 streak, D6 trend |
+| Device comm failure | simulation server "communication disturbance" items | D8 with `BAD_COMM_FAILURE`, D10 persistence |
+| Quality oscillation | toggling sim items | D11 flapping |
+| Stale/frozen tag | simulation server static item marked `expect_change` | D12 / D13 |
+| Mass bad quality | all sim items to bad | D14 |
+| Config errors | bad ProgID, unknown tag | fast fail + actionable message; `OPC_E_INVALIDITEMID` surfaced at startup |
+| Soak | 24 h healthy run | no leaks, heartbeats every 5 min, zero false positives |
+
+**Exit:** each row verified manually against the event log; AC2/AC3/AC4/AC5 confirmed.
+
+## 8. Phase 5 — Packaging & docs (≈0.5–1 day)
+
+- `pyproject.toml` wheel installable with pip into the chosen (32-bit) Python;
+  document the on-host install procedure (PyInstaller packaging for pywin32 services is
+  fragile — deferred).
+- `README.md` finalized (install, config, DCOM checklist, HRESULT table).
+- Ops runbook: how to read events, what each kind means (link to spec §12 matrix),
+  how to reconfigure thresholds, where evidence lives.
+- Version 1.0.0 tag.
+
+## 9. Testing strategy
+
+- **Unit** (no COM): detectors against scripted `PollOutcome` sequences with a fake
+  clock; quality decode; config validation; backoff/state transitions; JSONL rotation.
+- **Integration:** `test_poller_fake.py` runs the full poller+reporters against
+  `opc.fake` in-process — the whole event pipeline without Windows COM.
+- **Manual/chaos:** Phase 4 matrix (necessarily on Windows against a real/simulated
+  OPC DA server — cannot be automated in CI without the DCOM environment).
+- CI: unit + fake-integration tests run on any OS (COM adapters are import-guarded);
+  chaos tests are a documented manual procedure.
+
+## 10. Risk register
+
+| Risk | Impact | Mitigation |
+|------|--------|------------|
+| Wrapper DLL bitness mismatch (32-bit only) | Cannot load wrapper | Phase 0 decision; use 32-bit Python; keep client abstraction so backend can be swapped |
+| Automation Wrapper not present on diag host | No COM access | Phase 0 check; ship/distribute wrapper install in runbook (vendor redistribution rules apply) |
+| DCOM hardening (KB5004442) blocks activation | Connect fails with `E_ACCESSDENIED`/activation errors | Runbook covers `dcomcnfg` steps; events surface raw HRESULT |
+| COM calls hang indefinitely | Service appears alive but blind | Explicit watchdog + thread recycling (spec §10.4) |
+| COM apartment violations (cross-thread calls) | Random `RPC_E_WRONG_THREAD` crashes | Single dedicated COM thread; all COM handles stay on it |
+| Our client perturbs the system | Skewed diagnosis | One group, static small tag list, 1 s default poll, read-only |
+| pywin32 service + COM quirks (GIL, STA init) | Service instability | Phase 3 spike; foreground `run` mode always available as fallback |
+| Tag list too large / wrong item IDs | Noise, misleading RTT | `probe` + `check-config` validate up-front; per-poll error accounting |
+| SCADA/host clock skew | Misleading staleness events | Local receive time always recorded; `max_age_s` configurable |
+
+## 11. Effort estimate
+
+| Phase | Estimate |
+|-------|----------|
+| 0 Feasibility | 0.5–1 d |
+| 1 Core engine | 2–3 d |
+| 2 Reporters | 1 d |
+| 3 Service | 1 d |
+| 4 Chaos testing | 1–2 d |
+| 5 Packaging/docs | 0.5–1 d |
+| **Total v1.0** | **≈ 6–9 days** |
+
+## 12. Definition of done (v1.0)
+
+- All Phase 4 scenarios pass; AC1–AC10 (spec §13) verified.
+- 24 h soak clean.
+- README + runbook complete enough for an ops engineer to deploy unaided.
+- Code reviewed; no write path to OPC anywhere in the codebase.
+- Config example covers every configurable threshold.
@@ -0,0 +1,263 @@
+# OPC DA Diagnozer — Project Specification
+
+**Version:** 0.1 (draft)
+**Status:** Planning
+**Target platform:** Windows only (OPC DA is a COM/DCOM technology)
+
+---
+
+## 1. Problem statement
+
+We experience intermittent **disconnections** and/or **high communication latency** between a
+SCADA system and a control system that communicate over **OPC DA** (Data Access, the
+COM/DCOM-based predecessor of OPC UA). Today we have no objective evidence to answer the
+key triage question:
+
+> *Is the problem in the SCADA host, the network/DCOM layer, the OPC DA server, or the
+> link between the OPC server and the control devices?*
+
+OPC DA failure modes are notoriously hard to diagnose because failures surface as generic
+`HRESULT` errors, hung calls with **no client-side timeout**, or silently degraded item
+quality — long before the SCADA operator sees anything.
+
+## 2. Approach
+
+Deploy an **independent, read-only OPC DA reference client** that continuously exercises
+the same path the SCADA uses:
+
+- Connect to the OPC DA server (local or remote via DCOM) from a Windows machine.
+- Poll a small, pre-defined list of tags at a fixed interval.
+- Measure **read round-trip time (RTT)**, track **item quality** (GOOD / UNCERTAIN / BAD
+  and subcodes), **device timestamps**, and **connection state**.
+- Emit structured, timestamped **diagnostic events** whenever anything deviates from
+  normal — disconnects, slow reads, hung calls, quality degradation, frozen/stale values.
+
+Because the client is independent of the SCADA, its observations provide a **reference
+baseline** against which the SCADA's complaints can be triangulated (see §12).
+
+## 3. Deployment architecture
+
+```mermaid
+flowchart LR
+    subgraph SCADAzone["SCADA zone (or dedicated diag host)"]
+        DIAG["OPC DA Diagnozer<br/>(Windows service)"]
+    end
+    subgraph CTL["Control system / OPC DA server host"]
+        OPC["OPC DA server<br/>(COM/DCOM out-of-proc)"]
+        DEV["Control devices / PLC link"]
+    end
+    DIAG -- "DCOM reads (RPC 135 + dynamic ports)" --> OPC
+    OPC -.-> DEV
+    DIAG -- "Events (JSONL / Event Log / HTTP)" --> OPS["Engineers & monitoring"]
+```
+
+Placement note: run the tool **as close to the SCADA machine as possible** (same host or
+same network segment) so its measured latencies are comparable to what the SCADA
+experiences. Optionally run a second instance next to the OPC server host for differential
+diagnosis.
+
+## 4. Goals
+
+1. Run unattended as a **Windows service** (auto-start, SCM crash recovery).
+2. Connect to a configured OPC DA server (local or remote DCOM) and read a configured tag list.
+3. Detect and report, with precise timestamps:
+   - disconnections and reconnects (including DCOM error codes),
+   - hung / timed-out RPC calls,
+   - read latencies exceeding thresholds (single spikes, streaks, and trends),
+   - tags whose quality is BAD or UNCERTAIN (with decoded subcode),
+   - persistent-bad and flapping (oscillating) quality per tag,
+   - stale device timestamps / frozen values while quality reports GOOD,
+   - mass quality degradation (many tags BAD at once → control-link problem).
+4. Produce durable, machine-readable evidence (structured logs + Windows Event Log +
+   optional HTTP webhook) suitable for incident review and correlation.
+5. Be **strictly read-only** and lightweight — must not perturb the system it measures.
+
+## 5. Non-goals (v1)
+
+- No writes to the OPC server (no `Write` code path at all).
+- No OPC UA, OPC HDA, or Alarms & Events support.
+- No data historian / storage of every sample (only events + periodic summaries).
+- No UI/dashboard in v1 (JSONL logs + webhook are enough for Loki/Grafana ingestion).
+- No automatic remediation (diagnose only).
+
+## 6. Detection rules
+
+| ID | Condition | Default rule | Severity |
+|----|-----------|--------------|----------|
+| D1 | Connection lost | Any read/connect COM failure | CRITICAL (event incl. decoded HRESULT) |
+| D2 | Reconnect succeeded | After D1, reconnect works | INFO (event with outage duration) |
+| D3 | Reconnect flapping | > 3 disconnects in 10 min | CRITICAL |
+| D4 | Slow poll (spike) | Single read RTT > `warn_ms` (default 1000) | WARN |
+| D5 | Sustained slow polls | RTT > `warn_ms` for `slow_poll_streak` (5) consecutive polls | CRITICAL |
+| D6 | Latency trend | p95 RTT over last `window_polls` (60) > `warn_ms` | WARN |
+| D7 | Poll hang | Read call does not return within `timeout_ms` (5000) — see §10.4 | CRITICAL (immediate), see recovery behavior |
+| D8 | Tag quality → BAD | Any transition into BAD, decoded subcode | WARN (per tag) |
+| D9 | Tag quality → UNCERTAIN | Any transition into UNCERTAIN | INFO |
+| D10 | Persistent bad tag | Single tag BAD longer than `persistent_bad_s` (30) | WARN (escalated from D8) |
+| D11 | Quality flapping | > 5 quality transitions for one tag within 10 min | WARN |
+| D12 | Stale timestamp | Device timestamp older than `max_age_s` (120) while quality GOOD | WARN |
+| D13 | Frozen value | `expect_change: true` tag with unchanged value+timestamp for `max_age_s` | WARN |
+| D14 | Mass quality degradation | ≥ 50% of monitored tags BAD simultaneously while connection OK | CRITICAL (`CONTROL_LINK_DEGRADED` — points at OPC-server↔device link, not DCOM) |
+| D15 | Heartbeat / summary | Periodic stats snapshot every `heartbeat_s` (300) | INFO |
+
+All thresholds are configurable; severities are INFO / WARN / CRITICAL.
+
+## 7. OPC DA quality codes (reference)
+
+OPC DA quality is a 16-bit field. Bits 7–6 of the low byte give the class:
+`0xC0` GOOD, `0x40` UNCERTAIN, `0x00` BAD. Bits 5–0 are the substatus. Values the tool
+decodes and names:
+
+| Code | Meaning |
+|------|---------|
+| `0xC0` | GOOD |
+| `0xD8` | GOOD / local override |
+| `0x44` | UNCERTAIN / last usable value |
+| `0x40` | UNCERTAIN (unspecified) |
+| `0x00` | BAD (unspecified) |
+| `0x04` | BAD / configuration error |
+| `0x08` | BAD / not connected |
+| `0x0C` | BAD / device failure |
+| `0x10` | BAD / sensor failure |
+| `0x14` | BAD / last known value |
+| `0x18` | BAD / communication failure |
+| `0x1C` | BAD / out of service |
+
+Remaining UNCERTAIN subcodes (DA 2.05a) are decoded by the same table in `quality.py`.
+Every quality event carries the raw hex code **and** the decoded name.
+
+## 8. Event model
+
+Every event is a structured record:
+
+```json
+{
+  "time_utc": "2026-10-07T12:34:56.789Z",
+  "kind": "QUALITY_BAD",
+  "severity": "WARN",
+  "detail": {
+    "tag": "Channel1.Device1.Counter",
+    "quality_raw": "0x18",
+    "quality": "BAD_COMM_FAILURE",
+    "previous_quality": "GOOD",
+    "value": 1234.0,
+    "device_time_utc": "2026-10-07T12:34:55.100Z",
+    "rtt_ms": 42.1
+  },
+  "context": {
+    "uptime_s": 3600.5,
+    "reconnects_total": 2,
+    "rtt_p95_ms": 55.2,
+    "tags_bad": 3,
+    "tags_total": 25
+  }
+}
+```
+
+`time_utc` is always the **local client receive time** (wall clock), so events remain
+comparable even if the remote host's clock is wrong. Device timestamps are reported
+alongside and clearly labeled.
+
+## 9. Outputs (reporters)
+
+| Reporter | Purpose | Notes |
+|----------|---------|-------|
+| JSONL rotating file | Primary evidence trail | One JSON event per line; rotation by size; default dir `%ProgramData%\OPCDA-Diagnozer\logs` |
+| Windows Event Log | Ops visibility | WARN+ only; source `OPCDA-Diagnozer` |
+| Console | Foreground/debug runs | Human-readable |
+| HTTP webhook (optional) | Alert integration | POST one JSON event per request, optional bearer token |
+
+Reporters are pluggable (`Reporter.emit(event)`); a heartbeat event every 5 minutes
+proves liveness and enables baseline graphs even when everything is healthy.
+
+## 10. Operational behavior
+
+### 10.1 Run as a Windows service
+- Registered via pywin32 (`opcda-diagnozer install`), auto-start, runs under a
+  **dedicated service account** (required for DCOM authentication, see §11).
+- SCM recovery: restart on failure (first/second failure), plus internal watchdog.
+
+### 10.2 Connection state machine
+```mermaid
+stateDiagram-v2
+    [*] --> Connecting
+    Connecting --> Connected : connect OK
+    Connecting --> ReconnectWait : COM error
+    Connected --> ReconnectWait : read error / hang
+    ReconnectWait --> Connecting : backoff elapsed
+```
+Reconnect backoff: exponential 1 s → 2 s → … capped at 60 s. Every disconnect/reconnect
+pair is an event; outage duration is computed and reported.
+
+### 10.3 COM apartment handling
+All OPC/COM work runs on one dedicated STA thread inside the service process
+(`pythoncom.CoInitialize()` on that thread only). Detectors run on the same thread (pure
+Python, cheap); event reporting happens off-thread via a queue so slow I/O never blocks
+polling.
+
+### 10.4 The DCOM hang problem (important)
+DCOM/RPC calls have **no client-side timeout**. If the network path breaks silently
+(e.g., firewall drops), a read call can block for tens of seconds to minutes without
+erroring. The tool handles this explicitly:
+
+1. A watchdog flags a **poll hang** (D7) as soon as a call exceeds `timeout_ms`.
+2. If the hang exceeds `hard_hang_s` (default 120 s), the service abandons the stuck
+   thread and creates a fresh connection on a new thread (stale `RPC` objects are
+   released by COM).
+3. Hang events are distinguished from clean disconnects — this is often the smoking gun
+   for silent network drops.
+
+### 10.5 Safety constraints
+- Read-only by construction: no write API is called; the code has no write path.
+- Single OPC group, static item list, modest poll rate (default 1 s) to keep added
+  server load negligible.
+- Configuration errors fail fast at startup with actionable messages
+  (`opcda-diagnozer check-config` validates without connecting… and `--probe` connects
+  once and reports).
+
+## 11. Environment & constraints
+
+| Constraint | Consequence |
+|------------|-------------|
+| OPC DA is Windows/COM only | Service runs on Windows only (Server 2012 R2 … 2022, Win 10/11) |
+| OPC DA wrappers are often **32-bit** DLLs (OPC DA Automation Wrapper `OPCAutomation.dll`, Graybox `GBDaCtl.dll`) | Python interpreter bitness must match the wrapper DLL — likely **32-bit Python** for v1; decision in Phase 0 |
+| DCOM security | The service account needs Launch+Activation and Access permissions on the OPC server (via `dcomcnfg`); account must authenticate across the network (not `LocalSystem`) |
+| Post-2022 Windows DCOM hardening (KB5004442 family) | DCOM activation now requires explicit authentication; legacy OPC setups may fail activation — errors must be reported clearly |
+| RPC over TCP | Firewall: TCP 135 + RPC dynamic port range on the server host; misconfig shows up as `RPC_S_SERVER_UNAVAILABLE` |
+| Clock skew | Device timestamps may be misleading; every event also carries local receive time; recommend NTP on both hosts |
+
+## 12. Diagnostic interpretation matrix (why this tool exists)
+
+| Observation from the Diagnozer | Most likely problem location |
+|--------------------------------|------------------------------|
+| Clean reads, low RTT, all GOOD — while SCADA still complains | SCADA host itself: per-machine DCOM permissions, CPU/memory saturation, its own group/subscription load |
+| Diagnozer also sees disconnects / RPC errors | Shared path: network, firewall, DCOM layer, or OPC server host |
+| Connection stable, but `BAD_COMM_FAILURE` on many device tags | Link between the OPC server and the control devices (downstream of OPC) — **not** a DCOM/network problem |
+| Slow RTT spikes at consistent times of day | Server load pattern; correlate with heartbeat summaries |
+| Hangs without errors returned (D7) | Silent network drop (firewall/MTU/NIC) — classic DCOM pitfall |
+| One instance (SCADA zone) slow, other (server zone) fast | Network segment between the zones |
+
+## 13. Acceptance criteria (v1.0)
+
+- **AC1** Runs as a service, survives reboot, auto-starts, recovers from a killed process.
+- **AC2** Detects OPC server process stop within one poll interval (or one hang window for
+  silent drops).
+- **AC3** Detects injected latency above thresholds within 1–2 polls; event contains
+  measured RTT.
+- **AC4** Quality transitions produce events with raw code and decoded name.
+- **AC5** Staleness/freeze detection within tolerance of `max_age_s`.
+- **AC6** No OPC writes are ever issued (code review + no write path exists).
+- **AC7** Events land in JSONL and Event Log; heartbeats appear even when all is healthy.
+- **AC8** Invalid config fails fast with actionable messages.
+- **AC9** Footprint ≤ ~50 MB RAM, ~0% CPU while idle-polling on a healthy link.
+- **AC10** 24 h soak against a simulation server: no leaks, correct reconnect accounting.
+
+## 14. Future extensions (post-v1)
+
+- Async subscription monitoring (`IOPCAsyncIO2` DataChange) to compare push vs. poll
+  latency — validates the SCADA's actual callback path.
+- Per-item latency bisection (read items individually to isolate a slow tag).
+- SQLite time-series of samples for post-incident graphing.
+- Prometheus exporter / Grafana dashboard.
+- Email / Teams / webhook alerting with deduplication.
+- Multi-server monitoring in one service.