Compare commits

...
1 Commits
Author SHA1 Message Date
Jonas Bengtsson a1af8d68c9 🚧 WIP 2026-10-08 06:30:54 +02:00
12 changed files with 966 additions and 21 deletions
+90
View File
@@ -0,0 +1,90 @@
# Changelog
All notable changes to the OPC DA Diagnozer are documented here. The format
follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions
bump per [SemVer](https://semver.org/). Dates are YYYY-MM-DD.
## [Unreleased]
### Added — server discovery & browsing
- `list-servers` verb: enumerates OPC DA servers locally or on a DCOM host via
the Automation Wrapper (`GetOPCServers`), with a no-COM registry-scan fallback
(`--source registry|automation|auto`), JSON output, and DA-version metadata.
- Browsing of the connected server's address space, unified in the
`list-items [all|branches|tags]` verb: `all` prints the full structure tree
(branches with `/`, tags with `[ItemID: ...]`), `branches` the structure-only
flat list, `tags` the flat ItemID list for config `tags:`. `--branch`
(dotted path or repeated segments), `--depth N` (1 default, 0 = whole
subtree), `--json`. Replaces the earlier split `list-devices`/`list-tags`.
- `browse.max_depth` config key (default 32) as the address-space walk guard,
validated by `check-config`; cycle-safe tree walk in `opc/browse.py`.
- `SpaceNode` tree data model (branch/leaf) replacing the earlier
device-centric `DeviceInfo`.
### Added — reading & CLI
- `read` verb: one-shot read of explicit tags (or the config's tags) with a
table or `--json`, `--source device|cache`, and a non-zero exit when any tag
doesn't read GOOD — scriptable as a health check.
- `probe` verb: connect once and read every configured tag once.
- `--server` / `--host` overrides on every connecting verb; with `--server` no
config file is needed at all.
- Frozen-exe default config path: `config.yaml` next to the exe (a Windows
service starts with `CWD = System32`), not the working directory.
### Added — monitoring design (D16) & config
- `devices:` config section with `DeviceEntry(name, tag, ok_values)`: D16
device-link indicators. The normal (connected) reading is configurable per
indicator (`ok_values`, default `[true]`) — inverted fault flags
`[false, 0]` and string states `["RUN"]` are first-class. Strict validation:
duplicate indicators and indicator/tag-list overlap are rejected;
`check-config` prints the indicators.
- Detection rules D16 (`DEVICE_LINK_DOWN`/`DEVICE_LINK_UP`) and the plain-text
anomaly log reporter (`report/textlog.py` task) specified in the docs
(SPECIFICATION §6.1, §9.1; IMPLEMENTATION_PLAN Phases 1–2) — implementation
pending, see TODO.md.
- Clear-text status rule (spec §7): every reporter writes `NAME (0xNN)` —
e.g. `BAD / communication failure (0x18)`.
### Added — packaging
- `scripts/build_exe.ps1`: PyInstaller build of the 32-bit console exe with
hidden imports for the deferred COM modules (incl. `win32timezone` — see
Fixed), embedded version metadata, a PE-bitness check, and smoke tests
including a **frozen-read canary** against a live server.
- `--version` and this CHANGELOG.md / TODO.md.
### Fixed
- COM lifecycle: all COM references (browsers, groups, items) are released
**inside** the apartment before `CoUninitialize`; retained exception
tracebacks are cleared in `as_com_error` — both caused
"Win32 exception occurred releasing IUnknown" noise and hard crashes at exit.
- Frozen exe: every tag read failed with
`SystemError: ... InvokeTypes ... exception set` because pywin32 lazily
imports `win32timezone` during VARIANT-DATE conversion and PyInstaller never
bundles it — now a hidden import, with the build canary guarding it.
- Reads: per-tag failures now carry the underlying exception's own type and
text (`_per_tag_error`) instead of a bare "failed"; a degenerate device
timestamp (never-updated alias, DST math) can no longer fail a read — the
timestamp is metadata and is dropped to `None`.
- Matrikon browse quirk: servers may return an empty ItemID for special leaves
(alias roots); the adapter falls back to the bare browse name.
- HRESULT labels: `OPC_E_UNKNOWNITEMID` (0xC0040007) and
`OPC_E_INVALIDITEMID` (0xC0040008) were swapped; now corrected everywhere.
### Verified
- All CLI verbs live-tested against Matrikon OPC Simulation Server
(`Matrikon.OPC.Simulation.1`): discovery, browse (all kinds/depths), reads
(incl. alias tags after server-side alias repair), config-driven
`max_depth`, and the frozen exe (onedir + onefile) including the read canary.
- Test suite: 108 tests, all passing (pure logic + fake-client driven, runs on
any OS; COM paths are live-smoke-tested on dev boxes only).
## [0.1.0] - 2026-10-07
### Added
- Initial project skeleton: typed YAML config with strict actionable
validation (`check-config`), OPC DA quality decode tables (spec §7), event
model skeletons (`DiagEvent`, D1–D15 `EventKind`s), client/protocol
abstractions, and the fake in-process client powering cross-OS tests.
- Phase 0 decisions recorded: 32-bit interpreter mandatory (Automation Wrapper
is a 32-bit in-proc DLL), ruamel.yaml over PyYAML (no 32-bit cp314 wheels),
COM imports import-guarded for cross-OS test runs.
+17 -3
View File
@@ -10,9 +10,12 @@ disconnects, hung RPC calls, tags marked BAD/UNCERTAIN, stale timestamps, frozen
and flapping quality — as structured events (JSONL logs, Windows Event Log, optional and flapping quality — as structured events (JSONL logs, Windows Event Log, optional
HTTP webhook). HTTP webhook).
> **Status:** planning. Read the > **Status:** in active development — discovery, browsing (`list-items`), one-shot
> [specification](docs/SPECIFICATION.md) and the > reads (`read`/`probe`) and the exe build are **live and field-verified** against a
> [implementation plan](docs/IMPLEMENTATION_PLAN.md) first. > real server; the continuous monitoring loop, reporters and Windows service host are
> next. See the [TODO](TODO.md) for current status and the
> [implementation plan](docs/IMPLEMENTATION_PLAN.md) for the phases; read the
> [specification](docs/SPECIFICATION.md) for detection rules and behavior.
## Why ## Why
@@ -126,6 +129,15 @@ tags:
- { name: "Channel1.Device1.Counter", expect_change: true } - { name: "Channel1.Device1.Counter", expect_change: true }
- { name: "Channel1.Device1.Level" } - { name: "Channel1.Device1.Level" }
# D16 device-link indicators: tags saying whether the OPC server can reach
# the device. DOWN (DEVICE_LINK_DOWN, CRITICAL) when the tag reads non-GOOD,
# errors, or its GOOD value is NOT in ok_values. ok_values defaults to [true]
# (boolean connected flags); inverted fault flags -> [false, 0]; string
# states -> ["RUN"]. Do NOT also list these under tags:.
devices:
- { name: "Conveyor 3", tag: "Channel1.Device3.Connected" }
- { name: "Pump A", tag: "Channel1.PumpA.Fault", ok_values: [false, 0] }
latency: latency:
warn_ms: 1000 warn_ms: 1000
critical_ms: 5000 critical_ms: 5000
@@ -257,6 +269,8 @@ Deployment notes:
- [Specification](docs/SPECIFICATION.md) — detection rules, event model, operational behavior, acceptance criteria - [Specification](docs/SPECIFICATION.md) — detection rules, event model, operational behavior, acceptance criteria
- [Implementation plan](docs/IMPLEMENTATION_PLAN.md) — phases, milestones, testing, risks - [Implementation plan](docs/IMPLEMENTATION_PLAN.md) — phases, milestones, testing, risks
- [TODO](TODO.md) — implementation status: what's done, what's next
- [Changelog](CHANGELOG.md) — notable changes per release
## License ## License
+98
View File
@@ -0,0 +1,98 @@
# TODO — implementation status
Companion to `docs/IMPLEMENTATION_PLAN.md` (the authoritative plan) and
`CHANGELOG.md` (what shipped). One row per plan task; updated as work lands.
Legend: **[done]** complete & verified · **[partial]** some of it landed ·
**[todo]** not started · **[skip]** dropped by decision.
## Phase 0 — Feasibility & environment **[done]**
- [done] Wrapper backend decision: OPC DA Automation Wrapper (`OPC.Automation`)
→ 32-bit interpreter mandatory (spec §11).
- [done] Test server: Matrikon OPC Simulation Server, local ProgID
`Matrikon.OPC.Simulation.1`; baseline read RTT + wrapper error shapes
recorded (repo memory).
- [done] Dependencies chosen for a 32-bit, no-compiler install:
ruamel.yaml (PyYAML has no cp314 win32 wheels), pywin32.
## Phase 1 — Core engine
- [done] 1. `config.py` — typed YAML config + strict validation; **D16
`devices:` indicators with configurable `ok_values`**; `check-config`
prints them.
- [done] 2. `quality.py` — decode tables (spec §7).
- [todo] `status_text(raw) -> "NAME (0xNN)"` clear-text helper (spec §7
closing rule; used by every reporter).
- [done] 3. `opc/base.py` — client/browser protocols, `ReadResult`,
`OpcError` with decoded HRESULT.
- [done] 4. `opc/automation.py` — connect / browse / read (per-item
`Item.Read`; `SyncRead` is unmarshallable from pywin32 — verified), per-tag
errors, safe COM teardown.
- [skip] Single persistent group + static items (poller will create its own
per client; the throwaway-group read pattern is verified).
- [todo] 5. `monitor/state.py` — connection state machine, exponential
backoff (1 s → 60 s cap), reconnect accounting (only the enum exists).
- [todo] 6. `monitor/detectors.py` — D1–D15 pure detectors + rolling window
(only the `PollOutcome` dataclass exists).
- [todo] **D16 device-link detector** (`ok_values` semantics, edge-level,
`DEVICE_LINK_DOWN`/`DEVICE_LINK_UP`).
- [todo] 7. `monitor/poller.py` — the COM thread poll loop + hang watchdog
(abandon & rebuild past `hard_hang_s`).
- [todo] Startup tag validation: probe every configured tag (incl. device
indicators) before entering the loop; fail fast on invalid ItemIDs.
- [open] Poll source default (CACHE vs DEVICE) — needs a spec decision:
Matrikon shows BAD qualities on some tags only via CACHE.
- [todo] 8. `run` verb — foreground monitoring loop, event pipeline to
reporters, heartbeat (D15).
- [done] 9. `opc/fake.py` — scriptable fake client (reads already scriptable
via `read_results`; hang/error sequences land with the poller).
## Phase 2 — Reporters
- [todo] 1. `report/jsonl.py` — rotating JSONL (size + backups), config-digest
header, atomic-ish append.
- [todo] 2. `report/textlog.py` — **plain-text anomaly log** (spec §9.1):
`anomalies-YYYYMMDD.log`, one line per detected anomaly, clear-text status
`NAME (0xNN)`, recoveries logged so it reads as an incident narrative.
- [todo] 3. `report/eventlog.py` — Windows Event Log (WARN+), source
registration via `install`.
- [todo] 4. `report/http.py` — optional webhook, bounded queue, never blocks
polling.
- [todo] 5. Heartbeat assembly (D15) + reporter fan-out dispatcher.
## Phase 3 — Windows service
- [todo] 1. `service.py` — `ServiceFramework` host; clean stop
(`STOP_PENDING`, join-with-timeout, abandon hung COM thread).
- [todo] 2. CLI verbs `install`/`uninstall`/`start`/`stop` (sc.exe semantics).
- [todo] 3. SCM recovery (`sc failure ... restart`), auto-start, service
account docs.
- [todo] 4. `%ProgramData%\OPCDA-Diagnozer\` config/log dir conventions
(frozen exe already defaults there).
- [open] Frozen-exe SCM hosting: console exe must expose `ServiceMain` — one
careful build pass (pywin32 `ServiceControlDispatcher`).
## Phase 4 — Integration & chaos testing
- [todo] Scenario matrix vs the sim server: server kill / service stop /
firewall silent drop / latency injection / device comm failure / quality
oscillation / stale+frozen / mass bad / config errors / **D16: flip a device
indicator (e.g. a `P500x.Connected` alias)**.
- [todo] 24 h soak (AC10): no leaks, reconnect accounting correct, heartbeats
every 5 min, ≤ ~50 MB RAM, ~0 % CPU idle.
## Phase 5 — Packaging & docs
- [done] PyInstaller exe build (`scripts/build_exe.ps1`, onedir + onefile,
read canary) — live-verified.
- [done] README (install, config, DCOM checklist, HRESULT table).
- [todo] Ops runbook: reading events, kind meanings (spec §12 matrix),
threshold tuning, evidence locations.
- [todo] 1.0.0 tag once AC1–AC10 pass.
## Housekeeping
- [todo] Wire `status_text` into `read`/`probe` output once it exists
(currently shows `raw_hex` + decoded name separately).
- [todo] Record the CACHE-vs-DEVICE poll-source decision in the spec once
made; add fake-client scripted reads for both sources to the poller tests.
+13
View File
@@ -16,6 +16,19 @@ tags:
- { name: "Channel1.Device1.Counter", expect_change: true } # D13 frozen-value detection applies - { name: "Channel1.Device1.Counter", expect_change: true } # D13 frozen-value detection applies
- { name: "Channel1.Device1.Level" } - { name: "Channel1.Device1.Level" }
# D16 device-link indicators: tags saying whether the OPC server can reach the
# device. DOWN (= DEVICE_LINK_DOWN, CRITICAL, clear-text status in the log) when
# the tag reads non-GOOD, errors, or its GOOD value is NOT in ok_values.
# ok_values defaults to [true] (boolean connected-flags); set it per system
# convention: inverted fault flags -> [false, 0], string states -> ["RUN"].
# Compared by string equality on the rendered value ("True"/"1"/"RUN").
# Do NOT also list these under tags: - they are polled automatically.
devices:
- { name: "Conveyor 3", tag: "Channel1.Device3.Connected" } # ok: true/1
- { name: "Conveyor 4", tag: "Channel1.Device4.Connected", ok_values: [true, 1] }
- { name: "Pump A", tag: "Channel1.PumpA.Fault", ok_values: [false, 0] } # inverted: 0 = ok
- { name: "Boiler 2", tag: "Channel1.Boiler2.State", ok_values: ["RUN"] }
latency: latency:
warn_ms: 1000 # D4 single-spike threshold warn_ms: 1000 # D4 single-spike threshold
critical_ms: 5000 # D5 sustained-slow severity critical_ms: 5000 # D5 sustained-slow severity
+31 -6
View File
@@ -27,7 +27,7 @@ opcda-diagnozer/
├── src/opcda_diagnozer/ ├── src/opcda_diagnozer/
│ ├── __init__.py │ ├── __init__.py
│ ├── __main__.py # python -m opcda_diagnozer │ ├── __main__.py # python -m opcda_diagnozer
│ ├── main.py # CLI: run / install / uninstall / check-config / probe / list-devices / list-tags │ ├── main.py # CLI: run / install / uninstall / check-config / probe / read / list-items
│ ├── config.py # pydantic-free typed config (dataclasses) + YAML load + validation │ ├── config.py # pydantic-free typed config (dataclasses) + YAML load + validation
│ ├── events.py # DiagEvent, Severity, EventKind │ ├── events.py # DiagEvent, Severity, EventKind
│ ├── quality.py # OPC DA quality decode tables │ ├── quality.py # OPC DA quality decode tables
@@ -44,6 +44,7 @@ opcda-diagnozer/
│ ├── report/ │ ├── report/
│ │ ├── base.py # Reporter protocol │ │ ├── base.py # Reporter protocol
│ │ ├── jsonl.py # rotating JSONL file │ │ ├── jsonl.py # rotating JSONL file
│ │ ├── textlog.py # plain-text anomaly log (human-readable, §9.1)
│ │ ├── eventlog.py # Windows Event Log (WARN+) │ │ ├── eventlog.py # Windows Event Log (WARN+)
│ │ └── http.py # optional webhook │ │ └── http.py # optional webhook
│ └── service.py # pywin32 ServiceFramework wrapper │ └── service.py # pywin32 ServiceFramework wrapper
@@ -82,7 +83,13 @@ Tasks:
1. `config.py` — YAML config (see `config.example.yaml` sketch in README), dataclass 1. `config.py` — YAML config (see `config.example.yaml` sketch in README), dataclass
model, strict validation, `check-config` messages that explain exactly what is wrong. model, strict validation, `check-config` messages that explain exactly what is wrong.
2. `quality.py` — decode tables (spec §7), `decode_quality(raw: int) -> QualityInfo`. Includes the **device-link indicators** (spec §6.1 / D16): `devices:` — each entry a
human-readable device name plus its connection-indicator tag ItemID. Indicators are
added to the polled tag set automatically; `check-config` warns when an indicator tag
is also listed in `tags:` (it should not be) and fails on an empty device name.
2. `quality.py` — decode tables (spec §7), `decode_quality(raw: int) -> QualityInfo`,
plus the **clear-text status helper**: `status_text(raw) -> "NAME (0xNN)"` used by
every reporter (spec §7 closing rule — e.g. `BAD / communication failure (0x18)`).
3. `opc/base.py` — client abstraction: 3. `opc/base.py` — client abstraction:
```python ```python
class OpcClient(Protocol): class OpcClient(Protocol):
@@ -99,6 +106,14 @@ Tasks:
6. `monitor/detectors.py` — D1–D15 as small pure functions/classes over 6. `monitor/detectors.py` — D1–D15 as small pure functions/classes over
`PollOutcome` (list[ReadResult] + rtt_ms + error), using an injected clock for `PollOutcome` (list[ReadResult] + rtt_ms + error), using an injected clock for
testability. Rolling window (deque, size `window_polls`) for p95/streak logic. testability. Rolling window (deque, size `window_polls`) for p95/streak logic.
**D16 device-link detector** (spec §6.1): per configured indicator — quality
not GOOD, a read error, or a GOOD read whose value is **not in the
configurable `ok_values` list** (default `[true]`; string-equality compare
of the rendered value, so inverted fault flags `[false, 0]` and string states
`["RUN"]` both work) ⇒ `DEVICE_LINK_DOWN` (CRITICAL) carrying the device name,
indicator tag, clear-text status and value; recovery (`DEVICE_LINK_UP`) when
it reads GOOD with an ok value again. Edge-level per device, so flapping
links don't spam (one event per transition).
7. `monitor/poller.py` — the single COM thread; watchdog logic for hang detection 7. `monitor/poller.py` — the single COM thread; watchdog logic for hang detection
(spec §10.4): a second thread timestamps poll starts/ends; on `hard_hang_s` the COM (spec §10.4): a second thread timestamps poll starts/ends; on `hard_hang_s` the COM
thread is abandoned and rebuilt. thread is abandoned and rebuilt.
@@ -107,17 +122,26 @@ Tasks:
hangs, quality flips) → powers the entire unit-test suite without COM/hardware. hangs, quality flips) → powers the entire unit-test suite without COM/hardware.
**Exit:** `opcda-diagnozer run` (foreground console) produces correct events against the **Exit:** `opcda-diagnozer run` (foreground console) produces correct events against the
fake client; unit tests for all detectors. fake client; unit tests for all detectors — including D16 edge semantics (false /
non-GOOD / read-error / misconfigured value / recovery) against scripted indicator reads.
## 5. Phase 2 — Reporters (≈1 day) ## 5. Phase 2 — Reporters (≈1 day)
1. `report/jsonl.py` — rotation (size + backups), atomic-ish append, header line with 1. `report/jsonl.py` — rotation (size + backups), atomic-ish append, header line with
config digest for provenance. config digest for provenance.
2. `report/eventlog.py` — pywin32 `win32evtlog` usage; map severities; source 2. `report/textlog.py` — **plain-text anomaly log** (spec §9.1): `anomalies-YYYYMMDD.log`
in the same directory as the JSONL files, one human-readable line per detected
anomaly (`timestamp SEVERITY KIND message`) with clear-text status `NAME (0xNN)`.
Anomalies only — heartbeats/CONNECTED are filtered out (a small `is_anomaly(event)`
predicate on `EventKind`); recoveries (D2, `DEVICE_LINK_UP`, quality back to GOOD)
**are** logged, so the file reads as an incident narrative. Daily + size rotation
with `backups` kept, same scheme as the JSONL reporter. Every line corresponds to
a JSONL record (text = convenience view, JSONL = source of truth).
3. `report/eventlog.py` — pywin32 `win32evtlog` usage; map severities; source
registration via `install` step. registration via `install` step.
3. `report/http.py` — optional webhook with timeout + simple retry/drop on failure 4. `report/http.py` — optional webhook with timeout + simple retry/drop on failure
(never blocks the poll loop; queue with bound). (never blocks the poll loop; queue with bound).
4. Heartbeat assembly (D15): uptime, reconnects, RTT percentiles, tags-bad counts. 5. Heartbeat assembly (D15): uptime, reconnects, RTT percentiles, tags-bad counts.
**Exit:** events visible in file + Event Log; heartbeat cadence verified. **Exit:** events visible in file + Event Log; heartbeat cadence verified.
@@ -150,6 +174,7 @@ Scenario matrix against a simulation server (and, when available, the real serve
| Quality oscillation | toggling sim items | D11 flapping | | Quality oscillation | toggling sim items | D11 flapping |
| Stale/frozen tag | simulation server static item marked `expect_change` | D12 / D13 | | Stale/frozen tag | simulation server static item marked `expect_change` | D12 / D13 |
| Mass bad quality | all sim items to bad | D14 | | Mass bad quality | all sim items to bad | D14 |
| **Device link down** | indicator tag flipped to `false` / set BAD (alias a bool sim item per device, e.g. `P5001.Connected` on Matrikon) | D16 `DEVICE_LINK_DOWN`, recovery line when restored |
| Config errors | bad ProgID, unknown tag | fast fail + actionable message; `OPC_E_INVALIDITEMID` surfaced at startup | | Config errors | bad ProgID, unknown tag | fast fail + actionable message; `OPC_E_INVALIDITEMID` surfaced at startup |
| Soak | 24 h healthy run | no leaks, heartbeats every 5 min, zero false positives | | Soak | 24 h healthy run | no leaks, heartbeats every 5 min, zero false positives |
+66
View File
@@ -99,9 +99,44 @@ diagnosis.
| D13 | Frozen value | `expect_change: true` tag with unchanged value+timestamp for `max_age_s` | WARN | | D13 | Frozen value | `expect_change: true` tag with unchanged value+timestamp for `max_age_s` | WARN |
| D14 | Mass quality degradation | ≥ 50% of monitored tags BAD simultaneously while connection OK | CRITICAL (`CONTROL_LINK_DEGRADED` — points at OPC-server↔device link, not DCOM) | | D14 | Mass quality degradation | ≥ 50% of monitored tags BAD simultaneously while connection OK | CRITICAL (`CONTROL_LINK_DEGRADED` — points at OPC-server↔device link, not DCOM) |
| D15 | Heartbeat / summary | Periodic stats snapshot every `heartbeat_s` (300) | INFO | | D15 | Heartbeat / summary | Periodic stats snapshot every `heartbeat_s` (300) | INFO |
| D16 | Device link down | A configured device-indicator tag reads `false`, or any quality other than GOOD, or a read error | CRITICAL per device (`DEVICE_LINK_DOWN`); D1/D2 do **not** imply device-link state |
All thresholds are configurable; severities are INFO / WARN / CRITICAL. All thresholds are configurable; severities are INFO / WARN / CRITICAL.
### 6.1 Device connection-indicator tags (D16)
The config names one or more tags as **device-link indicators**: values the
control system maintains that say whether the OPC server can reach the device.
The **normal (connected) reading is configurable per indicator** — conventions
differ per control system, so it is never hardcoded:
```yaml
devices:
- { name: "Conveyor 3", tag: "...Connected" } # any truthy: true/1
- { name: "Pump A", tag: "...PumpAFault", ok_values: [false, 0] } # inverted fault flag
- { name: "Boiler 2", tag: "...State", ok_values: ["RUN"] } # enum/string state
```
Default `ok_values: [true]` (boolean connected-flags are the common case).
Value comparison is by **string equality** on the read value rendered as text
(`"True"`, `"1"`, `"RUN"`), so type quirks across systems cannot cause false
positives; quality must also be GOOD. Semantics (all DOWN cases are logged as
`DEVICE_LINK_DOWN`):
| Indicator reading | Interpretation |
|-------------------|------------------|
| GOOD + value ∈ `ok_values` | Device connected — normal |
| GOOD + value ∉ `ok_values` | **Device link down / fault** — CRITICAL (works for inverted fault-flags: a fault `1` is not in `[false, 0]`) |
| Any quality other than GOOD (BAD/UNCERTAIN) | **Something is wrong with the device link** — CRITICAL |
| Read error (tag unreadable) | **Something is wrong with the device link** — CRITICAL |
Device-link state is per indicator tag: the config maps each indicator to a
human-readable device name for the log lines (see `config.example.yaml`).
Recovery (`DEVICE_LINK_UP`) is logged when the same indicator reads GOOD with
an ok value again. A value that is neither in `ok_values` nor a recognized
fault shape still logs DOWN — the normal state is explicit, and everything
else is "not normal" by definition.
## 7. OPC DA quality codes (reference) ## 7. OPC DA quality codes (reference)
OPC DA quality is a 16-bit field. Bits 7–6 of the low byte give the class: OPC DA quality is a 16-bit field. Bits 7–6 of the low byte give the class:
@@ -126,6 +161,13 @@ decodes and names:
Remaining UNCERTAIN subcodes (DA 2.05a) are decoded by the same table in `quality.py`. Remaining UNCERTAIN subcodes (DA 2.05a) are decoded by the same table in `quality.py`.
Every quality event carries the raw hex code **and** the decoded name. Every quality event carries the raw hex code **and** the decoded name.
**Clear-text status rule:** wherever a status or quality is reported — JSONL
fields, the text log, Event Log, console — it is written in clear text with
the raw code in parentheses: `BAD / communication failure (0x18)`,
`GOOD (0xC0)`. Nobody should have to decode hex codes by hand, but the raw
code stays for unambiguous correlation with vendor documentation and the
HRESULT table.
## 8. Event model ## 8. Event model
Every event is a structured record: Every event is a structured record:
@@ -163,12 +205,36 @@ alongside and clearly labeled.
| Reporter | Purpose | Notes | | Reporter | Purpose | Notes |
|----------|---------|-------| |----------|---------|-------|
| JSONL rotating file | Primary evidence trail | One JSON event per line; rotation by size; default dir `%ProgramData%\OPCDA-Diagnozer\logs` | | JSONL rotating file | Primary evidence trail | One JSON event per line; rotation by size; default dir `%ProgramData%\OPCDA-Diagnozer\logs` |
| **Plain-text anomaly log** | Human-readable evidence | One line per **detected anomaly** (not heartbeats): timestamp, kind, tag/device, message with clear-text status `NAME (0xNN)`; same rotation scheme as JSONL. See §9.1 for the line format |
| Windows Event Log | Ops visibility | WARN+ only; source `OPCDA-Diagnozer` | | Windows Event Log | Ops visibility | WARN+ only; source `OPCDA-Diagnozer` |
| Console | Foreground/debug runs | Human-readable | | Console | Foreground/debug runs | Human-readable |
| HTTP webhook (optional) | Alert integration | POST one JSON event per request, optional bearer token | | HTTP webhook (optional) | Alert integration | POST one JSON event per request, optional bearer token |
Reporters are pluggable (`Reporter.emit(event)`); a heartbeat event every 5 minutes Reporters are pluggable (`Reporter.emit(event)`); a heartbeat event every 5 minutes
proves liveness and enables baseline graphs even when everything is healthy. proves liveness and enables baseline graphs even when everything is healthy.
The JSONL file stays the machine-readable source of truth; the text log is a
human-first convenience view of the same events — every text line corresponds
to a JSONL record.
### 9.1 Plain-text anomaly log format
File `anomalies-YYYYMMDD.log` (daily rotation, size-capped) in the same log
directory. One line per detected anomaly — anomalies only; heartbeats and
reconnect-OK events do not appear here. Clear text first, codes in parentheses:
```
2026-10-07 14:23:01.123 UTC WARN QUALITY_BAD tag 'Channel1.Device1.Counter' changed GOOD (0xC0) -> BAD / communication failure (0x18)
2026-10-07 14:24:31.500 UTC CRITICAL DEVICE_LINK_DOWN device 'Conveyor 3': link indicator 'Channel1.Device3.Connected' read false (GOOD)
2026-10-07 14:26:02.780 UTC CRITICAL DEVICE_LINK_DOWN device 'Conveyor 3': link indicator 'Channel1.Device3.Connected' quality BAD / not connected (0x08)
2026-10-07 14:27:10.001 UTC CRITICAL POLL_HANG read call hung > 5000 ms; COM thread abandoned and rebuilt
2026-10-07 14:30:00.000 UTC CRITICAL CONTROL_LINK_DEGRADED 12 of 14 monitored tags BAD while connection OK
```
Fields (pipe-free, fixed order): `timestamp SEVERITY KIND message`.
Timestamps are the local receive time in UTC (§8). The message carries the
clear-text status with the code between parentheses per §7. A matching
recovery (link back up, quality back to GOOD) is also logged as an anomaly
line, so the log reads as a continuous incident narrative.
## 10. Operational behavior ## 10. Operational behavior
+229
View File
@@ -0,0 +1,229 @@
# Build the Windows executable (dist\opcda-diagnozer.exe) with PyInstaller.
#
# Usage (from the repo root):
# powershell -NoProfile -ExecutionPolicy Bypass -File scripts\build_exe.ps1
# .\scripts\build_exe.ps1 # onedir (default): dist\opcda-diagnozer\
# .\scripts\build_exe.ps1 -Onefile # single exe: dist\opcda-diagnozer.exe
# .\scripts\build_exe.ps1 -Clean # remove build\ + dist\ first
#
# The exe is a console application hosting every CLI verb (check-config,
# list-servers, list-devices, list-tags, the future run/probe/service verbs).
# It is not itself a service binary: the Phase 3 service host will register
# this same exe with the SCM via the 'install' verb (IMPLEMENTATION_PLAN 6).
#
# Hard rules enforced here:
# 1. 32-bit interpreter (SPECIFICATION 11): the OPC DA Automation Wrapper
# (OPCAutomation.dll) is a 32-bit in-proc COM DLL - a 64-bit exe cannot
# load it and every COM call fails at runtime. The produced PE is
# re-verified to be 32-bit (IMAGE_FILE_MACHINE_I386) after the build.
# 2. PyInstaller must see the modules the code imports lazily inside
# functions (COM, browse, service host) - declared via -HiddenImport.
[CmdletBinding()]
param(
[switch]$Onefile,
[switch]$Clean
)
$ErrorActionPreference = 'Stop'
$Repo = Split-Path -Parent $PSScriptRoot
$VenvPython = Join-Path $Repo '.venv\Scripts\python.exe'
$Entry = Join-Path $Repo 'src\opcda_diagnozer\__main__.py'
$Dist = Join-Path $Repo 'dist'
$Build = Join-Path $Repo 'build'
$Name = 'opcda-diagnozer'
$VersionFile = Join-Path $PSScriptRoot '.exe_version_info.txt'
# Modules imported lazily inside functions (import guards, IMPLEMENTATION_PLAN 9);
# PyInstaller's static analysis misses those, so they are declared explicitly.
$HiddenImports = @(
# pywin32 / COM - deferred into methods in opc/automation.py, opc/discovery.py
'pythoncom',
'pywintypes',
'win32com.client',
# pywintypes.datetime(VARIANT DATE -> datetime conversion) lazily imports
# win32timezone at CONVERSION time - missing from the bundle, every read of
# item.TimeStamp raised SystemError('InvokeTypes ... exception set') frozen
# (observed 2026-10-07: all tags failed in the exe while the venv worked).
'win32timezone',
# opc backend modules imported lazily in main.py CLI verbs
'opcda_diagnozer.opc.automation',
'opcda_diagnozer.opc.browse',
# future service host (Phase 3) - keep it in the bundle from day one
'opcda_diagnozer.service'
)
function Fail {
param([string]$Message)
Write-Host 'build_exe: FAILED' -ForegroundColor Red
Write-Host " $Message" -ForegroundColor Red
exit 1
}
function Invoke-Captured {
# Run a native command and capture stdout+stderr with its exit code.
# $ErrorActionPreference must be relaxed around 2>&1 on PS 5.1: with 'Stop',
# redirecting native stderr raises a NativeCommandError and kills the script.
param([string]$FilePath, [string[]]$Arguments)
$prev = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
try {
$lines = & $FilePath @Arguments 2>&1
$text = ($lines | ForEach-Object { "$_" }) -join [Environment]::NewLine
[pscustomobject]@{ ExitCode = $LASTEXITCODE; Output = $text }
}
finally {
$ErrorActionPreference = $prev
}
}
# --- environment checks ------------------------------------------------------
if (-not (Test-Path $VenvPython)) {
Fail ("the 32-bit venv was not found: $VenvPython - create it first " +
"(README 'Development setup'): " +
'& "C:\...\Python314-32\python.exe" -m venv .venv; ' +
'& .\.venv\Scripts\python.exe -m pip install -e ".[dev]"')
}
$bitness = (Invoke-Captured -FilePath $VenvPython -Arguments @('-c', "import struct; print(struct.calcsize('P') * 8)")).Output.Trim()
if ($bitness -ne '32') {
Fail ("the venv interpreter is ${bitness}-bit, but the OPC DA wrappers are " +
'32-bit in-proc DLLs (SPECIFICATION 11): a 64-bit exe cannot load ' +
"them. Rebuild the venv with the 32-bit Python (README 'Development setup').")
}
Write-Host 'build_exe: OK - 32-bit interpreter (SPECIFICATION 11 bitness rule)'
$pyi = Invoke-Captured -FilePath $VenvPython -Arguments @('-m', 'PyInstaller', '--version')
if ($pyi.ExitCode -ne 0) {
Write-Host 'build_exe: pyinstaller missing - installing into the venv ...'
& $VenvPython -m pip install pyinstaller
if ($LASTEXITCODE -ne 0) { Fail 'pip install pyinstaller failed' }
}
if ($Clean) {
foreach ($folder in @($Build, $Dist)) {
if (Test-Path $folder) { Remove-Item -Recurse -Force $folder }
}
Write-Host 'build_exe: cleaned build/ and dist/'
}
# --- version metadata (exe properties -> Details tab) -------------------------
$initText = Get-Content (Join-Path $Repo 'src\opcda_diagnozer\__init__.py') -Raw
if ($initText -notmatch '__version__\s*=\s*"([^"]+)"') {
Fail 'could not read __version__ from src\opcda_diagnozer\__init__.py'
}
$Version = $Matches[1]
$parts = $Version.Split('.')
$major = $parts[0]
$minor = $parts[1]
# Keyword names must match PyInstaller's FixedFileInfo.__init__ exactly:
# filevers, prodvers, mask, flags, OS, fileType, subtype, date.
# [IO.File]::WriteAllText writes UTF-8 WITHOUT a BOM - Set-Content -Encoding
# UTF8 on PS 5.1 prepends one, and the BOM breaks PyInstaller's eval().
$VersionInfo = @"
# UTF-8
VSVersionInfo(
ffi=FixedFileInfo(
filevers=($major, $minor, 0, 0),
prodvers=($major, $minor, 0, 0),
mask=0x3F,
flags=0x0,
OS=0x40004,
fileType=0x1,
subtype=0x0,
date=(0, 0)
),
kids=[
StringFileInfo(
[
StringTable(
'040904B0',
[
StringStruct('CompanyName', 'Vivab'),
StringStruct('FileDescription', 'OPC DA Diagnozer - read-only OPC DA reference diagnostic client'),
StringStruct('FileVersion', '$Version.0'),
StringStruct('InternalName', 'opcda-diagnozer'),
StringStruct('OriginalFilename', 'opcda-diagnozer.exe'),
StringStruct('ProductName', 'OPC DA Diagnozer'),
StringStruct('ProductVersion', '$Version')
]
)
]
),
VarFileInfo([VarStruct('Translation', [1033, 1200])])
]
)
"@
[IO.File]::WriteAllText($VersionFile, $VersionInfo)
# --- the build itself ---------------------------------------------------------
$pyArgs = @(
'-m', 'PyInstaller',
'--noconfirm', '--clean',
'--name', $Name,
'--distpath', $Dist,
'--workpath', $Build,
'--specpath', $Build,
'--console',
'--version-file', $VersionFile
)
foreach ($hidden in $HiddenImports) { $pyArgs += @('--hidden-import', $hidden) }
$pyArgs += $(if ($Onefile) { '--onefile' } else { '--onedir' })
$pyArgs += $Entry
Write-Host 'build_exe: running PyInstaller...'
& $VenvPython @pyArgs
if ($LASTEXITCODE -ne 0) { Fail "PyInstaller exited $LASTEXITCODE" }
$Exe = if ($Onefile) { Join-Path $Dist "$Name.exe" } else { Join-Path $Dist "$Name\$Name.exe" }
if (-not (Test-Path $Exe)) { Fail "expected exe not found: $Exe" }
Write-Host "build_exe: OK - $Exe"
# --- final bitness guard: the produced PE must be 32-bit (SPECIFICATION 11) --
$bytes = [System.IO.File]::ReadAllBytes($Exe)
$peOffset = [System.BitConverter]::ToInt32($bytes, 0x3C)
$machine = [System.BitConverter]::ToUInt16($bytes, $peOffset + 4)
if ($machine -ne 0x14C) {
Fail ("the built exe is not 32-bit (PE machine 0x{0:X4}) - build from the 32-bit venv" -f $machine)
}
Write-Host 'build_exe: OK - exe is 32-bit (PE machine 0x014C, IMAGE_FILE_MACHINE_I386)'
# --- smoke test ---------------------------------------------------------------
Write-Host "build_exe: smoke-testing $Name.exe ..."
$ExampleConfig = Join-Path $Repo 'config.example.yaml'
$checks = @(
@{ Label = '--version'; Arguments = @('--version') },
@{ Label = 'check-config'; Arguments = @('check-config', '--config', $ExampleConfig) },
@{ Label = 'list-servers (registry)'; Arguments = @('list-servers', '--source', 'registry') }
)
foreach ($check in $checks) {
$result = Invoke-Captured -FilePath $Exe -Arguments $check.Arguments
$first = ($result.Output -split '\r?\n' | Where-Object { $_.Trim() } | Select-Object -First 1)
Write-Host (" {0} -> exit {1}: {2}" -f $check.Label, $result.ExitCode, $first)
if ($result.ExitCode -ne 0) { Fail ("smoke test failed: " + $check.Label) }
}
# Frozen-read canary: the COM timestamp conversion (item.TimeStamp) lazily
# imports win32timezone - if it is missing from the bundle, EVERY read fails
# with SystemError('InvokeTypes ... exception set'). The registry/discovery
# smoke tests cannot catch this; only a real read does. Skipped gracefully
# when no COM server is reachable on this build host.
$canary = Invoke-Captured -FilePath $Exe -Arguments @(
'read', '--server', 'Matrikon.OPC.Simulation.1', 'Bucket Brigade.Int1'
)
$canaryFirst = ($canary.Output -split '\r?\n' | Where-Object { $_.Trim() } | Select-Object -First 2) -join ' | '
if ($canary.ExitCode -eq 0) {
Write-Host " read canary (Matrikon sim) -> exit 0: $canaryFirst"
} elseif ($canary.Output -match 'SystemError.*InvokeTypes') {
Fail ("frozen read canary failed with the missing-win32timezone signature: $canaryFirst")
} else {
Write-Host " read canary skipped (no Matrikon sim reachable): exit $($canary.ExitCode)"
}
Write-Host 'build_exe: smoke test OK'
exit 0
+102 -1
View File
@@ -38,6 +38,23 @@ class TagEntry:
expect_change: bool = False expect_change: bool = False
@dataclass(frozen=True)
class DeviceEntry:
"""D16 device-link indicator: a tag saying if the device is reachable.
DOWN (``DEVICE_LINK_DOWN``, CRITICAL) when ``tag`` reads non-GOOD, errors,
or its GOOD value is **not** in ``ok_values`` — the normal (connected)
reading, configurable per indicator because conventions differ per
control system: connected-flags ``[true]`` (default), inverted fault-flags
``[false, 0]``, string states ``["RUN"]``. Compared by string equality on
the rendered value (SPECIFICATION §6.1).
"""
name: str
tag: str
ok_values: tuple[str, ...] = ("true",)
@dataclass(frozen=True) @dataclass(frozen=True)
class PollConfig: class PollConfig:
interval_s: float = 1.0 interval_s: float = 1.0
@@ -100,6 +117,7 @@ class Config:
opc: OpcConfig opc: OpcConfig
poll: PollConfig = field(default_factory=PollConfig) poll: PollConfig = field(default_factory=PollConfig)
tags: tuple[TagEntry, ...] = () tags: tuple[TagEntry, ...] = ()
devices: tuple[DeviceEntry, ...] = ()
latency: LatencyConfig = field(default_factory=LatencyConfig) latency: LatencyConfig = field(default_factory=LatencyConfig)
quality: QualityConfig = field(default_factory=QualityConfig) quality: QualityConfig = field(default_factory=QualityConfig)
stale: StaleConfig = field(default_factory=StaleConfig) stale: StaleConfig = field(default_factory=StaleConfig)
@@ -107,8 +125,41 @@ class Config:
report: ReportConfig = field(default_factory=ReportConfig) report: ReportConfig = field(default_factory=ReportConfig)
_ALLOWED_TOP_LEVEL = {"opc", "poll", "tags", "latency", "quality", "stale", "browse", "report"} _ALLOWED_TOP_LEVEL = {"opc", "poll", "tags", "devices", "latency", "quality", "stale", "browse", "report"}
_TAG_KEYS = {"name", "expect_change"} _TAG_KEYS = {"name", "expect_change"}
_DEVICE_KEYS = {"name", "tag", "ok_values"}
def _normalize_ok_values(raw: Any, where: str) -> tuple[str, ...]:
"""ok_values as a tuple of rendered strings ("true", "1", "RUN").
The D16 comparison is string equality on the value rendered as text, so
booleans/ints/strings from YAML all work uniformly: `[true]` -> `("true",)`,
`[false, 0]` -> `("false", "0")`.
"""
if raw is None:
return ("true",)
if isinstance(raw, (list, tuple)):
if not raw:
raise ConfigError(f"{where}: ok_values must not be empty - set the normal (connected) value(s)")
values = []
for item in raw:
if isinstance(item, bool):
values.append("true" if item else "false")
elif isinstance(item, (int, float, str)):
values.append(str(item).strip())
else:
raise ConfigError(
f"{where}: ok_values entries must be scalars (bool/int/str), got {type(item).__name__}"
)
cleaned = tuple(v for v in values if v)
if not cleaned:
raise ConfigError(f"{where}: ok_values must not be empty - set the normal (connected) value(s)")
return cleaned
# a single scalar is shorthand for a one-element list
if isinstance(raw, (bool, int, float, str)):
return _normalize_ok_values([raw], where)
raise ConfigError(f"{where}: ok_values must be a list of scalars, got {type(raw).__name__}")
def _build_section(cls: type[Any], data: Any, where: str) -> Any: def _build_section(cls: type[Any], data: Any, where: str) -> Any:
@@ -169,6 +220,37 @@ def _build_tags(data: Any) -> tuple[TagEntry, ...]:
return tuple(entries) return tuple(entries)
def _build_devices(data: Any) -> tuple[DeviceEntry, ...]:
"""Parse the devices: section (D16 link indicators) with actionable errors."""
if data is None:
return ()
if not isinstance(data, list):
raise ConfigError(f"devices: must be a list, got {type(data).__name__}")
entries: list[DeviceEntry] = []
for i, item in enumerate(data):
if not isinstance(item, dict):
raise ConfigError(
f"devices[{i}]: must be a mapping with 'name' and 'tag'"
)
unknown = sorted(set(item) - _DEVICE_KEYS)
if unknown:
raise ConfigError(
f"devices[{i}]: unknown key(s) {unknown}; accepted keys: {sorted(_DEVICE_KEYS)}"
)
name = item.get("name")
tag = item.get("tag")
if not isinstance(name, str) or not name.strip():
raise ConfigError(f"devices[{i}]: 'name' is required and must be a non-empty string")
if not isinstance(tag, str) or not tag.strip():
raise ConfigError(
f"devices[{i}]: 'tag' is required and must be the connection-indicator "
"tag ItemID (see list-items)"
)
ok_values = _normalize_ok_values(item.get("ok_values"), f"devices[{i}]")
entries.append(DeviceEntry(name=name, tag=tag, ok_values=ok_values))
return tuple(entries)
def _validate(config: Config) -> None: def _validate(config: Config) -> None:
"""Cross-field validation; each message names the offending key.""" """Cross-field validation; each message names the offending key."""
if not config.opc.server or not config.opc.server.strip(): if not config.opc.server or not config.opc.server.strip():
@@ -186,6 +268,24 @@ def _validate(config: Config) -> None:
duplicates = sorted({n for n in names if names.count(n) > 1}) duplicates = sorted({n for n in names if names.count(n) > 1})
if duplicates: if duplicates:
raise ConfigError(f"tags: duplicate tag name(s) {duplicates}") raise ConfigError(f"tags: duplicate tag name(s) {duplicates}")
device_names = [d.name for d in config.devices]
duplicate_devices = sorted({n for n in device_names if device_names.count(n) > 1})
if duplicate_devices:
raise ConfigError(f"devices: duplicate device name(s) {duplicate_devices}")
indicator_tags = [d.tag for d in config.devices]
duplicate_indicators = sorted({t for t in indicator_tags if indicator_tags.count(t) > 1})
if duplicate_indicators:
raise ConfigError(
f"devices: duplicate indicator tag(s) {duplicate_indicators} - one device per indicator"
)
# An indicator doubles as a monitored tag; listing it in tags: too would double-poll
# it and double-report its quality transitions (D8 AND D16 fire).
overlap = sorted(set(indicator_tags) & set(names))
if overlap:
raise ConfigError(
f"tags: {overlap} also appear as device indicators - remove them from "
"tags: (devices: entries are polled automatically)"
)
if config.poll.interval_s <= 0: if config.poll.interval_s <= 0:
raise ConfigError(f"poll.interval_s: must be > 0, got {config.poll.interval_s}") raise ConfigError(f"poll.interval_s: must be > 0, got {config.poll.interval_s}")
if config.poll.timeout_ms <= 0: if config.poll.timeout_ms <= 0:
@@ -253,6 +353,7 @@ def load_config(path: str | Path) -> Config:
opc=_build_section(OpcConfig, data.get("opc"), "opc"), opc=_build_section(OpcConfig, data.get("opc"), "opc"),
poll=_build_section(PollConfig, data.get("poll"), "poll"), poll=_build_section(PollConfig, data.get("poll"), "poll"),
tags=_build_tags(data.get("tags")), tags=_build_tags(data.get("tags")),
devices=_build_devices(data.get("devices")),
latency=_build_section(LatencyConfig, data.get("latency"), "latency"), latency=_build_section(LatencyConfig, data.get("latency"), "latency"),
quality=_build_section(QualityConfig, data.get("quality"), "quality"), quality=_build_section(QualityConfig, data.get("quality"), "quality"),
stale=_build_section(StaleConfig, data.get("stale"), "stale"), stale=_build_section(StaleConfig, data.get("stale"), "stale"),
+4
View File
@@ -217,6 +217,10 @@ def _cmd_check_config(config_path: str) -> int:
print("check-config: OK") print("check-config: OK")
print(f" server: {config.opc.server!r}{host}") print(f" server: {config.opc.server!r}{host}")
print(f" tags: {len(config.tags)} ({expect_change} with expect_change)") print(f" tags: {len(config.tags)} ({expect_change} with expect_change)")
if config.devices:
print(f" devices: {len(config.devices)} link indicator(s):")
for device in config.devices:
print(f" {device.name} <- {device.tag}")
print(f" poll: every {config.poll.interval_s} s, hang flagged after {config.poll.timeout_ms} ms") print(f" poll: every {config.poll.interval_s} s, hang flagged after {config.poll.timeout_ms} ms")
return 0 return 0
+37 -8
View File
@@ -47,6 +47,20 @@ from opcda_diagnozer.opc.base import (
_ORGANIZATION_LABELS = {1: "hierarchical", 2: "flat"} _ORGANIZATION_LABELS = {1: "hierarchical", 2: "flat"}
def _per_tag_error(what: str, exc: BaseException) -> str:
"""Render a per-tag failure without swallowing the underlying cause.
COM errors (``com_error`` with an ``hresult``) keep their decoded HRESULT
suffix via :func:`as_com_error`. **Plain Python exceptions** (e.g. a
``ValueError`` from timezone math on a degenerate device timestamp) must
show their own type and text — early builds reduced them to a bare
"read of tag 'X' failed", which made failures undiagnosable.
"""
if getattr(exc, "hresult", None) is None:
return f"{what}; {type(exc).__name__}: {exc}"
return str(as_com_error(what, exc))
class _AutomationBrowser: class _AutomationBrowser:
"""Thin, read-only :class:`Browser` over one wrapper browser object. """Thin, read-only :class:`Browser` over one wrapper browser object.
@@ -57,7 +71,10 @@ class _AutomationBrowser:
""" """
def __init__(self, raw_browser: Any) -> None: def __init__(self, raw_browser: Any) -> None:
self._browser = raw_browser # Explicit ``Any`` declaration: close() assigns None to drop the COM
# reference, and the inferred ``Any | None`` would flag every access
# below (the dispatch object is dynamic — always accessed raw).
self._browser: Any = raw_browser
@property @property
def organization(self) -> str: def organization(self) -> str:
@@ -260,23 +277,35 @@ class AutomationOpcClient:
value=None, value=None,
quality_raw=0, quality_raw=0,
device_time=None, device_time=None,
error=str(as_com_error(f"cannot add tag {tag!r}", exc)), error=_per_tag_error(f"cannot add tag {tag!r}", exc),
) )
) )
for tag, item in live_items: for tag, item in live_items:
try: try:
item.Read(int(source)) item.Read(int(source))
# Primary data first: value + quality are the read's
# purpose. The device timestamp is METADATA and is
# fetched last, fully guarded — a broken timestamp
# (missing win32timezone in a frozen exe, zero FILETIME
# 1601 on a never-updated alias, DST math on a
# historical local-time wrapper) must never fail a read
# whose value/quality are good.
value = item.Value
quality = int(item.Quality)
device_time = None
try:
raw_ts = item.TimeStamp raw_ts = item.TimeStamp
# pywintypes.datetime is a datetime subclass; normalize # pywintypes.datetime is a datetime subclass; normalize
# to UTC for the event model (SPECIFICATION §6). # to UTC for the event model (SPECIFICATION §6).
device_time = ( if raw_ts is not None:
raw_ts.astimezone(timezone.utc) if raw_ts is not None else None device_time = raw_ts.astimezone(timezone.utc)
) except Exception: # noqa: BLE001 — timestamp is metadata
device_time = None
results.append( results.append(
ReadResult( ReadResult(
tag=tag, tag=tag,
value=item.Value, value=value,
quality_raw=int(item.Quality), quality_raw=quality,
device_time=device_time, device_time=device_time,
) )
) )
@@ -287,7 +316,7 @@ class AutomationOpcClient:
value=None, value=None,
quality_raw=0, quality_raw=0,
device_time=None, device_time=None,
error=str(as_com_error(f"read of tag {tag!r} failed", exc)), error=_per_tag_error(f"read of tag {tag!r} failed", exc),
) )
) )
except Exception as exc: # noqa: BLE001 — group-level failure except Exception as exc: # noqa: BLE001 — group-level failure
+96
View File
@@ -23,6 +23,13 @@ def test_example_config_loads() -> None:
assert config.tags[0].name == "Channel1.Device1.Counter" assert config.tags[0].name == "Channel1.Device1.Counter"
assert config.tags[0].expect_change is True assert config.tags[0].expect_change is True
assert config.tags[1].expect_change is False assert config.tags[1].expect_change is False
# D16 device-link indicators (spec 6.1); normal reading configurable per device
assert [d.name for d in config.devices] == ["Conveyor 3", "Conveyor 4", "Pump A", "Boiler 2"]
assert config.devices[0].tag == "Channel1.Device3.Connected"
assert config.devices[0].ok_values == ("true",) # default: boolean connected-flag
assert config.devices[1].ok_values == ("true", "1")
assert config.devices[2].ok_values == ("false", "0") # inverted fault flag: 0 = ok
assert config.devices[3].ok_values == ("RUN",) # string state
assert config.poll.interval_s == 1.0 assert config.poll.interval_s == 1.0
assert config.poll.timeout_ms == 5000 assert config.poll.timeout_ms == 5000
assert config.latency.warn_ms == 1000 assert config.latency.warn_ms == 1000
@@ -65,6 +72,95 @@ def test_duplicate_tags_rejected(tmp_path: Path) -> None:
load_config(p) load_config(p)
def test_devices_missing_keys_rejected(tmp_path: Path) -> None:
p = _write(tmp_path, "opc:\n server: X\ntags:\n - name: t\ndevices:\n - name: Device\n")
with pytest.raises(ConfigError, match="'tag' is required"):
load_config(p)
def test_devices_blank_name_rejected(tmp_path: Path) -> None:
p = _write(
tmp_path,
"opc:\n server: X\ntags:\n - name: t\ndevices:\n - { name: '', tag: Ind.Tag }\n",
)
with pytest.raises(ConfigError, match="non-empty"):
load_config(p)
def test_devices_duplicate_indicator_rejected(tmp_path: Path) -> None:
p = _write(
tmp_path,
"opc:\n server: X\ntags:\n - name: t\n"
"devices:\n - { name: A, tag: Ind.Tag }\n - { name: B, tag: Ind.Tag }\n",
)
with pytest.raises(ConfigError, match="one device per indicator"):
load_config(p)
def test_devices_indicator_in_tags_rejected(tmp_path: Path) -> None:
"""An indicator is polled automatically; listing it in tags: double-reports."""
p = _write(
tmp_path,
"opc:\n server: X\n"
"tags:\n - name: Ind.Tag\ndevices:\n - { name: A, tag: Ind.Tag }\n",
)
with pytest.raises(ConfigError, match="remove them from"):
load_config(p)
def test_devices_empty_is_allowed(tmp_path: Path) -> None:
"""devices: is optional (no link monitoring) and may be an empty list."""
p = _write(tmp_path, "opc:\n server: X\ntags:\n - name: t\ndevices: []\n")
assert load_config(p).devices == ()
def test_devices_ok_values_default_is_true(tmp_path: Path) -> None:
p = _write(
tmp_path,
"opc:\n server: X\ntags:\n - name: t\ndevices:\n - { name: A, tag: I.Tag }\n",
)
device = load_config(p).devices[0]
assert device.ok_values == ("true",)
def test_devices_ok_values_normalized(tmp_path: Path) -> None:
"""Booleans/ints/strings all render to comparison strings; scalar = shorthand."""
p = _write(
tmp_path,
"opc:\n server: X\ntags:\n - name: t\n"
"devices:\n"
" - { name: A, tag: I.One, ok_values: [true, 1] }\n"
" - { name: B, tag: I.Two, ok_values: [false, 0] }\n"
" - { name: C, tag: I.Three, ok_values: [\"RUN\"] }\n"
" - { name: D, tag: I.Four, ok_values: 1 }\n",
)
devices = {d.name: d for d in load_config(p).devices}
assert devices["A"].ok_values == ("true", "1")
assert devices["B"].ok_values == ("false", "0")
assert devices["C"].ok_values == ("RUN",)
assert devices["D"].ok_values == ("1",) # single scalar shorthand
def test_devices_ok_values_empty_rejected(tmp_path: Path) -> None:
p = _write(
tmp_path,
"opc:\n server: X\ntags:\n - name: t\n"
"devices:\n - { name: A, tag: I.Tag, ok_values: [] }\n",
)
with pytest.raises(ConfigError, match="must not be empty"):
load_config(p)
def test_devices_ok_values_non_scalar_rejected(tmp_path: Path) -> None:
p = _write(
tmp_path,
"opc:\n server: X\ntags:\n - name: t\n"
"devices:\n - { name: A, tag: I.Tag, ok_values: [[1]] }\n",
)
with pytest.raises(ConfigError, match="scalars"):
load_config(p)
def test_latency_threshold_order_rejected(tmp_path: Path) -> None: def test_latency_threshold_order_rejected(tmp_path: Path) -> None:
p = _write( p = _write(
tmp_path, tmp_path,
+180
View File
@@ -121,6 +121,186 @@ def test_read_error_tag_reports_error_not_crash(
assert "0xC0040008" in captured.out assert "0xC0040008" in captured.out
# --------------------------------------------------------------------------- #
# per-tag error rendering (_per_tag_error) — no COM needed
# --------------------------------------------------------------------------- #
def _fake_com_error(hresult: int) -> Exception:
"""A stand-in with the pywin32 com_error shape for the hresult check."""
err = Exception("server rejected the call")
err.hresult = hresult # type: ignore[attr-defined]
return err
def test_per_tag_error_keeps_hresult_for_com_errors() -> None:
from opcda_diagnozer.opc.automation import _per_tag_error
rendered = _per_tag_error("cannot add tag 'X'", _fake_com_error(0x80020009))
assert "cannot add tag 'X'" in rendered
assert "HRESULT 0x80020009" in rendered # decoded COM suffix preserved
def test_per_tag_error_names_plain_exceptions() -> None:
"""A plain ValueError must show its type+text, not a bare 'failed'."""
from opcda_diagnozer.opc.automation import _per_tag_error
rendered = _per_tag_error("read of tag 'X' failed", ValueError("tz math broke"))
assert "read of tag 'X' failed" in rendered
assert "ValueError" in rendered
assert "tz math broke" in rendered
assert "HRESULT" not in rendered # never fake a COM suffix for non-COM errors
def test_timestamp_failure_never_fails_a_read() -> None:
"""A degenerate device timestamp is metadata: value/quality stay readable.
Reproduces the alias-tag failure shape end-to-end through the adapter's
read() with a stubbed COM item: a TimeStamp whose astimezone() explodes
(zero FILETIME 1601 / DST math on Matrikon aliases) must yield a GOOD
result with device_time=None — never a per-tag error.
"""
from datetime import timezone as _tz
from opcda_diagnozer.opc import automation
from opcda_diagnozer.opc.base import OPCDataSource
class _PoisonedItem:
"""A COM item stand-in: reads succeed, TimeStamp cannot be normalized."""
ItemID = "P5002.Connected"
def __init__(self) -> None:
self.read_calls: list[int] = []
def Read(self, source: int) -> None:
self.read_calls.append(source)
@property
def Value(self):
return 1
@property
def Quality(self):
return 0xC0
@property
def TimeStamp(self):
class _Bad:
def astimezone(self, tz):
raise ValueError("astimezone() cannot be applied to a naive datetime")
return _Bad()
class _Items:
def AddItem(self, item_id: str, handle: int):
assert item_id == "P5002.Connected"
return _PoisonedItem()
class _Group:
OPCItems = _Items()
class _Parent:
OPCGroups = None # set below
Parent = _Parent()
class _Groups:
def Add(self, name: str):
return _Group()
class _Opc: # minimal stand-in for the wrapper's dispatch object
OPCGroups = _Groups()
class _ClientStub(automation.AutomationOpcClient):
def __init__(self) -> None:
super().__init__(server="stub")
self._opc = _Opc() # read() requires a connected-looking client
def _release_and_uninit(self) -> None: # no COM to release on the stub
self._opc = None
client = _ClientStub()
results = client.read(["P5002.Connected"], source=OPCDataSource.DEVICE)
assert len(results) == 1
result = results[0]
assert result.error is None # the poison never surfaced
assert result.value == 1
assert result.quality_raw == 0xC0
assert result.device_time is None # metadata dropped, not the read
def test_timestamp_property_explosion_never_fails_a_read() -> None:
"""The frozen-exe failure shape: the TimeStamp PROPERTY itself explodes.
Missing win32timezone made pywin32 raise SystemError('InvokeTypes ...
exception set') when *accessing* item.TimeStamp (not in astimezone) — the
guard must therefore wrap the property fetch too, and the read must still
return the value/quality.
"""
from opcda_diagnozer.opc import automation
from opcda_diagnozer.opc.base import OPCDataSource
class _ExplodingTsItem:
ItemID = "Bucket Brigade.Int1"
def Read(self, source: int) -> None:
return None
@property
def Value(self):
return 42
@property
def Quality(self):
return 0xC0
@property
def TimeStamp(self):
# the exact frozen-exe signature: a non-COM SystemError raised by
# pywin32's conversion inside the property access
raise SystemError(
"<built-in method InvokeTypes of PyIDispatch object> "
"returned a result with an exception set"
)
class _Items:
def AddItem(self, item_id: str, handle: int):
return _ExplodingTsItem()
class _Group:
OPCItems = _Items()
class _Parent:
OPCGroups = None
Parent = _Parent()
class _Groups:
def Add(self, name: str):
return _Group()
class _Opc:
OPCGroups = _Groups()
class _ClientStub(automation.AutomationOpcClient):
def __init__(self) -> None:
super().__init__(server="stub")
self._opc = _Opc()
def _release_and_uninit(self) -> None:
self._opc = None
client = _ClientStub()
results = client.read(["Bucket Brigade.Int1"], source=OPCDataSource.DEVICE)
assert len(results) == 1
result = results[0]
assert result.error is None # the timestamp explosion never surfaced
assert result.value == 42
assert result.quality_raw == 0xC0
assert result.device_time is None
def test_probe_reads_config_tags(monkeypatch: pytest.MonkeyPatch, tmp_path: Path, capsys) -> None: def test_probe_reads_config_tags(monkeypatch: pytest.MonkeyPatch, tmp_path: Path, capsys) -> None:
code, fake = _run_cli(monkeypatch, tmp_path, [ code, fake = _run_cli(monkeypatch, tmp_path, [
"probe", "-c", str(tmp_path / "config.yaml"), "probe", "-c", str(tmp_path / "config.yaml"),