🎉 Initial commit
This commit is contained in:
Jonas Bengtsson
@@ -0,0 +1,198 @@
|
||||
# OPC DA Diagnozer
|
||||
|
||||
A read-only Windows service that acts as an independent OPC DA reference client to
|
||||
diagnose **disconnections**, **communication latency**, and **data-quality problems**
|
||||
between a SCADA system and a control system communicating over OPC DA.
|
||||
|
||||
It continuously connects to the OPC DA server, polls a pre-defined list of tags, and
|
||||
reports every anomaly with precise timestamps — unnatural read latencies, DCOM
|
||||
disconnects, hung RPC calls, tags marked BAD/UNCERTAIN, stale timestamps, frozen values,
|
||||
and flapping quality — as structured events (JSONL logs, Windows Event Log, optional
|
||||
HTTP webhook).
|
||||
|
||||
> **Status:** planning. Read the
|
||||
> [specification](docs/SPECIFICATION.md) and the
|
||||
> [implementation plan](docs/IMPLEMENTATION_PLAN.md) first.
|
||||
|
||||
## Why
|
||||
|
||||
OPC DA failures are notoriously hard to pin down because they hide behind generic COM
|
||||
errors, calls that hang with no timeout, or silently degraded item quality. This tool
|
||||
answers the triage question *where is the problem?* with evidence:
|
||||
|
||||
| Observation from the Diagnozer | Most likely problem location |
|
||||
|--------------------------------|------------------------------|
|
||||
| Clean reads while SCADA complains | SCADA host itself (DCOM perms, CPU, its own load) |
|
||||
| Diagnozer also sees disconnects/RPC errors | Shared path: network, firewall, DCOM, OPC server host |
|
||||
| Connection stable but many tags `BAD_COMM_FAILURE` | Link between OPC server and the control devices |
|
||||
| Hangs without errors | Silent network drop (firewall/MTU) — classic DCOM pitfall |
|
||||
|
||||
See the full interpretation matrix in the [specification](docs/SPECIFICATION.md).
|
||||
|
||||
## Planned features
|
||||
|
||||
- Windows service (auto-start, SCM crash recovery) with a foreground `run` mode for testing
|
||||
- Local or remote (DCOM) connections to any OPC DA server
|
||||
- Server browsing: `list-servers` (discovery), `list-devices` and `list-tags`
|
||||
(address-space walk) — live now
|
||||
- Latency monitoring: spikes, streaks, and p95 trends against configurable thresholds
|
||||
- Hang watchdog: flags RPC calls that never return — the silent killer
|
||||
- Quality monitoring: BAD/UNCERTAIN transitions with decoded subcodes, persistent-bad
|
||||
and flapping detection, mass-degradation alarm
|
||||
- Staleness & frozen-value detection while quality still reports GOOD
|
||||
- Structured events: rotating JSONL, Windows Event Log, optional HTTP webhook
|
||||
- Strictly read-only; single group, modest poll rate — does not perturb the measured system
|
||||
|
||||
## Quick start
|
||||
|
||||
```
|
||||
# list the OPC DA servers available locally (or on a DCOM host with --host)
|
||||
opcda-diagnozer list-servers
|
||||
opcda-diagnozer list-servers --host CONTROL-HOST01 # remote, via DCOM
|
||||
opcda-diagnozer list-servers --source registry --json # no-COM local scan
|
||||
|
||||
# list the devices (address-space branches) of the configured server
|
||||
opcda-diagnozer list-devices
|
||||
opcda-diagnozer list-devices --depth 2 # include sub-devices
|
||||
opcda-diagnozer list-devices --json # script-friendly output
|
||||
|
||||
# list the tags of one device (the item_id values go into config `tags:`)
|
||||
opcda-diagnozer list-tags "Simulation Items.Bucket Brigade"
|
||||
opcda-diagnozer list-tags "Simulation Items" "Bucket Brigade" # equivalent
|
||||
opcda-diagnozer list-tags --recursive "Simulation Items" # whole subtree
|
||||
opcda-diagnozer list-tags # every tag on the server
|
||||
|
||||
# validate config
|
||||
opcda-diagnozer check-config
|
||||
|
||||
# connect once and read every configured tag, print results
|
||||
opcda-diagnozer probe
|
||||
|
||||
# foreground run (console) — best for first deployment validation
|
||||
opcda-diagnozer run
|
||||
|
||||
# install as Windows service (dedicated account required for DCOM auth)
|
||||
opcda-diagnozer install --account DOMAIN\svc-opcdiag
|
||||
```
|
||||
`list-servers` uses two discovery mechanisms (`--source`): the OPC DA Automation
|
||||
Wrapper's `GetOPCServers` over COM (works remotely via DCOM; needs Launch+Activation
|
||||
permissions on the target host), and a local COM-registration scan that reads the OPC
|
||||
DA component categories directly from the registry — no COM, useful on locked-down
|
||||
hosts or when the wrapper DLL is not registered. `auto` tries the first, falls back to
|
||||
the second. Note the bitness rule still applies: a 32-bit process enumerates the
|
||||
32-bit registry view — exactly the servers this tool can load.
|
||||
|
||||
`list-devices` / `list-tags` browse the **connected server's** address space through
|
||||
the Automation Wrapper's browser (`CreateBrowser`): devices are *branches*, tags are
|
||||
*leaves*. They work locally and over DCOM (same Launch+Activation requirements). A
|
||||
device path is either one dotted string (`"Simulation Items.Bucket Brigade"`) or
|
||||
several quoted branch names — use the latter when a branch name itself contains a
|
||||
dot. On flat-namespace servers (`organization: flat`) every tag hangs directly off
|
||||
the root; `list-tags` without a device lists them all.
|
||||
### Configuration example (`config.example.yaml`)
|
||||
|
||||
```yaml
|
||||
opc:
|
||||
server: "Matrikon.OPC.Simulation.1" # server ProgID
|
||||
host: "CONTROL-HOST01" # DCOM host; omit for local
|
||||
connect_timeout_s: 10
|
||||
|
||||
poll:
|
||||
interval_s: 1.0
|
||||
timeout_ms: 5000 # beyond this the call is flagged as a hang
|
||||
|
||||
tags:
|
||||
- { name: "Channel1.Device1.Counter", expect_change: true }
|
||||
- { name: "Channel1.Device1.Level" }
|
||||
|
||||
latency:
|
||||
warn_ms: 1000
|
||||
critical_ms: 5000
|
||||
slow_poll_streak: 5
|
||||
window_polls: 60
|
||||
|
||||
quality:
|
||||
persistent_bad_s: 30
|
||||
flapping_transitions_per_window: 5
|
||||
|
||||
stale:
|
||||
enabled: true
|
||||
max_age_s: 120
|
||||
|
||||
report:
|
||||
heartbeat_s: 300
|
||||
jsonl: { dir: "C:/ProgramData/OPCDA-Diagnozer/logs", max_mb: 50, backups: 10 }
|
||||
eventlog: { source: "OPCDA-Diagnozer", min_severity: WARN }
|
||||
```
|
||||
|
||||
## DCOM troubleshooting quick reference
|
||||
|
||||
**Setup checklist for the service account (both hosts):**
|
||||
1. `dcomcnfg` → Component Services → DCOM Config → the OPC server → Properties:
|
||||
- Security → Launch & Activation: allow **Remote Activation** for the service account.
|
||||
- Security → Access Permissions: allow Local + Remote access.
|
||||
- Identity: prefer a dedicated account over "Launching user".
|
||||
2. Firewall on the OPC server host: TCP 135 (RPC endpoint mapper) + the RPC dynamic
|
||||
port range (optionally restricted via `netsh int ipv4 set dynamicport tcp ...`).
|
||||
3. NTP time sync on both hosts — OPC DA timestamps and staleness detection depend on it.
|
||||
4. Be aware of post-2022 Windows **DCOM hardening** (KB5004442 family): DCOM activation
|
||||
requires explicit authentication and may break legacy OPC setups with
|
||||
`E_ACCESSDENIED`-type failures.
|
||||
|
||||
**Common HRESULTs the tool decodes and reports:**
|
||||
|
||||
| HRESULT | Meaning |
|
||||
|---------|---------|
|
||||
| `0x800706BA` | RPC server unavailable — server process down, firewall, or DCOM activation refused |
|
||||
| `0x800706BE` | RPC call failed mid-flight — server crashed or connection reset |
|
||||
| `0x80010108` | Object disconnected from clients — OPC server dropped us (crash/restart) |
|
||||
| `0x80070005` | Access denied — DCOM security misconfiguration |
|
||||
| `0x80040154` | Class not registered — wrong ProgID, or wrapper/server registered under a different bitness |
|
||||
| `0xC0040007` | Invalid item ID — tag name wrong in config (surfaced by `probe`) |
|
||||
|
||||
**Bitness warning:** the OPC DA Automation Wrapper (`OPCAutomation.dll`) and the Graybox
|
||||
wrapper are typically **32-bit in-proc DLLs** — the Python interpreter must match
|
||||
(32-bit). This is decided and verified in Phase 0 of the implementation plan.
|
||||
|
||||
## Safety
|
||||
|
||||
- The tool never writes to the OPC server — there is no write path in the code.
|
||||
- It uses one OPC group and a static, small tag list at a modest poll rate (default 1 s)
|
||||
so it does not load the system it is measuring.
|
||||
|
||||
## Development setup
|
||||
|
||||
The OPC DA wrappers (`OPCAutomation.dll`, `GBDaCtl.dll`) are **32-bit in-proc COM DLLs**,
|
||||
so the Python interpreter **must be 32-bit** (SPECIFICATION §11 / Phase 0 decision).
|
||||
|
||||
```powershell
|
||||
# create the 32-bit venv (requires the 32-bit Python installed via the py launcher)
|
||||
& "C:\Users\<user>\AppData\Local\Programs\Python\Python314-32\python.exe" -m venv .venv
|
||||
& ".\.venv\Scripts\python.exe" -m pip install -e ".[dev]"
|
||||
|
||||
# verify bitness — must print 32
|
||||
& ".\.venv\Scripts\python.exe" -c "import struct; print(struct.calcsize('P') * 8)"
|
||||
|
||||
# run the (COM-free) unit + fake-integration tests
|
||||
& ".\.venv\Scripts\python.exe" -m pytest
|
||||
```
|
||||
|
||||
Notes:
|
||||
- Dependencies are chosen to be installable without a C compiler: **ruamel.yaml**
|
||||
(pure Python) is used instead of PyYAML, which publishes no 32-bit CPython 3.14
|
||||
wheels and whose sdist requires MSVC.
|
||||
- COM-dependent modules (Automation Wrapper, Event Log, service) are import-guarded;
|
||||
the unit test suite runs on any OS via the in-process fake client
|
||||
(`opcda_diagnozer.opc.fake`).
|
||||
- 32-bit CPython can be installed with the Python Install Manager
|
||||
(`py install 3.14 -a x86`) or the installer from python.org ("Download Windows
|
||||
installer (32-bit)").
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Specification](docs/SPECIFICATION.md) — detection rules, event model, operational behavior, acceptance criteria
|
||||
- [Implementation plan](docs/IMPLEMENTATION_PLAN.md) — phases, milestones, testing, risks
|
||||
|
||||
## License
|
||||
|
||||
TBD
|
||||
Reference in New Issue
Block a user