🚧 WIP
This commit is contained in:
Jonas Bengtsson
+182
-57
@@ -2,7 +2,7 @@
|
||||
|
||||
## SCADA Unattended Change Tracker
|
||||
|
||||
### 1. Purpose
|
||||
# 1. Purpose
|
||||
|
||||
The purpose of the application is to provide automated version tracking of configuration and project files associated with a SCADA system.
|
||||
|
||||
@@ -18,6 +18,14 @@ The resulting Git history shall provide an auditable record showing:
|
||||
|
||||
The application is intended to operate without normal user interaction and shall be capable of running on Windows, Linux and macOS.
|
||||
|
||||
## 1.1 Conformance language
|
||||
|
||||
In this specification:
|
||||
|
||||
* **shall** – the requirement is mandatory;
|
||||
* **should** – the requirement is recommended, but deviations are acceptable with documented justification;
|
||||
* **may** – the behaviour is optional.
|
||||
|
||||
---
|
||||
|
||||
# 2. Scope
|
||||
@@ -64,6 +72,7 @@ The application should be divided into independent functional components.
|
||||
v
|
||||
+-------------------------+
|
||||
| Application Controller |
|
||||
| (config, lock, exit) |
|
||||
+------------+------------+
|
||||
|
|
||||
v
|
||||
@@ -76,17 +85,17 @@ The application should be divided into independent functional components.
|
||||
| Change Detector |
|
||||
+------------+------------+
|
||||
|
|
||||
+-----+------+
|
||||
| |
|
||||
v v
|
||||
+-------------+ +----------------+
|
||||
| File Change | | Author Resolver|
|
||||
| Information | | |
|
||||
+------+------+ +-------+--------+
|
||||
| |
|
||||
+-------+--------+
|
||||
|
|
||||
v
|
||||
v
|
||||
+-------------------------+
|
||||
| Security Validator |
|
||||
+------------+------------+
|
||||
|
|
||||
v
|
||||
+-------------------------+
|
||||
| Author Resolver (chain) |
|
||||
+------------+------------+
|
||||
|
|
||||
v
|
||||
+-------------------------+
|
||||
| Commit Message Builder |
|
||||
+------------+------------+
|
||||
@@ -100,6 +109,9 @@ The application should be divided into independent functional components.
|
||||
+-------------------------+
|
||||
| Logging / Audit |
|
||||
+-------------------------+
|
||||
|
||||
Logging / Audit is a cross-cutting concern: every component reports to it
|
||||
directly rather than only through the final stage.
|
||||
```
|
||||
|
||||
The components should be loosely coupled so that SCADA-specific metadata extraction can be implemented independently of Git handling.
|
||||
@@ -115,7 +127,7 @@ The application shall support at least two operating modes.
|
||||
Example:
|
||||
|
||||
```text
|
||||
scada-change-tracker run
|
||||
git-auto-tracker run
|
||||
```
|
||||
|
||||
The application performs one complete inspection and exits.
|
||||
@@ -151,10 +163,11 @@ schedule:
|
||||
|
||||
The application should use local system time unless a timezone is explicitly configured.
|
||||
|
||||
Example:
|
||||
Example (see also Section 26):
|
||||
|
||||
```yaml
|
||||
timezone: Europe/Stockholm
|
||||
application:
|
||||
timezone: Europe/Stockholm
|
||||
```
|
||||
|
||||
---
|
||||
@@ -183,6 +196,17 @@ Before attempting any modification, the application shall verify:
|
||||
|
||||
If validation fails, the execution shall stop without modifying the repository.
|
||||
|
||||
## 5.1 Repository and index state
|
||||
|
||||
Before collecting changes the application shall additionally verify that:
|
||||
|
||||
* the repository index does not already contain staged changes;
|
||||
* the repository is not in an unfinished merge, rebase or cherry-pick state.
|
||||
|
||||
A non-empty index at startup indicates that a human or another process prepared changes outside the tracker. Because such changes carry no tracker attribution and may be incomplete, the default policy shall be to abort with a repository error (exit code 3, see Section 25). A configurable policy may allow such changes to be committed with `UNKNOWN` attribution. Staged changes left behind by an interrupted tracker execution are not special-cased; recovery requires manual review, which keeps unattended behaviour safe and predictable.
|
||||
|
||||
If the current branch differs from the configured branch, the application shall abort by default rather than switch branches, since switching modifies the working tree and is not an unattended-safe operation.
|
||||
|
||||
---
|
||||
|
||||
# 6. Change Detection
|
||||
@@ -202,6 +226,8 @@ The following change types shall be supported:
|
||||
|
||||
Rename detection may initially be treated as one deletion and one addition if native rename detection is not implemented.
|
||||
|
||||
Git does not report renames for unstaged working-tree changes; rename detection is only reliable after staging. The implementation should therefore either stage first and inspect the staged diff, or deliberately report renames as one deletion plus one addition.
|
||||
|
||||
---
|
||||
|
||||
## 6.2 Git status
|
||||
@@ -211,13 +237,15 @@ The application should use Git itself as the authoritative mechanism for determi
|
||||
The implementation should therefore use an equivalent of:
|
||||
|
||||
```text
|
||||
git status --porcelain
|
||||
git status --porcelain -z
|
||||
```
|
||||
|
||||
rather than independently comparing file timestamps.
|
||||
|
||||
This avoids false change detection caused by modification times alone.
|
||||
|
||||
The NUL-terminated (`-z`) output format shall be used for all Git output that contains file paths. Line-based parsing of the default porcelain output is unsafe because Git C-quotes filenames containing spaces, quotes or non-ASCII characters. All Git output shall be decoded as UTF-8, and commit messages shall be written as UTF-8.
|
||||
|
||||
---
|
||||
|
||||
# 7. File Inclusion and Exclusion
|
||||
@@ -244,6 +272,8 @@ Glob-style matching should be supported.
|
||||
|
||||
Files excluded by configuration shall neither be committed nor included in commit messages.
|
||||
|
||||
Files ignored by the repository's own `.gitignore` shall also be excluded. The tracker shall not override ignore rules unless explicitly configured to do so.
|
||||
|
||||
---
|
||||
|
||||
# 8. Last Modifier Identification
|
||||
@@ -265,7 +295,7 @@ Modified:
|
||||
Last modified by: DOMAIN\jsmith
|
||||
```
|
||||
|
||||
Deleted files do not require a last-modifier lookup because the original file may no longer be available.
|
||||
Deleted files do not require a last-modifier lookup because the original file may no longer be available. Resolvers may nevertheless attribute deletions when possible, for example through the SCADA audit log or the file's previous commit history. Attribution for deletions is permitted but not mandatory.
|
||||
|
||||
---
|
||||
|
||||
@@ -303,6 +333,15 @@ Example:
|
||||
}
|
||||
```
|
||||
|
||||
The `confidence` field shall use one of the following levels:
|
||||
|
||||
| Level | Meaning |
|
||||
| --------------- | ------------------------------------------------------------- |
|
||||
| `authoritative` | The source records the change actor in an authenticated way |
|
||||
| `probable` | The source is reliable but not an authenticated change record |
|
||||
| `weak` | Weak correlation, e.g. ownership or workstation inference |
|
||||
| `unknown` | No identity could be determined |
|
||||
|
||||
---
|
||||
|
||||
# 10. Author Resolver Strategies
|
||||
@@ -390,12 +429,13 @@ author_resolution:
|
||||
- scada_audit_log
|
||||
- sidecar_metadata
|
||||
|
||||
minimum_confidence: probable
|
||||
fallback: unknown
|
||||
```
|
||||
|
||||
The resolvers shall execute in configured priority order.
|
||||
|
||||
The first authoritative result shall be used.
|
||||
The first result whose confidence meets the configured minimum confidence (default: `probable`) shall be used. Results below the threshold shall be logged but not used. If no resolver produces a usable result, the configured fallback shall be used.
|
||||
|
||||
---
|
||||
|
||||
@@ -470,6 +510,16 @@ git add --all
|
||||
|
||||
unless the repository is exclusively dedicated to tracked SCADA data.
|
||||
|
||||
Staging shall instead use pathspec-limited operations that stage additions, modifications and deletions for the selected paths only.
|
||||
|
||||
Step 5 (verification) shall compare the actual staged change set against the expected change set using an operation equivalent to:
|
||||
|
||||
```text
|
||||
git diff --cached --name-status -z
|
||||
```
|
||||
|
||||
If the staged change set does not match the expected change set, the commit shall be aborted and the index restored to its pre-execution state, so that the next execution starts from a well-defined state.
|
||||
|
||||
---
|
||||
|
||||
# 15. Commit Message Format
|
||||
@@ -530,7 +580,7 @@ Alternative:
|
||||
SCADA: 2 added, 3 modified, 1 deleted
|
||||
```
|
||||
|
||||
The detailed information shall be placed in the commit body.
|
||||
Exactly one subject format shall be used consistently; the timestamp format shall be the default. The detailed information shall be placed in the commit body.
|
||||
|
||||
---
|
||||
|
||||
@@ -628,10 +678,12 @@ Only one tracker execution shall be allowed against a repository at the same tim
|
||||
|
||||
The application shall implement a repository-level execution lock.
|
||||
|
||||
The lock file shall be stored **outside** the tracked repository, for example as a sibling of the repository directory or in a configured state directory. A lock file placed inside the repository would itself appear as an untracked change and be committed by the tracker.
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
.scada-change-tracker.lock
|
||||
<repository-path>.git-auto-tracker.lock
|
||||
```
|
||||
|
||||
The lock shall include information such as:
|
||||
@@ -642,7 +694,7 @@ hostname
|
||||
start_time
|
||||
```
|
||||
|
||||
Stale locks shall be detected safely.
|
||||
Stale locks shall be detected safely. A lock may be considered stale when the recorded process no longer exists (verifiable only on the same host) or when the lock is older than a configurable age threshold, which is required for locks left by other hosts. Removing a stale lock shall be logged as a warning.
|
||||
|
||||
---
|
||||
|
||||
@@ -687,13 +739,15 @@ Example:
|
||||
|
||||
```yaml
|
||||
logging:
|
||||
file: "logs/scada-change-tracker.log"
|
||||
file: "logs/git-auto-tracker.log"
|
||||
max_size_mb: 20
|
||||
backup_count: 10
|
||||
```
|
||||
|
||||
Console logging shall also be supported.
|
||||
|
||||
Relative log paths shall be resolved relative to the configuration file location. The log file shall not be placed inside the tracked repository unless explicitly excluded; otherwise the tracker's own log would be detected as a change and committed.
|
||||
|
||||
---
|
||||
|
||||
# 24. Error Handling
|
||||
@@ -709,6 +763,7 @@ FILE_ACCESS_ERROR
|
||||
AUTHOR_RESOLUTION_ERROR
|
||||
GIT_ERROR
|
||||
LOCK_ERROR
|
||||
SECURITY_ERROR
|
||||
INTERNAL_ERROR
|
||||
```
|
||||
|
||||
@@ -723,6 +778,8 @@ Reason: Audit log unavailable
|
||||
|
||||
A Git or repository integrity error shall prevent the commit.
|
||||
|
||||
A security violation (see Section 33) shall prevent the commit and shall be reported through the `SECURITY_ERROR` category and the dedicated exit code.
|
||||
|
||||
---
|
||||
|
||||
# 25. Exit Codes
|
||||
@@ -738,6 +795,9 @@ Suggested process exit codes:
|
||||
| 4 | Git operation failure |
|
||||
| 5 | Repository locked |
|
||||
| 6 | Change collection failed |
|
||||
| 7 | Security violation |
|
||||
|
||||
Exit code `0` shall be returned both when changes were committed and when no changes were detected; the execution log distinguishes the two cases.
|
||||
|
||||
This makes the application suitable for external monitoring.
|
||||
|
||||
@@ -759,7 +819,7 @@ repository:
|
||||
|
||||
git:
|
||||
user_name: "SCADA Change Tracker"
|
||||
user_email: "scada-change-tracker@localhost"
|
||||
user_email: "git-auto-tracker@localhost"
|
||||
|
||||
files:
|
||||
include:
|
||||
@@ -775,18 +835,28 @@ author_resolution:
|
||||
- scada_metadata
|
||||
- scada_audit_log
|
||||
|
||||
minimum_confidence: probable
|
||||
fallback: unknown
|
||||
|
||||
commit:
|
||||
create_when_no_changes: false
|
||||
|
||||
logging:
|
||||
level: INFO
|
||||
console: true
|
||||
file: "logs/scada-change-tracker.log"
|
||||
file: "logs/git-auto-tracker.log"
|
||||
```
|
||||
|
||||
Environment variables should be capable of overriding sensitive or deployment-specific settings.
|
||||
Environment variables should be capable of overriding sensitive or deployment-specific settings, using a consistent naming convention such as:
|
||||
|
||||
```text
|
||||
GIT_AUTO_TRACKER_<SECTION>__<KEY>
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
GIT_AUTO_TRACKER_REPOSITORY__PATH=D:/SCADA/Project
|
||||
```
|
||||
|
||||
The prohibition of empty commits (FR-014) is unconditional and therefore deliberately not configurable.
|
||||
|
||||
---
|
||||
|
||||
@@ -845,7 +915,7 @@ The Git integration may use either:
|
||||
|
||||
Using the Git CLI is recommended initially because it closely follows native Git behaviour and makes troubleshooting straightforward.
|
||||
|
||||
The Git executable shall therefore be treated as an external prerequisite.
|
||||
The Git executable shall therefore be treated as an external prerequisite. A minimum Git version of 2.20 shall be assumed.
|
||||
|
||||
At startup the application shall verify:
|
||||
|
||||
@@ -874,10 +944,18 @@ The application should avoid platform-specific Python dependencies unless they a
|
||||
Suggested commands:
|
||||
|
||||
```text
|
||||
scada-change-tracker run
|
||||
scada-change-tracker status
|
||||
scada-change-tracker validate
|
||||
scada-change-tracker config
|
||||
git-auto-tracker run
|
||||
git-auto-tracker status
|
||||
git-auto-tracker validate
|
||||
git-auto-tracker config
|
||||
```
|
||||
|
||||
Global options shall include:
|
||||
|
||||
```text
|
||||
--config PATH Path to the configuration file (default: git-auto-tracker.yaml)
|
||||
--log-level LEVEL Override the configured log level
|
||||
--version Print the application version and exit
|
||||
```
|
||||
|
||||
### `run`
|
||||
@@ -909,7 +987,7 @@ Display the effective configuration with secrets redacted.
|
||||
The application shall support:
|
||||
|
||||
```text
|
||||
scada-change-tracker run --dry-run
|
||||
git-auto-tracker run --dry-run
|
||||
```
|
||||
|
||||
Dry-run mode shall perform:
|
||||
@@ -971,7 +1049,7 @@ security:
|
||||
- "**/credentials.*"
|
||||
```
|
||||
|
||||
If such a file is detected, the default behaviour should be to abort the commit and log a security warning.
|
||||
If such a file is detected, the default behaviour shall be to abort the commit, log a security warning, and exit with the dedicated security exit code (see Section 25). A configurable alternative may skip the forbidden file and commit the remaining changes, but skipping shall never be the default.
|
||||
|
||||
---
|
||||
|
||||
@@ -1001,7 +1079,7 @@ PUSH_FAILURE
|
||||
|
||||
# 35. Audit Information
|
||||
|
||||
Each collection should internally generate an execution identifier.
|
||||
Each collection should internally generate an execution identifier. A ULID, or an equivalent lexicographically sortable and timezone-independent identifier, is recommended.
|
||||
|
||||
For example:
|
||||
|
||||
@@ -1037,7 +1115,7 @@ Tracker-Host: SCADA-SRV-01
|
||||
Tracker-Changes: 6
|
||||
```
|
||||
|
||||
This enables later processing without parsing the human-readable sections.
|
||||
This enables later processing without parsing the human-readable sections. The trailer block shall follow the Git trailer convention so that standard tooling such as `git interpret-trailers` can extract it.
|
||||
|
||||
---
|
||||
|
||||
@@ -1139,12 +1217,32 @@ A normal execution shall produce at most one commit representing the complete de
|
||||
|
||||
Configured forbidden files shall not automatically be committed.
|
||||
|
||||
## FR-025 – Concurrent modification protection
|
||||
|
||||
The system shall detect files modified during collection and abort or retry according to a configured policy. The default policy shall abort the commit.
|
||||
|
||||
## FR-026 – Index-state validation
|
||||
|
||||
The system shall refuse to execute when the repository index already contains staged changes or the repository is in an unfinished merge/rebase state.
|
||||
|
||||
## FR-027 – Security abort
|
||||
|
||||
The system shall abort the commit and exit with a dedicated exit code when a forbidden file is detected.
|
||||
|
||||
## FR-028 – Self-exclusion
|
||||
|
||||
The system shall never commit its own operational artefacts, such as lock files, logs and state files.
|
||||
|
||||
## FR-029 – Filename safety
|
||||
|
||||
The system shall correctly process file paths containing spaces, quotes and non-ASCII characters on all supported platforms.
|
||||
|
||||
---
|
||||
|
||||
# 38. Suggested Python Package Structure
|
||||
|
||||
```text
|
||||
scada-change-tracker/
|
||||
git-auto-tracker/
|
||||
│
|
||||
├── pyproject.toml
|
||||
├── README.md
|
||||
@@ -1154,7 +1252,7 @@ scada-change-tracker/
|
||||
│ └── example.yaml
|
||||
│
|
||||
├── src/
|
||||
│ └── scada_change_tracker/
|
||||
│ └── git_auto_tracker/
|
||||
│ │
|
||||
│ ├── __init__.py
|
||||
│ ├── __main__.py
|
||||
@@ -1232,29 +1330,29 @@ Each item is represented by a `ChangeRecord`.
|
||||
A normal execution shall approximately perform the following sequence:
|
||||
|
||||
```text
|
||||
1. Load configuration
|
||||
2. Validate configuration
|
||||
3. Acquire repository lock
|
||||
4. Verify Git availability
|
||||
5. Validate repository
|
||||
6. Determine current repository status
|
||||
7. Apply inclusion/exclusion rules
|
||||
8. Create ChangeSet
|
||||
9. Resolve authors for ADDED files
|
||||
1. Load configuration
|
||||
2. Validate configuration
|
||||
3. Verify Git availability
|
||||
4. Acquire repository lock
|
||||
5. Validate repository, branch and index state
|
||||
6. Determine current repository status
|
||||
7. Apply inclusion/exclusion and security rules
|
||||
8. Create ChangeSet
|
||||
9. Resolve authors for ADDED files
|
||||
10. Resolve authors for MODIFIED files
|
||||
11. Validate files against security rules
|
||||
12. Generate commit message
|
||||
13. Stage selected changes
|
||||
14. Verify staged changes
|
||||
15. Create commit
|
||||
16. Record commit hash
|
||||
17. Write audit result
|
||||
18. Release lock
|
||||
19. Exit
|
||||
11. Generate commit message
|
||||
12. Stage selected changes
|
||||
13. Verify staged change set
|
||||
14. Create commit
|
||||
15. Record commit hash and write audit result
|
||||
16. Release lock
|
||||
17. Exit
|
||||
```
|
||||
|
||||
If no changes exist, execution proceeds directly from step 8 to audit logging and termination.
|
||||
|
||||
If any step from staging onwards fails, the index shall be restored to its pre-execution state before the lock is released.
|
||||
|
||||
---
|
||||
|
||||
# 41. Testing Requirements
|
||||
@@ -1295,6 +1393,16 @@ Tests should cover at minimum:
|
||||
* excluded file;
|
||||
* forbidden file.
|
||||
|
||||
### Safety and edge cases
|
||||
|
||||
* pre-existing staged changes in the index;
|
||||
* repository in an unfinished merge/rebase state;
|
||||
* current branch differs from the configured branch;
|
||||
* forbidden-file abort produces the dedicated exit code;
|
||||
* stale lock left by a dead process is recovered;
|
||||
* lock file and log file never appear in `git status`;
|
||||
* configured timezone is honoured in commit timestamps.
|
||||
|
||||
### Cross-platform
|
||||
|
||||
Automated testing should run against at least:
|
||||
@@ -1382,3 +1490,20 @@ Version 1.0 can be considered complete when it can:
|
||||
15. execute unattended through an external scheduler.
|
||||
|
||||
The initial implementation should deliberately keep **SCADA-specific change attribution behind the `AuthorResolver` interface**. That makes the generic Git/change-tracking engine reusable while allowing a resolver tailored to the actual SCADA platform to be added independently.
|
||||
|
||||
---
|
||||
|
||||
# Appendix A – Glossary
|
||||
|
||||
| Term | Definition |
|
||||
| ------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| Collection | One complete detection-and-commit cycle performed by the tracker |
|
||||
| ChangeSet | The complete set of changes detected during one collection |
|
||||
| ChangeRecord | A single detected change together with its metadata |
|
||||
| AuthorResolver | Pluggable component attempting to determine the last modifying identity of a file |
|
||||
| AuthorResult | The outcome of one author-resolution attempt, including source and confidence |
|
||||
| Confidence | Trustworthiness of an author result: `authoritative`, `probable`, `weak` or `unknown` |
|
||||
| Fallback | Result used when no resolver produces a usable result (default: `UNKNOWN`) |
|
||||
| Forbidden file | A file matching `security.forbidden_patterns`; never committed by default |
|
||||
| Tracker-Run | Unique execution identifier linking logs, commits and monitoring (Section 35) |
|
||||
| Staged verification | Comparison of the actual staged change set against the expected change set before commit |
|
||||
|
||||
Reference in New Issue
Block a user