1644 lines
44 KiB
Markdown
1644 lines
44 KiB
Markdown
# Functional Specification
|
||
|
||
## SCADA Unattended Change Tracker
|
||
|
||
# 1. Purpose
|
||
|
||
The purpose of the application is to provide automated version tracking of configuration and project files associated with a SCADA system.
|
||
|
||
The application shall periodically inspect a configured set of files or directories for changes and automatically record detected changes in a Git repository.
|
||
|
||
The resulting Git history shall provide an auditable record showing:
|
||
|
||
* when changes were detected,
|
||
* which files were added,
|
||
* which files were modified,
|
||
* which files were removed,
|
||
* and, where applicable, who last modified each added or modified file.
|
||
|
||
The application is intended to operate without normal user interaction and shall be capable of running on Windows, Linux and macOS.
|
||
|
||
## 1.1 Conformance language
|
||
|
||
In this specification:
|
||
|
||
* **shall** – the requirement is mandatory;
|
||
* **should** – the requirement is recommended, but deviations are acceptable with documented justification;
|
||
* **may** – the behaviour is optional.
|
||
|
||
---
|
||
|
||
# 2. Scope
|
||
|
||
The application shall:
|
||
|
||
1. monitor one or more configured SCADA project directories;
|
||
2. identify file-system changes relative to the current Git repository state;
|
||
3. determine whether files have been:
|
||
|
||
* added,
|
||
* modified,
|
||
* deleted,
|
||
* optionally renamed;
|
||
4. determine the last modifying user for added and modified files;
|
||
5. generate a structured Git commit message describing the detected changes;
|
||
6. stage the changes;
|
||
7. create a Git commit automatically;
|
||
8. maintain logs of application execution and errors;
|
||
9. support unattended scheduled execution;
|
||
10. operate consistently across supported operating systems.
|
||
|
||
The initial implementation does not need to:
|
||
|
||
* interpret the semantic meaning of SCADA configuration changes;
|
||
* restore files;
|
||
* approve or reject changes;
|
||
* provide a graphical user interface;
|
||
* automatically push commits to a remote Git repository.
|
||
|
||
These functions may be added later.
|
||
|
||
---
|
||
|
||
# 3. General Architecture
|
||
|
||
The application should be divided into independent functional components.
|
||
|
||
```text
|
||
+-------------------------+
|
||
| Scheduler / Invocation |
|
||
+------------+------------+
|
||
|
|
||
v
|
||
+-------------------------+
|
||
| Application Controller |
|
||
| (config, lock, exit) |
|
||
+------------+------------+
|
||
|
|
||
v
|
||
+-------------------------+
|
||
| Repository Inspector |
|
||
+------------+------------+
|
||
|
|
||
v
|
||
+-------------------------+
|
||
| Artifact Extractor |
|
||
| (Section 45, optional) |
|
||
+------------+------------+
|
||
|
|
||
v
|
||
+-------------------------+
|
||
| Change Detector |
|
||
+------------+------------+
|
||
|
|
||
v
|
||
+-------------------------+
|
||
| Security Validator |
|
||
+------------+------------+
|
||
|
|
||
v
|
||
+-------------------------+
|
||
| Author Resolver (chain) |
|
||
+------------+------------+
|
||
|
|
||
v
|
||
+-------------------------+
|
||
| Commit Message Builder |
|
||
+------------+------------+
|
||
|
|
||
v
|
||
+-------------------------+
|
||
| Git Commit Manager |
|
||
+------------+------------+
|
||
|
|
||
v
|
||
+-------------------------+
|
||
| Logging / Audit |
|
||
+-------------------------+
|
||
|
||
Logging / Audit is a cross-cutting concern: every component reports to it
|
||
directly rather than only through the final stage.
|
||
```
|
||
|
||
The components should be loosely coupled so that SCADA-specific metadata extraction can be implemented independently of Git handling.
|
||
|
||
---
|
||
|
||
# 4. Operating Modes
|
||
|
||
The application shall support at least two operating modes.
|
||
|
||
## 4.1 Run-once mode
|
||
|
||
Example:
|
||
|
||
```text
|
||
git-auto-tracker run
|
||
```
|
||
|
||
The application performs one complete inspection and exits.
|
||
|
||
This mode is suitable for execution through:
|
||
|
||
* Windows Task Scheduler,
|
||
* cron,
|
||
* systemd timers,
|
||
* CI/CD systems,
|
||
* Docker scheduled execution,
|
||
* other external schedulers.
|
||
|
||
Run-once mode should be the primary operating mode because it simplifies reliability and cross-platform deployment.
|
||
|
||
---
|
||
|
||
## 4.2 Scheduled service mode
|
||
|
||
An optional internal scheduler may allow the application to remain running and execute according to configured times.
|
||
|
||
Example configuration:
|
||
|
||
```yaml
|
||
schedule:
|
||
enabled: true
|
||
times:
|
||
- "06:00"
|
||
- "12:00"
|
||
- "18:00"
|
||
- "23:30"
|
||
```
|
||
|
||
The application should use local system time unless a timezone is explicitly configured.
|
||
|
||
Example (see also Section 26):
|
||
|
||
```yaml
|
||
application:
|
||
timezone: Europe/Stockholm
|
||
```
|
||
|
||
---
|
||
|
||
# 5. Repository Configuration
|
||
|
||
The application shall operate against an existing Git repository.
|
||
|
||
Example:
|
||
|
||
```yaml
|
||
repository:
|
||
path: "/path/to/scada/project"
|
||
branch: "main"
|
||
```
|
||
|
||
The configured directory shall contain a valid Git repository.
|
||
|
||
Before attempting any modification, the application shall verify:
|
||
|
||
* the directory exists;
|
||
* the directory is accessible;
|
||
* the directory is a Git repository;
|
||
* the configured branch can be determined;
|
||
* Git operations can be performed.
|
||
|
||
If validation fails, the execution shall stop without modifying the repository.
|
||
|
||
## 5.1 Repository and index state
|
||
|
||
Before collecting changes the application shall additionally verify that:
|
||
|
||
* the repository index does not already contain staged changes;
|
||
* the repository is not in an unfinished merge, rebase or cherry-pick state.
|
||
|
||
A non-empty index at startup indicates that a human or another process prepared changes outside the tracker. Because such changes carry no tracker attribution and may be incomplete, the default policy shall be to abort with a repository error (exit code 3, see Section 25). A configurable policy may allow such changes to be committed with `UNKNOWN` attribution. Staged changes left behind by an interrupted tracker execution are not special-cased; recovery requires manual review, which keeps unattended behaviour safe and predictable.
|
||
|
||
If the current branch differs from the configured branch, the application shall abort by default rather than switch branches, since switching modifies the working tree and is not an unattended-safe operation.
|
||
|
||
---
|
||
|
||
# 6. Change Detection
|
||
|
||
## 6.1 Change comparison
|
||
|
||
Changes shall be detected relative to the current Git HEAD.
|
||
|
||
The following change types shall be supported:
|
||
|
||
| Change | Identifier |
|
||
| ------------- | ---------- |
|
||
| Added file | `ADDED` |
|
||
| Modified file | `MODIFIED` |
|
||
| Deleted file | `DELETED` |
|
||
| Renamed file | `RENAMED` |
|
||
|
||
Rename detection may initially be treated as one deletion and one addition if native rename detection is not implemented.
|
||
|
||
Git does not report renames for unstaged working-tree changes; rename detection is only reliable after staging. The implementation should therefore either stage first and inspect the staged diff, or deliberately report renames as one deletion plus one addition.
|
||
|
||
---
|
||
|
||
## 6.2 Git status
|
||
|
||
The application should use Git itself as the authoritative mechanism for determining changed files.
|
||
|
||
The implementation should therefore use an equivalent of:
|
||
|
||
```text
|
||
git status --porcelain -z
|
||
```
|
||
|
||
rather than independently comparing file timestamps.
|
||
|
||
This avoids false change detection caused by modification times alone.
|
||
|
||
The NUL-terminated (`-z`) output format shall be used for all Git output that contains file paths. Line-based parsing of the default porcelain output is unsafe because Git C-quotes filenames containing spaces, quotes or non-ASCII characters. All Git output shall be decoded as UTF-8, and commit messages shall be written as UTF-8.
|
||
|
||
---
|
||
|
||
# 7. File Inclusion and Exclusion
|
||
|
||
The application shall support configurable inclusion and exclusion rules.
|
||
|
||
Example:
|
||
|
||
```yaml
|
||
files:
|
||
include:
|
||
- "**/*"
|
||
|
||
exclude:
|
||
- "**/*.tmp"
|
||
- "**/*.log"
|
||
- "**/~*"
|
||
- ".git/**"
|
||
- "runtime/**"
|
||
- "cache/**"
|
||
```
|
||
|
||
Glob-style matching should be supported.
|
||
|
||
Files excluded by configuration shall neither be committed nor included in commit messages.
|
||
|
||
Files ignored by the repository's own `.gitignore` shall also be excluded. The tracker shall not override ignore rules unless explicitly configured to do so.
|
||
|
||
---
|
||
|
||
# 8. Last Modifier Identification
|
||
|
||
## 8.1 Requirement
|
||
|
||
For every:
|
||
|
||
* added file;
|
||
* modified file;
|
||
|
||
the application shall attempt to determine the person or account responsible for its most recent modification.
|
||
|
||
Example:
|
||
|
||
```text
|
||
Modified:
|
||
configuration/plant1.xml
|
||
Last modified by: DOMAIN\jsmith
|
||
```
|
||
|
||
Deleted files do not require a last-modifier lookup because the original file may no longer be available. Resolvers may nevertheless attribute deletions when possible, for example through the SCADA audit log or the file's previous commit history. Attribution for deletions is permitted but not mandatory.
|
||
|
||
---
|
||
|
||
# 9. Author Resolution
|
||
|
||
Determining the last modifying user shall be implemented as a separate abstraction.
|
||
|
||
Conceptually:
|
||
|
||
```python
|
||
class AuthorResolver:
|
||
def resolve(file_path) -> AuthorResult:
|
||
...
|
||
```
|
||
|
||
A result should contain at least:
|
||
|
||
```text
|
||
name
|
||
identifier
|
||
source
|
||
confidence
|
||
timestamp
|
||
```
|
||
|
||
Example:
|
||
|
||
```json
|
||
{
|
||
"name": "John Smith",
|
||
"identifier": "DOMAIN\\jsmith",
|
||
"source": "scada_metadata",
|
||
"confidence": "authoritative",
|
||
"timestamp": "2026-09-11T14:31:22+02:00"
|
||
}
|
||
```
|
||
|
||
The `confidence` field shall use one of the following levels:
|
||
|
||
| Level | Meaning |
|
||
| --------------- | ------------------------------------------------------------- |
|
||
| `authoritative` | The source records the change actor in an authenticated way |
|
||
| `probable` | The source is reliable but not an authenticated change record |
|
||
| `weak` | Weak correlation, e.g. ownership or workstation inference |
|
||
| `unknown` | No identity could be determined |
|
||
|
||
---
|
||
|
||
# 10. Author Resolver Strategies
|
||
|
||
The application shall support configurable resolver implementations.
|
||
|
||
Suggested resolver types are:
|
||
|
||
### SCADA metadata resolver
|
||
|
||
Reads user information embedded in the SCADA project file.
|
||
|
||
This should be preferred when the SCADA platform records information such as:
|
||
|
||
* last modified by;
|
||
* engineering user;
|
||
* operator;
|
||
* change timestamp;
|
||
* workstation;
|
||
* project revision.
|
||
|
||
---
|
||
|
||
### Companion metadata resolver
|
||
|
||
Reads metadata associated with the file.
|
||
|
||
For example:
|
||
|
||
```text
|
||
pumpstation.xml
|
||
pumpstation.xml.metadata
|
||
```
|
||
|
||
or from a project database.
|
||
|
||
---
|
||
|
||
### SCADA audit-log resolver
|
||
|
||
Looks up the changed file or object in the SCADA system's audit log.
|
||
|
||
This is likely to provide the strongest attribution where the SCADA system maintains authenticated engineering change records.
|
||
|
||
---
|
||
|
||
### Operating-system resolver
|
||
|
||
An OS-specific resolver may be implemented if the environment provides authoritative file auditing.
|
||
|
||
File ownership alone shall **not** be interpreted as proof of the last modifying user.
|
||
|
||
For example, on Windows:
|
||
|
||
```text
|
||
DOMAIN\SCADAService
|
||
```
|
||
|
||
could own the file even though another engineering user performed the modification.
|
||
|
||
---
|
||
|
||
### Unknown resolver
|
||
|
||
If no reliable identity can be determined:
|
||
|
||
```text
|
||
Last modified by: UNKNOWN
|
||
```
|
||
|
||
The application shall still create the commit unless configured otherwise.
|
||
|
||
---
|
||
|
||
# 11. Resolver Priority
|
||
|
||
Multiple author resolvers may be configured.
|
||
|
||
Example:
|
||
|
||
```yaml
|
||
author_resolution:
|
||
resolvers:
|
||
- scada_metadata
|
||
- scada_audit_log
|
||
- sidecar_metadata
|
||
|
||
minimum_confidence: probable
|
||
fallback: unknown
|
||
```
|
||
|
||
The resolvers shall execute in configured priority order.
|
||
|
||
The first result whose confidence meets the configured minimum confidence (default: `probable`) shall be used. Results below the threshold shall be logged but not used. If no resolver produces a usable result, the configured fallback shall be used.
|
||
|
||
---
|
||
|
||
# 12. Change Data Model
|
||
|
||
Each detected change should internally be represented using a structure similar to:
|
||
|
||
```text
|
||
ChangeRecord
|
||
|
||
path
|
||
change_type
|
||
previous_path
|
||
size
|
||
file_timestamp
|
||
author
|
||
author_source
|
||
author_timestamp
|
||
```
|
||
|
||
For example:
|
||
|
||
```json
|
||
{
|
||
"path": "Stations/PS101/config.xml",
|
||
"change_type": "MODIFIED",
|
||
"previous_path": null,
|
||
"size": 34127,
|
||
"file_timestamp": "2026-09-11T13:44:12+02:00",
|
||
"author": "DOMAIN\\jsmith",
|
||
"author_source": "scada_audit_log",
|
||
"author_timestamp": "2026-09-11T13:43:58+02:00"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
# 13. Commit Generation
|
||
|
||
If no relevant files have changed, the application shall:
|
||
|
||
1. record the execution in its log;
|
||
2. not create an empty Git commit;
|
||
3. return a successful execution result.
|
||
|
||
Example log:
|
||
|
||
```text
|
||
2026-09-11 18:00:02 INFO No SCADA changes detected.
|
||
```
|
||
|
||
---
|
||
|
||
# 14. Git Staging
|
||
|
||
When changes are detected the application shall:
|
||
|
||
1. collect the complete change set;
|
||
2. resolve author information;
|
||
3. generate the commit message;
|
||
4. stage the relevant files;
|
||
5. verify the staged change set;
|
||
6. create the Git commit.
|
||
|
||
Files outside the configured scope shall not be staged.
|
||
|
||
The application should avoid blindly executing:
|
||
|
||
```text
|
||
git add --all
|
||
```
|
||
|
||
unless the repository is exclusively dedicated to tracked SCADA data.
|
||
|
||
Staging shall instead use pathspec-limited operations that stage additions, modifications and deletions for the selected paths only.
|
||
|
||
Step 5 (verification) shall compare the actual staged change set against the expected change set using an operation equivalent to:
|
||
|
||
```text
|
||
git diff --cached --name-status -z
|
||
```
|
||
|
||
If the staged change set does not match the expected change set, the commit shall be aborted and the index restored to its pre-execution state, so that the next execution starts from a well-defined state.
|
||
|
||
---
|
||
|
||
# 15. Commit Message Format
|
||
|
||
Commit messages shall be human-readable while retaining a predictable structure that can later be parsed automatically.
|
||
|
||
Each changed file shall occupy exactly one line, in the style of `git diff --name-status`. A single-letter change type is followed by the file path, the last modifying user and the filesystem last-modified time, with the parts joined by a configurable separator (see Section 26, `commit.file_separator`; default `;`).
|
||
|
||
Line grammar:
|
||
|
||
```text
|
||
A <sep> <path> <sep> <author> <sep> <modified-at> added
|
||
M <sep> <path> <sep> <author> <sep> <modified-at> modified
|
||
R <sep> <old-path> -> <new-path> renamed (bare)
|
||
D <sep> <path> deleted (bare)
|
||
```
|
||
|
||
Recommended format:
|
||
|
||
```text
|
||
SCADA automatic change capture - 2026-09-11 18:00:03+02:00
|
||
|
||
Summary:
|
||
Added: 2
|
||
Modified: 3
|
||
Deleted: 1
|
||
Renamed: 0
|
||
|
||
Changes:
|
||
A ; Stations/PS101/pump2.xml ; DOMAIN\jsmith ; 2026-09-11 15:41:22+02:00
|
||
A ; Stations/PS104/settings.xml ; DOMAIN\adoe ; 2026-09-11 16:02:14+02:00
|
||
M ; Stations/PS101/project.xml ; DOMAIN\jsmith ; 2026-09-11 15:42:09+02:00
|
||
M ; Stations/PS103/alarm.xml ; DOMAIN\adoe ; 2026-09-11 16:13:54+02:00
|
||
M ; System/global.xml ; UNKNOWN ; 2026-09-11 16:20:01+02:00
|
||
D ; Stations/PS027/old-config.xml
|
||
```
|
||
|
||
Deleted files are listed without author or time information (FR-013). Renames use git's `old -> new` arrow notation and stay bare: a pure rename preserves the original modification time, so a timestamp would be misleading. The time column is the filesystem last-modified time (Section 12: `file_timestamp`) rendered in local time with an explicit UTC offset, matching the log timestamp style (Section 22).
|
||
|
||
The one-line format keeps the message proportional to the number of changed files, which matters for large collections.
|
||
|
||
---
|
||
|
||
# 16. Commit Subject
|
||
|
||
The first commit line should be concise.
|
||
|
||
Recommended format:
|
||
|
||
```text
|
||
SCADA automatic change capture - YYYY-MM-DD HH:MM:SS
|
||
```
|
||
|
||
Alternative:
|
||
|
||
```text
|
||
SCADA: 2 added, 3 modified, 1 deleted
|
||
```
|
||
|
||
Exactly one subject format shall be used consistently; the timestamp format shall be the default. The detailed information shall be placed in the commit body.
|
||
|
||
---
|
||
|
||
# 17. Git Commit Identity
|
||
|
||
The Git commit author shall represent the automated tracking system rather than the person who changed an individual file.
|
||
|
||
Example:
|
||
|
||
```yaml
|
||
git:
|
||
user_name: "SCADA Change Tracker"
|
||
user_email: "scada-change-tracker@localhost"
|
||
```
|
||
|
||
The engineers responsible for individual changes are instead recorded within the commit message.
|
||
|
||
This avoids incorrectly assigning a single engineer as the author of a commit containing changes made by several people.
|
||
|
||
---
|
||
|
||
# 18. Multiple Authors
|
||
|
||
One execution may contain changes made by multiple individuals.
|
||
|
||
Example:
|
||
|
||
```text
|
||
Changes:
|
||
M ; plant1.xml ; DOMAIN\jsmith ; 2026-09-11 15:42:09+02:00
|
||
M ; plant2.xml ; DOMAIN\adoe ; 2026-09-11 16:02:14+02:00
|
||
```
|
||
|
||
The application shall therefore store attribution per file rather than per commit.
|
||
|
||
---
|
||
|
||
# 19. Atomic Processing
|
||
|
||
A commit shall represent one complete collection cycle.
|
||
|
||
Conceptually:
|
||
|
||
```text
|
||
Start collection
|
||
↓
|
||
Detect all changes
|
||
↓
|
||
Resolve metadata
|
||
↓
|
||
Build complete change record
|
||
↓
|
||
Stage changes
|
||
↓
|
||
Validate staged state
|
||
↓
|
||
Commit
|
||
↓
|
||
Finish collection
|
||
```
|
||
|
||
The application shall not create multiple commits for one collection unless explicitly configured to do so.
|
||
|
||
---
|
||
|
||
# 20. Concurrent Changes
|
||
|
||
Files may be modified while the collection is taking place.
|
||
|
||
The application should therefore detect this situation.
|
||
|
||
For each file the application may record attributes before and after processing:
|
||
|
||
```text
|
||
size
|
||
modification timestamp
|
||
optional content hash
|
||
```
|
||
|
||
If the file changes while being processed, the application shall either:
|
||
|
||
1. restart analysis for that file; or
|
||
2. abort the complete commit and retry during the next execution.
|
||
|
||
The default behaviour should favour data integrity over forcing a commit.
|
||
|
||
---
|
||
|
||
# 21. Repository Lock
|
||
|
||
Only one tracker execution shall be allowed against a repository at the same time.
|
||
|
||
The application shall implement a repository-level execution lock.
|
||
|
||
The lock file shall be stored **outside** the tracked repository, for example as a sibling of the repository directory or in a configured state directory. A lock file placed inside the repository would itself appear as an untracked change and be committed by the tracker.
|
||
|
||
For example:
|
||
|
||
```text
|
||
<repository-path>.git-auto-tracker.lock
|
||
```
|
||
|
||
The lock shall include information such as:
|
||
|
||
```text
|
||
process_id
|
||
hostname
|
||
start_time
|
||
```
|
||
|
||
Stale locks shall be detected safely. A lock may be considered stale when the recorded process no longer exists (verifiable only on the same host) or when the lock is older than a configurable age threshold, which is required for locks left by other hosts. Removing a stale lock shall be logged as a warning.
|
||
|
||
---
|
||
|
||
# 22. Logging
|
||
|
||
Application activity shall be logged independently of Git.
|
||
|
||
Minimum log information shall include:
|
||
|
||
```text
|
||
execution start
|
||
execution completion
|
||
repository
|
||
number of detected changes
|
||
files processed
|
||
author resolution result
|
||
Git operations
|
||
warnings
|
||
errors
|
||
commit hash
|
||
execution duration
|
||
```
|
||
|
||
Example:
|
||
|
||
```text
|
||
2026-09-11 18:00:01 INFO Starting SCADA change collection
|
||
2026-09-11 18:00:01 INFO Repository: D:\SCADA\Project
|
||
2026-09-11 18:00:02 INFO Detected 6 changes
|
||
2026-09-11 18:00:02 INFO Author resolved: Stations/PS101/project.xml -> DOMAIN\jsmith
|
||
2026-09-11 18:00:03 INFO Commit created: 93ab4e21
|
||
2026-09-11 18:00:03 INFO Collection completed
|
||
```
|
||
|
||
---
|
||
|
||
# 23. Log Rotation
|
||
|
||
File-based logging shall support rotation.
|
||
|
||
Example:
|
||
|
||
```yaml
|
||
logging:
|
||
file: "logs/git-auto-tracker.log"
|
||
max_size_mb: 20
|
||
backup_count: 10
|
||
```
|
||
|
||
Console logging shall also be supported.
|
||
|
||
Relative log paths shall be resolved relative to the configuration file location. The log file shall not be placed inside the tracked repository unless explicitly excluded; otherwise the tracker's own log would be detected as a change and committed.
|
||
|
||
---
|
||
|
||
# 24. Error Handling
|
||
|
||
Errors shall be classified.
|
||
|
||
Recommended categories:
|
||
|
||
```text
|
||
CONFIGURATION_ERROR
|
||
REPOSITORY_ERROR
|
||
FILE_ACCESS_ERROR
|
||
AUTHOR_RESOLUTION_ERROR
|
||
GIT_ERROR
|
||
LOCK_ERROR
|
||
SECURITY_ERROR
|
||
INTERNAL_ERROR
|
||
```
|
||
|
||
An author-resolution error shall normally not prevent a commit.
|
||
|
||
For example:
|
||
|
||
```text
|
||
Author: UNKNOWN
|
||
Reason: Audit log unavailable
|
||
```
|
||
|
||
A Git or repository integrity error shall prevent the commit.
|
||
|
||
A security violation (see Section 33) shall prevent the commit and shall be reported through the `SECURITY_ERROR` category and the dedicated exit code.
|
||
|
||
---
|
||
|
||
# 25. Exit Codes
|
||
|
||
Suggested process exit codes:
|
||
|
||
| Code | Meaning |
|
||
| ---: | --------------------------- |
|
||
| 0 | Successful execution |
|
||
| 1 | General application failure |
|
||
| 2 | Configuration error |
|
||
| 3 | Repository error |
|
||
| 4 | Git operation failure |
|
||
| 5 | Repository locked |
|
||
| 6 | Change collection failed |
|
||
| 7 | Security violation |
|
||
|
||
Exit code `0` shall be returned both when changes were committed and when no changes were detected; the execution log distinguishes the two cases.
|
||
|
||
This makes the application suitable for external monitoring.
|
||
|
||
---
|
||
|
||
# 26. Configuration
|
||
|
||
Configuration should primarily be stored in YAML.
|
||
|
||
Example:
|
||
|
||
```yaml
|
||
application:
|
||
timezone: Europe/Stockholm
|
||
|
||
repository:
|
||
path: "D:/SCADA/Project"
|
||
branch: main
|
||
|
||
git:
|
||
user_name: "SCADA Change Tracker"
|
||
user_email: "git-auto-tracker@localhost"
|
||
|
||
files:
|
||
include:
|
||
- "**/*"
|
||
|
||
exclude:
|
||
- "**/*.tmp"
|
||
- "**/*.log"
|
||
- ".git/**"
|
||
|
||
author_resolution:
|
||
resolvers:
|
||
- scada_metadata
|
||
- scada_audit_log
|
||
|
||
minimum_confidence: probable
|
||
fallback: unknown
|
||
|
||
commit:
|
||
# Separator between the parts of a one-line file entry (Section 15).
|
||
# Must not occur inside paths, author identifiers or timestamps.
|
||
file_separator: ";"
|
||
|
||
extraction:
|
||
# Derived artifacts (Section 45): extract configured binary databases
|
||
# into plain-text artifacts before change detection. Absent block or
|
||
# enabled: false disables the feature entirely.
|
||
enabled: true
|
||
output_root: "extracted" # inside the repository
|
||
format: "jsonl" # one JSON object per row
|
||
sources:
|
||
- path: "history/alarms.sqlite" # repository-relative source
|
||
tables: # optional; default: all user tables
|
||
- "alarms"
|
||
- "events"
|
||
|
||
logging:
|
||
level: INFO
|
||
console: true
|
||
file: "logs/git-auto-tracker.log"
|
||
```
|
||
|
||
Environment variables should be capable of overriding sensitive or deployment-specific settings, using a consistent naming convention such as:
|
||
|
||
```text
|
||
GIT_AUTO_TRACKER_<SECTION>__<KEY>
|
||
```
|
||
|
||
Example:
|
||
|
||
```text
|
||
GIT_AUTO_TRACKER_REPOSITORY__PATH=D:/SCADA/Project
|
||
```
|
||
|
||
The prohibition of empty commits (FR-014) is unconditional and therefore deliberately not configurable.
|
||
|
||
---
|
||
|
||
# 27. Cross-Platform Requirements
|
||
|
||
The project shall support:
|
||
|
||
* Windows,
|
||
* Linux,
|
||
* macOS.
|
||
|
||
The implementation shall therefore avoid assumptions about:
|
||
|
||
* path separators;
|
||
* shell syntax;
|
||
* drive letters;
|
||
* filesystem ownership models;
|
||
* username formats;
|
||
* case sensitivity.
|
||
|
||
Python's:
|
||
|
||
```text
|
||
pathlib
|
||
```
|
||
|
||
shall be preferred for filesystem paths.
|
||
|
||
Shell command strings should not be constructed manually where avoidable.
|
||
|
||
For subprocess execution, argument arrays should be used.
|
||
|
||
Conceptually:
|
||
|
||
```python
|
||
subprocess.run(
|
||
["git", "status", "--porcelain"],
|
||
...
|
||
)
|
||
```
|
||
|
||
rather than:
|
||
|
||
```python
|
||
os.system("git status --porcelain")
|
||
```
|
||
|
||
---
|
||
|
||
# 28. Git Integration
|
||
|
||
The Git integration may use either:
|
||
|
||
* the Git CLI through `subprocess`; or
|
||
* a Python Git library.
|
||
|
||
Using the Git CLI is recommended initially because it closely follows native Git behaviour and makes troubleshooting straightforward.
|
||
|
||
The Git executable shall therefore be treated as an external prerequisite. A minimum Git version of 2.20 shall be assumed.
|
||
|
||
At startup the application shall verify:
|
||
|
||
```text
|
||
git --version
|
||
```
|
||
|
||
and fail gracefully if Git is unavailable.
|
||
|
||
---
|
||
|
||
# 29. Python Version
|
||
|
||
Recommended minimum:
|
||
|
||
```text
|
||
Python >= 3.12
|
||
```
|
||
|
||
The application should avoid platform-specific Python dependencies unless they are isolated behind optional adapters.
|
||
|
||
---
|
||
|
||
# 30. Command-Line Interface
|
||
|
||
Suggested commands:
|
||
|
||
```text
|
||
git-auto-tracker run
|
||
git-auto-tracker status
|
||
git-auto-tracker validate
|
||
git-auto-tracker config
|
||
```
|
||
|
||
Global options shall include:
|
||
|
||
```text
|
||
--config PATH Path to the configuration file (default: git-auto-tracker.yaml)
|
||
--log-level LEVEL Override the configured log level
|
||
--version Print the application version and exit
|
||
```
|
||
|
||
### `run`
|
||
|
||
Perform change collection and create a commit if necessary.
|
||
|
||
### `status`
|
||
|
||
Display currently detected changes without staging or committing them.
|
||
|
||
### `validate`
|
||
|
||
Validate:
|
||
|
||
* configuration,
|
||
* Git installation,
|
||
* repository,
|
||
* file access,
|
||
* configured author resolvers.
|
||
|
||
### `config`
|
||
|
||
Display the effective configuration with secrets redacted.
|
||
|
||
---
|
||
|
||
# 31. Dry-Run Mode
|
||
|
||
The application shall support:
|
||
|
||
```text
|
||
git-auto-tracker run --dry-run
|
||
```
|
||
|
||
Dry-run mode shall perform:
|
||
|
||
```text
|
||
change detection
|
||
author resolution
|
||
commit message generation
|
||
validation
|
||
```
|
||
|
||
but shall not:
|
||
|
||
```text
|
||
stage files
|
||
create commits
|
||
push data
|
||
modify the repository
|
||
```
|
||
|
||
The generated commit message should be displayed.
|
||
|
||
The derived-artifact refresh (Section 45) is skipped in dry-run mode because it writes to the working tree; dry-run analyses the artifacts from the previous collection and reports this in its output.
|
||
|
||
---
|
||
|
||
# 32. Security
|
||
|
||
The application shall follow least-privilege principles.
|
||
|
||
The execution account should require only:
|
||
|
||
* read access to source metadata;
|
||
* read/write access to the tracked repository;
|
||
* access required to query SCADA audit information.
|
||
|
||
SCADA engineering credentials shall not be embedded in source code.
|
||
|
||
Secrets shall be provided through:
|
||
|
||
* environment variables;
|
||
* operating-system secret stores;
|
||
* external secret-management systems.
|
||
|
||
Secrets must never appear in Git commit messages or application logs.
|
||
|
||
---
|
||
|
||
# 33. Sensitive File Protection
|
||
|
||
The application should support rules preventing accidental versioning of sensitive data.
|
||
|
||
Example:
|
||
|
||
```yaml
|
||
security:
|
||
forbidden_patterns:
|
||
- "**/*.key"
|
||
- "**/*.pem"
|
||
- "**/secrets.*"
|
||
- "**/credentials.*"
|
||
```
|
||
|
||
If such a file is detected, the default behaviour shall be to abort the commit, log a security warning, and exit with the dedicated security exit code (see Section 25). A configurable alternative may skip the forbidden file and commit the remaining changes, but skipping shall never be the default.
|
||
|
||
---
|
||
|
||
# 34. Remote Repository
|
||
|
||
Remote synchronization shall be optional.
|
||
|
||
Future configuration could contain:
|
||
|
||
```yaml
|
||
remote:
|
||
enabled: true
|
||
name: origin
|
||
push_after_commit: true
|
||
```
|
||
|
||
A failed push shall not invalidate the locally created commit.
|
||
|
||
The application shall distinguish:
|
||
|
||
```text
|
||
COMMIT_SUCCESS
|
||
PUSH_FAILURE
|
||
```
|
||
|
||
---
|
||
|
||
# 35. Audit Information
|
||
|
||
Each collection should internally generate an execution identifier. A ULID, or an equivalent lexicographically sortable and timezone-independent identifier, is recommended.
|
||
|
||
For example:
|
||
|
||
```text
|
||
01J7K8HYV84ZKFH1J4T9R8DZQN
|
||
```
|
||
|
||
The identifier may be included in the commit:
|
||
|
||
```text
|
||
Tracker-Run: 01J7K8HYV84ZKFH1J4T9R8DZQN
|
||
```
|
||
|
||
This provides traceability between:
|
||
|
||
* application logs;
|
||
* Git commits;
|
||
* external monitoring;
|
||
* SCADA audit records.
|
||
|
||
---
|
||
|
||
# 36. Structured Commit Metadata
|
||
|
||
To simplify future machine processing, the end of the commit message may include structured trailers.
|
||
|
||
Example:
|
||
|
||
```text
|
||
Tracker-Version: 1.0.0
|
||
Tracker-Run: 01J7K8HYV84ZKFH1J4T9R8DZQN
|
||
Tracker-Host: SCADA-SRV-01
|
||
Tracker-Changes: 6
|
||
```
|
||
|
||
This enables later processing without parsing the human-readable sections. The trailer block shall follow the Git trailer convention so that standard tooling such as `git interpret-trailers` can extract it.
|
||
|
||
---
|
||
|
||
# 37. Functional Requirements
|
||
|
||
## FR-001 – Repository inspection
|
||
|
||
The system shall inspect a configured Git repository for uncommitted file changes.
|
||
|
||
## FR-002 – Added files
|
||
|
||
The system shall identify newly added files.
|
||
|
||
## FR-003 – Modified files
|
||
|
||
The system shall identify modified files.
|
||
|
||
## FR-004 – Deleted files
|
||
|
||
The system shall identify deleted files.
|
||
|
||
## FR-005 – Rename detection
|
||
|
||
The system should identify renamed files where practical.
|
||
|
||
## FR-006 – Author identification
|
||
|
||
The system shall attempt to determine the last modifying identity for every added and modified file.
|
||
|
||
## FR-007 – Unknown author
|
||
|
||
The system shall support an `UNKNOWN` author state.
|
||
|
||
## FR-008 – Per-file attribution
|
||
|
||
Author information shall be associated with individual files rather than the complete commit.
|
||
|
||
## FR-009 – Commit generation
|
||
|
||
The system shall automatically generate a Git commit for detected changes.
|
||
|
||
## FR-010 – Commit summary
|
||
|
||
The commit message shall contain counts for each change type.
|
||
|
||
## FR-011 – Commit details
|
||
|
||
The commit message shall list every affected file.
|
||
|
||
## FR-012 – Attribution
|
||
|
||
Added and modified files shall contain last-modifier information in the commit message.
|
||
|
||
## FR-013 – Deleted files
|
||
|
||
Deleted files shall be listed without requiring modifier information.
|
||
|
||
## FR-014 – Empty executions
|
||
|
||
The system shall not create an empty commit when no changes are detected.
|
||
|
||
## FR-015 – Logging
|
||
|
||
Every execution shall produce an audit log.
|
||
|
||
## FR-016 – Cross-platform support
|
||
|
||
The system shall operate on Windows, Linux and macOS.
|
||
|
||
## FR-017 – Dry run
|
||
|
||
The system shall support change analysis without repository modification.
|
||
|
||
## FR-018 – Configuration
|
||
|
||
Application behaviour shall be configurable without modifying source code.
|
||
|
||
## FR-019 – File filters
|
||
|
||
The system shall support file inclusion and exclusion patterns.
|
||
|
||
## FR-020 – Concurrency protection
|
||
|
||
The system shall prevent simultaneous tracker executions against the same repository.
|
||
|
||
## FR-021 – Error isolation
|
||
|
||
Failure to identify an author shall not normally prevent an otherwise valid change from being committed.
|
||
|
||
## FR-022 – Repository protection
|
||
|
||
Git, filesystem or repository integrity errors shall prevent commit creation.
|
||
|
||
## FR-023 – Atomic collection
|
||
|
||
A normal execution shall produce at most one commit representing the complete detected change set.
|
||
|
||
## FR-024 – Sensitive-data protection
|
||
|
||
Configured forbidden files shall not automatically be committed.
|
||
|
||
## FR-025 – Concurrent modification protection
|
||
|
||
The system shall detect files modified during collection and abort or retry according to a configured policy. The default policy shall abort the commit.
|
||
|
||
## FR-026 – Index-state validation
|
||
|
||
The system shall refuse to execute when the repository index already contains staged changes or the repository is in an unfinished merge/rebase state.
|
||
|
||
## FR-027 – Security abort
|
||
|
||
The system shall abort the commit and exit with a dedicated exit code when a forbidden file is detected.
|
||
|
||
## FR-028 – Self-exclusion
|
||
|
||
The system shall never commit its own operational artefacts, such as lock files, logs and state files.
|
||
|
||
## FR-029 – Filename safety
|
||
|
||
The system shall correctly process file paths containing spaces, quotes and non-ASCII characters on all supported platforms.
|
||
|
||
## FR-030 – Derived artifact extraction
|
||
|
||
The system shall support extraction of configured binary database sources into plain-text derived artifacts inside the repository, refreshed before change detection on every collection.
|
||
|
||
## FR-031 – Deterministic artifacts
|
||
|
||
Derived artifacts shall be byte-identical between collections whenever the underlying content is unchanged.
|
||
|
||
## FR-032 – Extraction failure isolation
|
||
|
||
A source that cannot be read, is not a SQLite 3 database, or whose artifacts cannot be written shall be skipped with a logged warning while the remaining configured sources are still extracted.
|
||
|
||
## FR-033 – Stale artifact removal
|
||
|
||
Artifacts whose source table or database no longer exists shall be removed during extraction.
|
||
|
||
## FR-034 – Read-only source access
|
||
|
||
The system shall open source databases read-only and shall never write to them.
|
||
|
||
---
|
||
|
||
# 38. Suggested Python Package Structure
|
||
|
||
```text
|
||
git-auto-tracker/
|
||
│
|
||
├── pyproject.toml
|
||
├── README.md
|
||
├── LICENSE
|
||
│
|
||
├── config/
|
||
│ └── example.yaml
|
||
│
|
||
├── src/
|
||
│ └── git_auto_tracker/
|
||
│ │
|
||
│ ├── __init__.py
|
||
│ ├── __main__.py
|
||
│ ├── cli.py
|
||
│ ├── config.py
|
||
│ ├── application.py
|
||
│ │
|
||
│ ├── git/
|
||
│ │ ├── repository.py
|
||
│ │ ├── status.py
|
||
│ │ └── commit.py
|
||
│ │
|
||
│ ├── changes/
|
||
│ │ ├── detector.py
|
||
│ │ ├── models.py
|
||
│ │ └── filters.py
|
||
│ │
|
||
│ ├── extraction/
|
||
│ │ ├── base.py
|
||
│ │ └── sqlite.py
|
||
│ │
|
||
│ ├── authors/
|
||
│ │ ├── base.py
|
||
│ │ ├── resolver.py
|
||
│ │ ├── unknown.py
|
||
│ │ └── scada/
|
||
│ │ └── ...
|
||
│ │
|
||
│ ├── commits/
|
||
│ │ └── message_builder.py
|
||
│ │
|
||
│ ├── security/
|
||
│ │ └── validator.py
|
||
│ │
|
||
│ ├── locking/
|
||
│ │ └── repository_lock.py
|
||
│ │
|
||
│ └── logging/
|
||
│ └── setup.py
|
||
│
|
||
└── tests/
|
||
├── unit/
|
||
├── integration/
|
||
└── fixtures/
|
||
```
|
||
|
||
---
|
||
|
||
# 39. Core Domain Objects
|
||
|
||
The project should initially define the following central objects:
|
||
|
||
```text
|
||
Repository
|
||
ChangeSet
|
||
ChangeRecord
|
||
AuthorIdentity
|
||
AuthorResolution
|
||
TrackerRun
|
||
CommitResult
|
||
```
|
||
|
||
A `ChangeSet` conceptually contains:
|
||
|
||
```text
|
||
ChangeSet
|
||
├── added[]
|
||
├── modified[]
|
||
├── deleted[]
|
||
└── renamed[]
|
||
```
|
||
|
||
Each item is represented by a `ChangeRecord`.
|
||
|
||
---
|
||
|
||
# 40. Processing Sequence
|
||
|
||
A normal execution shall approximately perform the following sequence:
|
||
|
||
```text
|
||
1. Load configuration
|
||
2. Validate configuration
|
||
3. Verify Git availability
|
||
4. Acquire repository lock
|
||
5. Validate repository, branch and index state
|
||
6. Refresh derived artifacts (Section 45)
|
||
7. Determine current repository status
|
||
8. Apply inclusion/exclusion and security rules
|
||
9. Create ChangeSet
|
||
10. Resolve authors for ADDED files
|
||
11. Resolve authors for MODIFIED files
|
||
12. Generate commit message
|
||
13. Stage selected changes
|
||
14. Verify staged change set
|
||
15. Create commit
|
||
16. Record commit hash and write audit result
|
||
17. Release lock
|
||
18. Exit
|
||
```
|
||
|
||
If no changes exist, execution proceeds directly from step 9 to audit logging and termination.
|
||
|
||
If any step from staging onwards fails, the index shall be restored to its pre-execution state before the lock is released.
|
||
|
||
---
|
||
|
||
# 41. Testing Requirements
|
||
|
||
Tests should cover at minimum:
|
||
|
||
### Change detection
|
||
|
||
* new file;
|
||
* modified file;
|
||
* deleted file;
|
||
* rename;
|
||
* multiple simultaneous changes;
|
||
* Unicode filenames;
|
||
* filenames containing spaces.
|
||
|
||
### Git
|
||
|
||
* clean repository;
|
||
* dirty repository;
|
||
* invalid repository;
|
||
* failed commit;
|
||
* unavailable Git executable.
|
||
|
||
### Author resolution
|
||
|
||
* one successful resolver;
|
||
* multiple resolver fallback;
|
||
* unknown user;
|
||
* unavailable audit source;
|
||
* malformed SCADA metadata.
|
||
|
||
### File handling
|
||
|
||
* inaccessible file;
|
||
* file removed during collection;
|
||
* file modified during collection;
|
||
* excluded file;
|
||
* forbidden file.
|
||
|
||
### Extraction (Section 45)
|
||
|
||
* enabled extraction produces an artifact file for every configured table;
|
||
* unchanged database content produces byte-identical artifacts (no spurious commit);
|
||
* a modified row appears as an artifact change in the commit;
|
||
* a removed table or database removes its artifact files;
|
||
* an unreadable or locked source database aborts with exit code 6;
|
||
* source databases are opened read-only.
|
||
|
||
### Safety and edge cases
|
||
|
||
* pre-existing staged changes in the index;
|
||
* repository in an unfinished merge/rebase state;
|
||
* current branch differs from the configured branch;
|
||
* forbidden-file abort produces the dedicated exit code;
|
||
* stale lock left by a dead process is recovered;
|
||
* lock file and log file never appear in `git status`;
|
||
* configured timezone is honoured in commit timestamps.
|
||
|
||
### Cross-platform
|
||
|
||
Automated testing should run against at least:
|
||
|
||
```text
|
||
Windows
|
||
Linux
|
||
```
|
||
|
||
with macOS included where practical.
|
||
|
||
A CI test matrix is recommended.
|
||
|
||
---
|
||
|
||
# 42. Future Functional Extensions
|
||
|
||
The architecture should permit future introduction of:
|
||
|
||
* automatic push to central Git server;
|
||
* signed Git commits;
|
||
* commit signing using a machine identity;
|
||
* SCADA semantic comparison;
|
||
* XML-aware comparison;
|
||
* JSON-aware comparison;
|
||
* proprietary SCADA project parsers;
|
||
* alarm configuration comparison;
|
||
* PLC source comparison;
|
||
* automatic reports;
|
||
* REST API;
|
||
* web interface;
|
||
* Grafana integration;
|
||
* email/Teams notifications;
|
||
* change approval workflow;
|
||
* correlation against maintenance or change requests;
|
||
* integration with Active Directory;
|
||
* integration with SCADA audit logs;
|
||
* cryptographic file hashes;
|
||
* asset/reference-designation correlation;
|
||
* automatic mapping of files to IEC/ISO 81346 objects;
|
||
* detection of changes outside approved maintenance windows.
|
||
|
||
---
|
||
|
||
# 43. Important Design Principle
|
||
|
||
The application shall distinguish between three separate concepts:
|
||
|
||
```text
|
||
Git commit author
|
||
≠
|
||
File last modifier
|
||
≠
|
||
SCADA change originator
|
||
```
|
||
|
||
The Git commit author is the automated service.
|
||
|
||
The file last modifier is information obtained from the best available metadata source.
|
||
|
||
The actual SCADA change originator should only be stated when an authoritative source, such as the SCADA audit trail, provides that information.
|
||
|
||
The application shall not infer a human change originator merely from filesystem ownership.
|
||
|
||
---
|
||
|
||
# 44. Definition of Done – Initial Release
|
||
|
||
Version 1.0 can be considered complete when it can:
|
||
|
||
1. run on Windows and Linux;
|
||
2. inspect an existing Git repository;
|
||
3. detect added, modified and deleted files;
|
||
4. filter files through configurable include/exclude rules;
|
||
5. invoke a configurable author resolver;
|
||
6. record per-file author information for additions and modifications;
|
||
7. generate the defined structured commit message;
|
||
8. automatically stage and commit changes;
|
||
9. avoid empty commits;
|
||
10. provide dry-run functionality;
|
||
11. produce rotating application logs;
|
||
12. prevent concurrent execution;
|
||
13. return useful process exit codes;
|
||
14. safely handle unresolved authors;
|
||
15. execute unattended through an external scheduler.
|
||
|
||
The initial implementation should deliberately keep **SCADA-specific change attribution behind the `AuthorResolver` interface**. That makes the generic Git/change-tracking engine reusable while allowing a resolver tailored to the actual SCADA platform to be added independently.
|
||
|
||
---
|
||
|
||
# 45. Derived Artifacts (Content Extraction)
|
||
|
||
This feature is prioritized for the next release.
|
||
|
||
## 45.1 Purpose
|
||
|
||
Some SCADA engineering data is stored in binary databases (e.g. SQLite). Git cannot show readable history for such files: every commit stores the whole binary blob and a diff reports only "binary files differ". To make this content auditable, the tracker shall extract selected database content into plain-text derived artifacts inside the repository before change detection (Section 40, step 6). The artifacts are then versioned like any other file, and Git history shows precise, row-level changes.
|
||
|
||
Source databases are identified by content (the SQLite file header read during extraction), not by file-name extension.
|
||
|
||
The implementation shall follow the same pluggable-interface principle as author resolution (Section 44): an `ArtifactExtractor` abstraction with a SQLite implementation as the first plugin, so further formats can be added independently.
|
||
|
||
## 45.2 Output layout
|
||
|
||
Extraction writes into a configurable output root inside the repository (default: `extracted/`). One sub-folder is created per configured source database, named after the database file. Within each sub-folder, one file is written per extracted table.
|
||
|
||
```text
|
||
extracted/
|
||
└── alarms.sqlite/
|
||
├── alarms.jsonl
|
||
└── events.jsonl
|
||
```
|
||
|
||
This layout gives each table its own artifact file, so Git diffs are limited to the tables that actually changed, and table names cannot collide between databases.
|
||
|
||
## 45.3 Configuration
|
||
|
||
```yaml
|
||
extraction:
|
||
enabled: true
|
||
output_root: "extracted" # relative to the repository root
|
||
format: "jsonl" # one JSON object per row
|
||
sources:
|
||
- path: "history/alarms.sqlite" # repository-relative source
|
||
tables: # optional; default: all user tables
|
||
- "alarms"
|
||
- "events"
|
||
```
|
||
|
||
An absent `extraction:` block, or `enabled: false`, disables the feature entirely; collection then proceeds without touching any database.
|
||
|
||
## 45.4 Artifact format
|
||
|
||
The default format shall be JSON Lines (`jsonl`): one JSON object per row with sorted keys and a stable encoding, so a changed row produces a minimal line-level diff. Rows shall be ordered deterministically by primary key (falling back to `rowid`), so unchanged content always produces a byte-identical file.
|
||
|
||
Artifact content shall not contain run-dependent values such as extraction timestamps: an artifact must be byte-identical whenever the underlying content is unchanged (FR-031). Table names that are not valid file names on the target platform shall be sanitized deterministically for artifact file names (FR-029).
|
||
|
||
## 45.5 Source access
|
||
|
||
Source databases shall be opened read-only; the tracker shall never write to them (FR-034). Because the SCADA platform may keep databases in WAL mode while running, extraction shall operate on a snapshot: the database file and its WAL sidecar shall be copied to a temporary location before reading, so a partially written transaction is never captured (see also Section 20).
|
||
|
||
## 45.6 Stale artifacts
|
||
|
||
Extraction shall also remove artifacts that no longer have a source:
|
||
|
||
* a table removed from the database → its artifact file is deleted;
|
||
* a source database removed from the configuration or the file system → its sub-folder is deleted.
|
||
|
||
Without this rule, stale artifacts would look like current data in Git history (FR-033).
|
||
|
||
## 45.7 Failure policy
|
||
|
||
Sources are isolated from each other: a source that cannot be read, is not a SQLite 3 database, or whose artifacts cannot be written, is reported as a warning and skipped; extraction then continues with the remaining configured sources. One bad file shall never block extraction of the others. When every configured source fails, the collection proceeds with the artifacts from the previous collection and a summary warning lists all skipped sources. Source validity is established during extraction by the SQLite engine itself - a non-database file raises `sqlite3.DatabaseError: file is not a database` - rather than by file-name extension: databases with vendor-specific or missing extensions are accepted (Section 45.5).
|
||
|
||
An extraction failure is not silent, however: every skipped source is logged, and a source that fails persistently must be visible in monitoring through the log (Section 22).
|
||
|
||
## 45.8 Raw source files
|
||
|
||
The configured binary source files should be excluded from direct tracking (`files.exclude`, e.g. `**/*.sqlite`). The repository then stores the readable artifacts instead of the opaque blob; tracking both duplicates every change and lets the binary churn dominate history. Direct tracking of the raw database remains possible where a deployment wants the authoritative blob as well.
|
||
|
||
## 45.9 Attribution
|
||
|
||
Generated artifacts have no human last-modifier of their own. Author resolution for artifact files shall therefore report `UNKNOWN` unless an extractor emits sidecar metadata (Section 10) that the resolver chain can consume, for example a `last_modified_by` column extracted from the database content.
|
||
|
||
---
|
||
|
||
# Appendix A – Glossary
|
||
|
||
| Term | Definition |
|
||
| ------------------- | ---------------------------------------------------------------------------------------- |
|
||
| Collection | One complete detection-and-commit cycle performed by the tracker |
|
||
| ChangeSet | The complete set of changes detected during one collection |
|
||
| ChangeRecord | A single detected change together with its metadata |
|
||
| AuthorResolver | Pluggable component attempting to determine the last modifying identity of a file |
|
||
| AuthorResult | The outcome of one author-resolution attempt, including source and confidence |
|
||
| Confidence | Trustworthiness of an author result: `authoritative`, `probable`, `weak` or `unknown` |
|
||
| Fallback | Result used when no resolver produces a usable result (default: `UNKNOWN`) |
|
||
| Forbidden file | A file matching `security.forbidden_patterns`; never committed by default |
|
||
| Tracker-Run | Unique execution identifier linking logs, commits and monitoring (Section 35) |
|
||
| Staged verification | Comparison of the actual staged change set against the expected change set before commit |
|
||
| Derived artifact | Plain-text file generated from a binary source (e.g. a SQLite table) and versioned in the repository (Section 45) |
|
||
| Artifact extractor | Pluggable component that refreshes derived artifacts from configured sources before change detection |
|