git-auto-tracker

Unattended change tracker that records modifications to SCADA project and configuration files in a Git repository.

The tracker periodically inspects a configured repository, detects added / modified / deleted files, resolves the last modifying user for each change (where a metadata source is available), and creates one Git commit per collection cycle with a structured, auditable commit message.

See functional specification.md for the full specification.

Features

Run-once execution, designed for Windows Task Scheduler / cron / systemd timers
Change detection via git status --porcelain -z (NUL-safe, UTF-8)
Configurable include/exclude file filters (globs)
Security validation with forbidden patterns (abort by default)
Pluggable author resolvers with priority chain and confidence levels
Per-file attribution recorded in the commit message
One-line-per-file commit entries in git diff --name-status style (letter, path, author, last-modified time), with a configurable separator (commit.file_separator, default ;)
Derived artifacts: SQLite content extraction into JSON Lines (one sub-folder per database, one file per table), byte-stable, read-only source access (Section 45)
Structured commit trailers (Tracker-Run, Tracker-Host, ...)
Repository-level lock with stale-lock detection (stored outside the repo)
Rotating file logs plus console logging
Dry-run mode, exit codes suitable for external monitoring
Cross-platform: Windows / Linux / macOS

Installation

python -m venv .venv
.venv/Scripts/activate   # Windows
source .venv/bin/activate  # Linux/macOS
pip install -e .

Usage

git-auto-tracker run [--dry-run]   # collect changes and commit
git-auto-tracker status            # show detected changes without committing
git-auto-tracker validate          # validate configuration / repository / resolvers
git-auto-tracker config            # print effective configuration

Global options:

--config PATH      Path to configuration file (default: git-auto-tracker.yaml)
--log-level LEVEL  Override the configured log level
--version          Print version and exit

Configuration

See config/example.yaml. Configuration is YAML; deployment-specific values can be overridden through environment variables using the GIT_AUTO_TRACKER_<SECTION>__<KEY> convention.

Specification status

Progress against functional specification.md (FR-001 – FR-034). The v1 scope is defined by Section 44, Definition of Done – Initial Release; FR-030 – FR-034 (Section 45, derived artifacts) are prioritized for the next release.

Implemented

Inspect an existing Git repository (FR-001)
Detect added, modified and deleted files (FR-002 – FR-004)
Rename detection via the add+delete fallback (FR-005, Section 6.1)
Configurable author resolver chain with confidence levels and UNKNOWN fallback; sidecar metadata resolver implemented (FR-006, FR-007, Sections 9 – 11)
Per-file attribution in the commit message (FR-008, FR-012)
Automatic commit generation (FR-009)
Structured commit message: summary counts, one-line-per-file entries with change letter / path / author / local-time mtime, Tracker-* trailers (FR-010, FR-011, Sections 15/16/36)
No empty commits (FR-014)
Rotating application logs, console + file (FR-015, Sections 22 – 23)
Cross-platform code base: pathlib, argument arrays, UTF-8/NUL-safe Git parsing (FR-016, FR-029)
Dry-run mode (FR-017, Section 31)
YAML configuration with environment overrides (FR-018, Section 26)
Include/exclude file filters with gitignore-style ** globs (FR-019, Section 7)
Repository lock with stale-lock recovery, stored outside the repository (FR-020, FR-028, Section 21)
Author-resolution errors never block an otherwise valid commit (FR-021)
Git / repository integrity errors prevent the commit; index restored on failure (FR-022, Section 14)
One atomic commit per collection cycle (FR-023)
Forbidden-file abort with dedicated security exit code (FR-024, FR-027, Section 33)
Index-state validation: pre-existing staged changes / unfinished merge abort (FR-026, Section 5.1)
Exit codes 0 – 7 for external monitoring (Section 25)
CLI: run / status / validate / config with secret redaction (Section 30)
ULID run identifiers for log/commit traceability (Section 35)
Commit author is the tracker service identity, engineers recorded per file (Section 17/18/43)
Derived artifacts: SQLite content extraction – extraction: config block, deterministic JSON Lines output, stale artifact removal, extraction failure aborts (exit 6), read-only snapshot source access, skipped in dry-run (FR-030 – FR-034, Section 45)

Left to do

Required by the specification (v1 gaps):

Concurrent-modification detection (FR-025, Section 20) – deferred, risk accepted: engineering-only repository, once-daily midnight cadence; staged-set verification already prevents broken commits. Revisit if run frequency increases (Section 4.2).
Log execution duration per collection (Section 22)
validate: also verify file access (Section 30)
status: apply include/exclude filters and preview author resolution (Section 30)
Configurable alternative policy for pre-existing staged changes: commit with UNKNOWN attribution (Section 5.1)
Configurable alternative to skip (instead of abort on) forbidden files (Section 33)
Native rename detection instead of the add+delete fallback (FR-005 "should", Section 6.1)
Missing tests (Section 41): failed commit, unavailable Git executable, inaccessible file, file removed during collection, macOS in the CI matrix. (The "file modified during collection" case is deferred together with FR-025.)

Blocked on SCADA platform knowledge:

SCADA metadata resolver – embedded project metadata (Section 10)
SCADA audit-log resolver (Section 10)
Operating-system audit resolver (Section 10, optional)

Optional / future extensions (Section 42, "may be added later"):

Scheduled service mode (Section 4.2)
Remote push with COMMIT_SUCCESS / PUSH_FAILURE distinction (Section 34)
Signed Git commits
SCADA semantic / XML-aware / JSON-aware comparison
Proprietary SCADA project parsers
Automatic reports, REST API, web interface, Grafana integration
Email/Teams notifications
Change approval workflow, correlation with maintenance/change requests
Active Directory integration
Cryptographic file hashes
IEC/ISO 81346 asset mapping
Detection of changes outside approved maintenance windows
Honor application.timezone in log and commit-message timestamps (Sections 22, 26); add the timezone test from Section 41

Development

pip install -e . --group dev
pytest
ruff check src tests

Building an executable (local dev option)

scripts/build_exe.py builds a self-contained executable with PyInstaller and smoke-tests it. It creates a dedicated clean build environment (.venv-build) and installs the package non-editable there — the reliable input for PyInstaller analysis. Git is not bundled; it remains an external prerequisite on the target machine (Section 28).

python scripts/build_exe.py               # build + smoke test (onedir)
python scripts/build_exe.py --onefile     # single-file binary
python scripts/build_exe.py --skip-smoke  # build only

Output: dist/git-auto-tracker/ (onedir — deploy the whole folder) or a single dist/git-auto-tracker.exe with --onefile (slower startup, more prone to antivirus false positives — prefer onedir for SCADA hosts).

7.6 KiB Raw Blame History Unescape Escape