Configuration Scan

---
name: configuration-scan
description: Reads how the software is configured: sources (config files, env vars, CLI flags, profiles, remote settings) and their precedence, the settings surface and defaults, how configuration reaches the code, how secrets are supplied and held, validation and failure on bad values, feature flags, runtime reload. Use for how an app is configured, config files, env vars, precedence, defaults, feature flags, where credentials come from, or a configuration review.
---

# Configuration scan

Configuration is the part of a program that its users write. It has a schema nobody declared, a precedence order nobody wrote down, and defaults that decide most installations' behaviour — and when it is wrong, the program usually starts anyway and misbehaves later. This scanner reconstructs the configuration system as a system: what can be set, from where, in what order, with what default, checked how, and what happens when it is absent or nonsense.

**First read `sokrates-scan-core/SKILL.md`** (sibling skill) — output format, evidence rules, validate/render scripts, `_sokrates` layout. This file adds only what is specific to configuration scanning.

## The one question

*For one setting, where can it come from, which source wins, what is it if nobody sets it, and what happens if it is wrong?* Answer it end to end for the two or three most consequential settings, then state the system that governs all of them.

**Settings vs. specification.** Some tools are driven by a config file that is really an *input document* — an analysis specification, a pipeline definition, a scene description — where most keys describe the work to do rather than tune how the program behaves. Split the surface in two and say so in `settings-inventory` and `stats.count_rule`: **tuning keys** (an operator would change them to get different behaviour on the same input — thresholds, paths, toggles, credentials) get the full treatment below; **specification keys** (they *are* the input — the components to analyse, the rules to apply) are described once as a group, with their schema and validation, and are not traced through precedence one by one. When the split is genuinely unclear, count them together and state that in `count_rule` rather than inventing a boundary.

## Scope and boundaries with sibling scanners

- **`tech-stack-scan`** names the configuration *libraries* (Viper, figment, dotenv, Spring `@ConfigurationProperties`, pydantic-settings). Read it first; describe usage, not inventory.
- **`functionality-scan`** (`entry-points`) may list configuration as a product surface. This scanner owns the mechanics — precedence, defaults, validation, reload — and references functionality's finding for what the surface is *for*.
- **`security-scan`** owns whether a credential leaks, whether protection is adequate, and permission bits (`secrets/at-rest`, `secrets/in-tree`). This scanner owns the **plumbing**: which sources supply secrets, how a secret travels from that source into a client, what redaction exists as a configuration mechanism, and what an operator has to do to configure one. Do not sweep the tree for credentials and do not re-rate a leak security found — reference it. A dangerous *default* (a security feature off unless configured) is security's when it is the mechanism's default, and this scanner's when it is a property of the configuration system itself (a config file read from a world-writable location, a value that silently falls back to an insecure mode).
- **`iac-scan`** owns what infrastructure declares — a manifest's `env:` block, Helm values, Terraform variables. This scanner owns what the *application* does with them, including which env vars it reads. Describe the wiring once in whichever scanner reads the file and reference the other: the manifest key is iac's, the `env::var` lookup and its default are this scanner's.
- **`cicd-scan`** owns pipeline configuration and CI-supplied secrets as pipeline mechanics; this scanner owns how the *shipped product* is configured, including how CI-injected values reach it.
- **`storage-scan`** owns config *files as data on disk* (write safety, read-modify-write races, format versioning of a config file the tool rewrites). This scanner owns their meaning as settings. When storage already rated the write pattern, reference it and keep only the configuration consequence.
- **`reliability-scan`** owns error handling generally; misconfiguration handling (fail-fast vs silent default) is this scanner's, referenced to reliability's error-model finding.
- **`observability-scan`** owns telemetry; whether telemetry is opt-in or opt-out is a configuration default and belongs here with a cross-reference.

**Cross-referencing.** List existing ids first (`grep -h '"id"' _sokrates/reports/ai-insights/*.json --exclude=combined-report.json`) and reference siblings as `sokrates_refs: ["finding:<scanner>/<group>/<slug>"]` — only ids you saw. Never copy another scanner's evidence blocks.

## Workflow

1. **Orient per the core skill.** From `tech-stack-scan` (config libraries), `functionality-scan` (`entry-points`, the configuration surface), `architecture-scan` (which component owns settings), `security-scan` (secrets, if it ran), `iac-scan` (what infrastructure supplies). Note the system kind: a **CLI/desktop tool** with per-user and per-project files; a **service** configured by environment and mounted files; a **library** whose configuration is its API; a **batch tool** driven by one config file per run.
2. **Count with the script, then find the loader.** Run
   ```bash
   python3 <this-skill-path>/scripts/count_config_sites.py <src-root> --json <scratch>/config-counts.json
   ```
   It counts, per ecosystem and excluding tests: environment-variable reads (with the variable names it saw), CLI-flag definitions, config-file loads and format parses, config-library usage, default-value shapes, validation and required-value checks, secret-source lookups (keychains, vaults, token files, `.env`), feature-flag checks, reload/watch mechanisms, and the config files present in the tree. The `env_var_names` and `flag_names` lists are the settings surface's raw material — the script resolves names given as same-file string constants (the `getenv(ENV_FOO)` / `new Option(ARG_BAR, …)` idiom), but a name assembled at runtime or held in another file stays invisible: when it warns that few names were extracted from many declaration sites, read the declaration sites (`Commands.java`, the `Arg`/`Option` constructors) and rebuild the surface by hand. `*_candidates`/`*_keyword_files` are reading lists only. Its `config_files_in_tree` list is the config files **this repository ships**; for a tool whose config lives in the projects it processes, that list will be near-empty and the real surface is in the schema types and the docs — do not read it as "this tool has no configuration".
3. **Read the loader completely.** There is almost always one place where sources are combined — a `Config::load`, a settings module, a `figment`/`viper` builder, a constructor chain. **When the loader is a single deserialization** (`readFileToString` then `fromJson`, with no merging at all), that is the answer to steps 3 and the precedence question both: say so in two sentences, note where the defaults actually live instead (field initializers, a `Default` impl, an `init` command that writes them out), and spend the reading budget on the schema types and step 4's surface instead. Read it end to end: which sources it consults, in which order, how it merges (whole-file replacement, per-key override, deep merge of tables), where defaults live (a `Default` impl, a constant map, the schema, or inline `unwrap_or`), and what it does with an unknown key or an unparseable file. This one reading produces `sources/config-sources`, `sources/precedence` and much of `schema-and-defaults`.
4. **Build the settings surface.** Not one finding per setting: group by area (network, model/provider, sandbox, logging, UI) and state per area how many settings, where they are defined, and the notable defaults. Name individually only the settings that change behaviour materially or are dangerous when wrong. Cross-check the documented surface against the code's own list: settings the code reads but no document mentions are `undocumented-settings`; documented settings the code never reads are `stale-settings` — both are grep-verifiable and among the most useful findings this scanner produces. **When the docs are partial** (the normal case — some settings documented, some not), do not attempt a key-by-key verdict on all of them: grep each key once, report the *counts* with the method stated ("of 96 keys, 61 appear in `docs/config.md`"), and name individually only the undocumented settings that are **dangerous when wrong** and the stale ones you confirmed the code never reads (a grep for the key returning no non-doc hit). A key that is mentioned in passing but not explained is documented for this count; say so in `count_rule` rather than adjudicating prose quality.
5. **Trace the settings that actually have something to trace.** Pick the ones that matter (the endpoint, the credential, the sandbox or safety switch): every source that can set it, the winner, the default, the type coercion, the validation, and the behaviour when absent. Two or three such traces are the findings a reader will actually use — but trace only settings with more than one source or a non-trivial default: on a single-source target there may be exactly one, or none, and "for the other N settings the file value is used as-is" is the honest statement to make once rather than a trace to repeat. Write each as its own `sources/traced-<setting>` finding — that slug exists so the trace is not squeezed into `precedence`'s evidence budget.
6. **Read secrets plumbing.** Which sources supply credentials (env var, config file field, key file, OS keychain, cloud secret manager, CI injection, interactive login); how the value travels from source to client; whether a plaintext fallback exists when the secure source is unavailable; what redaction the configuration layer applies when printing or logging settings; what an operator must do to set one up (the `.env.example`, the docs line). Reference security-scan for the protection verdict; report the mechanism here.
7. **Read validation and failure behaviour.** Is there a schema (types, required fields, enums, ranges)? Is an unknown key rejected, warned about, or silently ignored? Does a malformed file abort startup or fall back to defaults? Is a required-but-missing value caught at startup or at first use, thousands of lines later? Startup-time validation with a clear message is a strength worth an `info` finding.
8. **Read flags and dynamic behaviour.** Feature flags and experiments: where defined, who evaluates them, whether they are static constants, config keys or a remote service; dead flags whose branches are unreachable. Then runtime change: reload on SIGHUP, file watching, hot-reload of some keys but not others, values cached at startup — and whether the code documents which settings need a restart.
9. **Synthesize the posture.** One `configuration-posture/posture` finding, `severity: info`, `confidence: likely`: the configuration model in three sentences (sources, precedence, where defaults live), how an operator finds out what they can set, the riskiest default, the behaviour on bad input, and the three highest-leverage changes as `finding:` refs. Evidence cites the loader or the defaults definition.
10. Write findings, validate, render; re-run the merge script if a `combined-report.json` exists. Report per the core workflow. Scanner id: `configuration-scan`, version `1.1`.

## Group taxonomy

| group | contents |
|---|---|
| `sources` | Where configuration comes from and how it composes: files (formats, locations, discovery), environment variables, CLI flags, profiles/named configurations, remote or database-held settings, compiled-in constants — and the precedence and merge rules that turn them into one effective value |
| `settings-surface` | What is configurable: the settings inventory by area, the notable individual settings, undocumented settings the code reads, stale settings the docs promise but the code ignores, deprecated keys and their migration |
| `schema-and-defaults` | How configuration is represented in the code: typed schema or ad-hoc lookups, where defaults are defined and whether one setting has several conflicting defaults, type coercion, the default *profile* (what an unconfigured installation actually does) |
| `secrets-management` | The plumbing that supplies credentials: sources (env, key file, keychain, vault, CI, interactive login), the path from source to client, plaintext fallbacks, redaction in logs and error messages, what an operator must set up — deferring the protection verdict to `security-scan` |
| `validation` | Schema and value validation, required-value checks, unknown-key policy, when validation runs (startup vs first use), and the behaviour on bad, missing or partially valid configuration |
| `flags-and-runtime` | Feature flags and experiments, kill switches, and dynamic behaviour: reload, watch, restart-required settings, values cached at startup, per-request or per-session overrides |
| `configuration-posture` | The synthesis (one finding, `info`, id `configuration-posture/posture`) |

**Precedence**: a *source* of values → `sources`; what those values *are* → `settings-surface`; where a value comes from when nobody sets it → `schema-and-defaults` (a dangerous default is a `schema-and-defaults` finding naming the setting, not a `settings-surface` one); a check on a value → `validation`, the *consequence* of a failed check → `validation` too (reference reliability); a credential-bearing setting → `secrets-management` for the mechanism, `security-scan` for the protection; a flag that gates a feature → `flags-and-runtime`, a flag that is really a setting with two values → `settings-surface`; a config file the tool itself rewrites → reference `storage-scan` for the write mechanics, keep the settings meaning here.

## Stable ids

Slugs are **source, mechanism, setting or area names, never consequences**. Fixed slugs (use only when the subject exists; parametrised slugs take the name the code itself uses — the env-var prefix, the file name, the area name in lower kebab case):

| group | fixed slugs |
|---|---|
| `sources` | `config-sources` (the one inventory: every source with its location and format), `precedence` (the order and merge semantics, with one setting traced through it — **keep this slug when there is only one source** and state the absence in it: "there is no precedence; the file is the only source", which is a real and useful finding, not a missing one), `file-discovery` (how the file's path is found: home dir, project walk-up, env override, `--config`), `env-vars` (the env-var surface: prefix convention, count, notable names), `cli-flags` (flags that set configuration, as opposed to selecting an action), `profiles` (named configurations, workspaces, `--profile`), `remote-config` (settings *fetched at runtime* from a service or database — a bundle downloaded and then merged as an ordinary layer is a source in `config-sources`, not this), `traced-<setting>` (one end-to-end trace of a consequential setting: every source that can set it, the winner, the default, the validation — the flagship finding of step 5, and the one place where evidence may reach the 3-citation cap on the layer, the override and the default), `source-restrictions` (keys a lower-trust source is forbidden to set: a project-config denylist, an env-var prefix guard, a tenant allowlist) |
| `settings-surface` | `settings-inventory` (the count and the areas, areas listed in `attributes`), `<area>-settings` (one per area whose settings deserve a verdict — `model-settings`, `sandbox-settings`, `logging-settings`), `undocumented-settings`, `stale-settings`, `deprecated-keys` |
| `schema-and-defaults` | `schema-representation` (typed struct/schema vs scattered lookups), `defaults-location` (where defaults live and whether they are single-sourced), `default-profile` (what an unconfigured installation does), `<setting>-default` (only for an individual default that carries its own risk), `type-coercion`, `unwired-settings` (a configuration surface that exists in the code but nothing can reach — a settings object whose fields no loader populates, a switch only one of whose values is selectable; name the surface, say how much of it is reachable) |
| `secrets-management` | `secret-sources` (the one inventory of credential sources), `credential-flow` (source to client for the primary credential), `plaintext-fallback`, `redaction`, `operator-setup` (what a new operator must configure, from `.env.example` or the docs) |
| `validation` | `schema-validation`, `required-values`, `unknown-keys`, `failure-behaviour` (what happens on malformed or missing configuration), `validation-timing` |
| `flags-and-runtime` | `feature-flags`, `kill-switches`, `experiments`, `dead-flags`, `reload`, `restart-required` |
| `configuration-posture` | `posture` |

Project-specific findings get a free slug naming the setting or mechanism (`sources/toml-project-overrides`, `secrets-management/auth-json-refresh`), never the consequence. Several mechanisms sharing a slug are listed in `attributes`. A fixed slug whose subject is absent (no profiles, no remote config, no flags) gets **no finding** — record the absence in `stats` and in one line of the posture.

## What a good finding looks like

Evidence is the `env::var("...")` line, the `#[serde(default = ...)]` attribute, the `merge`/`join` call in the loader, the constant that holds a default, the `.env.example` line, the flag definition, the `expect`/`bail` on a missing value, the docs table row that a setting is missing from. Descriptions speak per source or per area: "settings come from four sources merged per key — built-in defaults, `~/.codex/config.toml`, `CODEX_*` environment variables, then `-c key=value` flags, last wins; the sandbox mode is the one setting also overridable per session".

Expect 12–18 findings for an application with a real configuration system, 8–12 for a service configured by environment alone, 5–8 for a library or a batch tool whose surface is genuinely small; the driver is **the size of the settings surface, not the number of sources** — a batch tool with one config file and eighty keys, two schemas and real documentation drift is a 15–20 finding target, and merging distinct subjects to land inside a band is the one thing the band must never cause. roughly half `info` (the model and the mechanisms that work). Mandatory slots: `config-sources`, `precedence` (which states the absence of precedence when there is one source — see its slug row), `settings-inventory`, `defaults-location`, `failure-behaviour`, `posture`. Everything else exists only when its subject does; a group whose whole mechanism class is absent (no secrets, no flags) contributes a line to `stats.absent_mechanisms` and one clause of the posture, never a finding.

Counting conventions, so re-runs compare: a **setting** is one key an operator can set, counted once no matter how many sources can set it; an **env var** is one name the code reads (the script's `env_var_names` list minus the ones that are not settings — `HOME`, `PATH`, `CI` are environment queries, not configuration, and belong in `count_notes`); a **flag** is one CLI option that sets a configuration value, not one that selects a command — the script cannot make that split (its `flag_names` is every option in the binary, subcommands and all), so derive the configuration-setting subset by reading the option declarations and state both numbers (`cli_flags_total`, `config_setting_flags`). Many mature tools have exactly one such flag (a generic `-c key=value`); that is a finding about the design, not a miscount. State the rule you used in `stats.count_rule`.

## Severity calibration

- `critical` — a default that exposes a production system with no action by the operator (an admin interface bound to all interfaces with a fixed default password); a credential that the configuration system writes to a location or a log where security-scan confirms it leaks (report the mechanism, reference the leak).
- `high` — a safety or security switch that is **off** unless configured and whose absence is silent (sandboxing, TLS verification, authentication); configuration that silently falls back to an insecure or destructive mode when a value is missing or unparseable; a credential with a plaintext fallback that engages without telling the operator.
- `medium` — **a documented setting the code does not read** (the operator sets it, nothing happens, and no error says so — `high` when the tool also rewrites the file and deletes the key, because the operator's record of their own intent is destroyed); a malformed config file that is silently ignored so the program runs with unintended values; required values checked only at first use, so a misconfigured deployment fails long after start; the same setting with two different defaults in two places; a setting the code reads that no documentation mentions and whose wrong value is harmful; precedence that is undocumented and surprising (a lower-priority source winning); secrets that must be passed as CLI arguments (visible to `ps`) with no alternative.
- `low` — undocumented settings that are harmless; unknown keys silently ignored (typos become silent no-ops); stale documented settings the code no longer reads; dead feature flags whose branches are unreachable; defaults scattered across the codebase without a single source; settings requiring a restart with no indication of which ones.
- `info` — the source inventory, the precedence rules, the settings surface, the secret-source map, validation that works, the posture.
- Mitigations lower a finding one rung: the tool is single-user and interactive, the value is validated later on the same path, the insecure default is prominently documented and warned about at startup. When unsure between two levels, pick the lower.

## Output

Follow the core workflow: write `_sokrates/reports/ai-insights/configuration-scan.json`, validate until OK, render the explorer, re-merge if a combined report exists, report leading with the configuration model in two sentences (sources and precedence, where defaults live) and any above-info findings.

`stats` — the script's **facts** are counts of *shapes*, not of verified configuration mechanisms: read the sites behind any key before you copy it, and behind every key a finding will rest on. `precedence_sites`, `validation_sites`, `reload_sites` and `default_value_sites` are the ones that mislead — a high count can be entirely domain-data merges, UI refreshes or report defaults, and `precedence` is a mandatory slot, so an uninspected count there can produce a finding about a mechanism that does not exist. Copy the keys you verified (zeros included — 0 validation sites, 0 reload mechanisms are facts), name every key you dropped or corrected in `count_notes` with the true number where you established it, add `count_rule`, and on top:

- `config_sources` — ordered list, lowest precedence first, e.g. `["built-in defaults", "~/.codex/config.toml", "CODEX_* env vars", "-c CLI overrides"]`
- `config_files` — object of path pattern to format, e.g. `{"~/.codex/config.toml": "TOML", ".env": "dotenv"}`
- `settings_count` — integer or `"unknown"` with the reason in `count_notes`. Tie-breaks, so re-runs and scanners compare: count the keys of the **primary operator-facing schema**; a separate admin/managed schema is counted under `admin_settings_count`, feature flags under `feature_flags_count`, and profile keys are not counted again if they are a subset of the primary schema. State which schema you counted in `count_rule`.
- `settings_areas` — list of the areas used in the findings
- `env_var_prefixes` — list, e.g. `["CODEX_", "RUST_LOG"]`
- `precedence` — one of `documented`, `implicit`, `single-source`, `inconsistent`
- `defaults_location` — one of `single-source`, `schema`, `scattered`, `mixed`
- `validation` — one of `schema`, `partial`, `none`, with what is checked in `attributes` of the validation finding
- `on_invalid_config` — one of `fail-fast`, `warn-and-default`, `silent-default`, `mixed`
- `secret_sources` — list, e.g. `["env var", "auth.json key file", "interactive login"]`
- `reload` — one of `restart-required`, `watch`, `signal`, `partial`, `n/a`
- `undocumented_settings` / `stale_settings` — counts, with the names in the corresponding findings' `attributes`
- `absent_mechanisms` — list of whole mechanism classes this product does not have at all (`"secrets"`, `"feature flags"`, `"profiles"`, `"remote config"`), so a group with no findings is visibly *swept* rather than *missed*, and a re-run can see one appear