Pre Pr Validation

---
name: pre-pr-validation
description: Use before pushing an npa change to pick which gates apply and run them locally in cheapest-first order — the map from each CI job to its exact local command.
---

# Pre-PR Validation

Every pull request has one automatic candidate workflow. PRs
run the complete eight-shard Python 3.12 coverage suite, Cypress, focused
Python 3.10/3.14 compatibility checks, lint, docs drift, guardrails, security
regressions, secret scanning, and confidentiality scanning. Cross-subsystem
coverage must pass before queue admission. Full shards include smoke tests;
the browser job runs the Python 3.12 CLI install check before compatibility
tests, without duplicate subsystem jobs. Trusted test selection shares the
secret-scanning runner so it adds no separate scheduling dependency. Only
narrowly recognized prose edits skip runtime suites; those retain smoke and every documentation,
repository, and security gate. See `CONTRIBUTING.md` for the trusted-base selector,
which uses the base's merge-candidate policy for both events during rollout.
The queue verifies successful, less-than-24-hour PR evidence for the identical
combined Git tree using trusted-base `ci_queue_evidence.py`, then reruns fresh
secret, confidentiality and source/dependency scans. Changed combined trees rerun
all tests, lint, guardrails and hostile-input checks; image checks rerun when the
trusted base policy identifies changed image inputs. Missing, failed or stale
evidence restores the full queue gate, which must pass before merging. Older
PRs can adopt the policy without a forced branch refresh.
`pr-precheck` gives a five-minute early signal without replacing full admission.
The queue execution target is ten minutes; hosted-runner waits can add delay.
A daily audit covers all supported Python versions instead of starting a full
audit after every merge. Lint, docs drift, and guardrails remain candidate gates
and can be dispatched manually, without duplicate main-push runs competing with
the next queue entry. The daily full suite includes guardrails; post-merge
security audits remain automatic. Independent validation jobs use available
GitHub runner
capacity without job-level concurrency locks or matrix `max-parallel` caps.
Preserve per-PR supersession on the parent and distinct workflow group prefixes
for reusable children. Merge candidates use their own SHA-specific groups.
The aggregate gate uses `!cancelled()` to report failed dependencies while
allowing obsolete runs to terminate; `always()` can keep their final job queued
ahead of the replacement candidate.
`test_ci_concurrency` rejects shared validation locks and matrix caps;
`test_ci_workflows` guards cancellation and required results. See the contributor
concurrency guide for organization runner limits and rollout behavior. Refresh
older PR branches after a scheduling change lands; rerunning an old commit
retains its original workflow configuration.
`Security regression / security-regression` requires
every PR component and image-security validation, or verified identical-tree
evidence plus fresh security scans for queue candidates. Changed combined trees
require fresh test results and any affected image checks.
The required workflows have no top-level path filters. Image scope is classified
inside always-reporting jobs: image, packaging, workflow, or security-policy
changes run the deep checks; unrelated source changes take the fast path. Main,
scheduled, and manual image audits always run deeply. Verify actual required
contexts in branch protection before claiming merge enforcement.

All of them are reproducible locally. Run them in cost order so the cheap ones
catch the common mistakes before you spend minutes on the full suite.

For how the test suite itself behaves — markers, hermetic fixtures, parallel
runs, live-infra expectations — use `skills/atomic/testing-conventions/SKILL.md`.
This skill is the gate map.

## Use The Repo Virtualenv

`npa/.venv/bin/python` (Python 3.12). Never bare `python`. The `make` targets
select that virtualenv automatically and use its absolute path before changing
into `npa/`. Override `PYTHON` only for a different environment:

```bash
make test PYTHON=/workspace/npa/.venv/bin/python
```

If that venv is shared across checkouts (worktree, agent sandbox) rather than
installed fresh in this one, `make test`/`test-smoke`/`test-guardrails` fail
fast via `make check-env` before running anything, rather than silently
testing a different checkout's code — see `testing-conventions` for why.

## The Ladder

Start with `make precheck` for dependency fingerprints, lint, formatting, and
focused CI contracts. It reads the working tree and fails before expensive tests.
Before pushing committed work, run `git fetch origin main` and
`make merge-precheck` to catch combined-tree conflicts and stale CI fingerprints
without altering your checkout or index. This second command checks committed
HEAD only; neither command replaces the full Linux or security gates.

```bash
# 1. Lint — seconds. This matches CI and `make lint` across all of npa/.
npa/.venv/bin/python -m ruff check npa

# 2. Onboarding smoke — ~20s.
make test-smoke PYTHON=/workspace/npa/.venv/bin/python

# 3. Guardrails — ~50s, ~2000 static contract assertions. Highest signal per second.
npa/.venv/bin/python -m pytest npa/tests/guardrails -q

# 4. The tests for what you touched — seconds to a minute.
npa/.venv/bin/python -m pytest npa/tests/cli/test_<tool>_cli.py -q

# 5. Full unit suite — minutes. This is the PR gate.
make test PYTHON=/workspace/npa/.venv/bin/python

# 6. Docs drift — ~1-2 min. Only when CLI commands or options changed.
bash scripts/build_docs.sh --check

# 7. Confidentiality, diff-scoped — seconds.
npa/.venv/bin/python -m npa.guardrails.confidentiality \
  --repo-root . --diff-range origin/main..HEAD --built-in-nebius-infra
```

Steps 1 through 3 catch most failures and cost about a minute together. Run them
after every meaningful edit; save 5 and 6 for before you push.

## Which Gates Apply To Your Change

| You changed | Also run |
|---|---|
| A CLI command or any of its options | `bash scripts/build_docs.sh --check`, plus the argv guardrails — a renamed option breaks every `toolRef` that passes it |
| A `toolRef` or the catalog | `test_tool_catalog_argv`, `test_module_toolref_argv`, `test_shown_workflow_catalog`, and the catalog doc sync test |
| An `npa.workflow` spec | `npa workbench workflow validate-spec <path>`, plus live-matrix registration per the testing conventions |
| A workbench tool's surface | `test_three_tier_contract`, and the tool's CLI and workbench tests |
| A Dockerfile or image tag | `npa/.venv/bin/python npa/docker/workbench/check_tag_consistency.py`, plus `npa/tests/docker/` |
| The complete-byte scanner, its dependencies, or publication wiring | `npa/tests/docker/test_image_byte*.py`, plus the actual native integration sequence in `.github/workflows/image-security-scan.yml`; hermetic protocol tests do not replace that native gate |
| A `SKILL.md` or `skills/index.yaml` | `test_skills_index` and `test_develop_skills` |
| Security gate code, scanner pins, or its workflow | Follow `docs/security/merge-security-gate.md`: run the actual scanner regression workload and full base/candidate comparison, plus `test_security_source` and `test_security_gate` |
| Terraform or the agent deploy path | `test_terraform_provisioner_shell`, plus a real destroy/deploy cycle |
| Docs only | Lint and `pytest --collect-only` as a smoke check |

## Gate Details Worth Knowing

### Required Security Regressions

The security job exercises hostile inputs and supported paths, including real
CPU checkpoint decoding, authenticated transports, private staging, storage
containment, and isolated controller cleanup. Use the clone's own development
virtualenv with `npa[dev,adapter]`. CI additionally installs the official CPU
`torch==2.14.0` wheel and asserts the version and CPU runtime before testing, so
checkpoint cases cannot silently skip. Its exact test command from the repo root
is:

```bash
npa/.venv/bin/python -m pytest \
  npa/tests/guardrails/test_image_security_gate.py \
  npa/tests/clients/test_download_containment.py \
  npa/tests/clients/test_ssh_private_staging.py \
  npa/tests/cli/test_agent_source_archive.py \
  npa/tests/cli/test_service_credential_security.py \
  npa/tests/cli/test_fiftyone_env_security.py \
  npa/tests/cli/test_fiftyone_private_access.py \
  npa/tests/workbench/test_checkpoint_security.py \
  npa/tests/workbench/test_s3_tree_security.py \
  npa/tests/workbench/test_storage_scope.py \
  npa/tests/workbench/test_policy_container_security.py \
  npa/tests/workbench/test_alpamayo2_super_service_security.py \
  npa/tests/workbench/test_cosmos3_ray_inputs.py \
  npa/tests/workbench/test_cosmos3_ray_serve.py \
  npa/tests/workbench/test_cosmos3_nano_video_server.py \
  npa/tests/workflows/test_paidf_download_security.py \
  npa/tests/workflows/test_openpi_gcs.py \
  npa/tests/workflows/test_rerun_serve_auth.py \
  npa/tests/orchestration/skypilot/test_controller_clone.py \
  npa/tests/orchestration/skypilot/test_local_api.py \
  npa/tests/orchestration/skypilot/test_controller_identity_transaction.py \
  -q
```

Run this gate before pushing, in addition to the full suite and applicable live
workload validation. Only the unified parent belongs in
`AUTOMATIC_PR_WORKFLOWS` in `npa/tests/guardrails/test_ci_workflows.py`; component
workflows are reusable, with separate main/scheduled security audits. The image workflow is covered by
`test_image_security_gate`;
preserve its distinct concurrency group, minimal caller permissions, and the
existing PR concurrency controls. Image findings must not produce a passing
`security-regression` result.

**`make test` is not identical to CI.** It deselects live and GPU markers and
sets a 180s timeout; CI runs with coverage and enforces `--cov-fail-under=60`.
A local pass is a strong signal, not proof of the CI result.

Run the equivalent coverage floor from the package directory. Merge-queue CI
uses eight deterministic, duration-balanced shards; the daily three-interpreter
audit retains four per interpreter. Both merge their coverage data:

```bash
cd npa
.venv/bin/python -m pytest tests/ -v --tb=short --cov=src/npa --cov-report=term-missing --cov-fail-under=60
```

Dependabot groups daily version updates across Python, npm, and GitHub Actions
into one `dependencies` PR. Review overlapping package declarations together;
a bot edit of `npa/ci/requirements.txt` does not prove its input fingerprint is
current. Regenerate the CI pins after Python declaration changes and run the
combined candidate through the same required gates.

Keep the coverage source scoped to `src/npa`. Selecting the import name with
`--cov=npa` can also trace temporary test modules that deliberately impersonate
that package, and those files no longer exist when CI merges shard data. From the
repository root, `--cov=npa` can additionally select the enclosing directory and
include tests and scripts in the coverage total. That percentage does not prove
package coverage. Check the report's file population, including package modules
that no test executed. If correcting a report from retained traces, preserve the
original data, add no executed lines, and distinguish the report correction from
a new test run.

**Some local failures are missing host tools, not broken code.** The suite shells
out to real binaries that CI has and a bare workstation or container may not.
Before investigating a failure, check whether it is one of these:

- `Required executable not found: kubectl` in `npa/tests/test_provisioning.py`
  means `kubectl` is not installed. Install it or accept the skip.
- `FileNotFoundError: 'npa'` in `npa/tests/workflows/test_bdd100k_pipeline.py`
  means the `npa` console script is not on `PATH`. Fix it by prepending the
  venv: `PATH="$PWD/npa/.venv/bin:$PATH"`.

When a failure looks unrelated to your change, confirm it against a clean base
before spending time on it. `git worktree add` never brings an untracked
`npa/.venv` (it is gitignored), so the new worktree needs one of:

```bash
git worktree add /tmp/main-check origin/main

# Option A: this worktree's own venv (works standalone, costs an install).
python3 -m venv /tmp/main-check/npa/.venv
/tmp/main-check/npa/.venv/bin/pip install -e "/tmp/main-check/npa[dev,adapter]"
/tmp/main-check/npa/.venv/bin/python -m pytest /tmp/main-check/npa/tests/<the failing test> -q

# Option B: reuse this checkout's already-installed venv, corrected with
# PYTHONPATH so it resolves `main-check`'s source, not this checkout's
# (see "Use The Repo Virtualenv" above for why the correction is required).
PYTHONPATH=/tmp/main-check/npa/src npa/.venv/bin/python -m pytest \
  /tmp/main-check/npa/tests/<the failing test> -q
```

The shared dev/operator VM has both of those binaries, so the full suite passes
there with nothing deselected. It is also shared, so take an isolated worktree +
venv + tmux session rather than checking out in the common clone:

```bash
npa/scripts/dev_vm_isolated_session.sh start <branch> <run-id>
```

Give that session its own extras — `pip install -e "npa[dev,adapter]"` — before
running the suite, and never install into the shared venv. Do not set
`NPA_ISOLATED_FAST=1` for a full-suite run: fast mode skips the per-run venv and
borrows the shared one, which lacks those extras, so collection dies on
`jsonschema` and `av` before a single test executes.

**Docs drift is expensive to re-run blind.** Change all CLI options first, then
regenerate once with `bash scripts/build_docs.sh`. Per-subcommand option changes
do not alter the generated pages; a new top-level command adds a page and a
README index line.

**The confidentiality scan has two modes, and only one is reproducible locally.**
CI scans with an operator denylist supplied as a repository secret, which you do
not have. The built-in Nebius pattern set needs no secret and covers the
deterministic leak classes, so it is the local substitute — not the same check.

Scope it to your diff:

```bash
# Your change only — this is the one that must be clean.
npa/.venv/bin/python -m npa.guardrails.confidentiality \
  --repo-root . --diff-range origin/main..HEAD --built-in-nebius-infra
```

Do not run the built-in patterns over `--tree` and read the result as a verdict
on your change. That scan exits non-zero on this repository today, with dozens
of hits in long-standing files — including `.gitleaks.toml`, which necessarily
contains the patterns being searched for. Those are allowlisted in CI. A tree
scan tells you nothing about whether you introduced a leak; the diff scan does.

**Gitleaks needs full history.** CI checks out with `fetch-depth: 0` and scans
the PR range. Locally: `gitleaks detect --source . --config .gitleaks.toml
--redact --no-banner --log-opts="origin/main..HEAD"`.

**Typecheck is advisory.** The mypy workflow is manual-dispatch only and does not
block.

## Interpreting A Failure

Guardrail failures map to fixes through
`skills/atomic/guardrail-failures/SKILL.md`. Report numeric results — pass
counts and exact error messages — rather than a subjective assessment; the repo
convention is evidence-based convergence.