Windows Qa

---
name: windows-qa
description: Build, install and verify a libi desktop build on the Windows QA VM. Use before shipping anything Windows-facing, or whenever a build must be put in front of a real Windows user. Encodes the traps that have each cost hours — npm ci vs npm install, CI=1, processes dying with the SSH session, what "installed" actually means, and the big one — the VM is an elevated administrator, so an elevated pass proves nothing about a real user.
---

# Windows QA — driving a test build onto the VM

Every rule here comes from a failure that happened. Do not relax one without
evidence.

**Connection details, the VM's name and its firewall rules are NOT in this
repo** — they are in `docs-local/release/windows-qa-runbook.md`, which also
carries the fuller narrative. Read that first; this file is the rule list.

**Start with "The VM is an ADMINISTRATOR" below.** It is the newest rule and the only one that has let a broken build reach users.

## Getting on the box, and putting it away

The VM is not left running between sessions, and it is not left *existing*
either. The end-of-session move snapshots it and deletes the VM, disk, NIC and
public IP, which drops the standing cost from ~$15/month to ~$3.

Two consequences that will bite you if you assume otherwise:

- **Start with `restore`, not `up`.** `up` builds a fresh box and re-runs an
  hour of provisioning; `restore` attaches the snapshot and comes back with the
  installed app, the app's data directory and the build tree exactly as they
  were left.
- **The public IP does not survive.** It is released on the way down and a new
  one is allocated on the way back. Never hardcode it, in a script or in a doc
  — resolve it, or use the lab wrapper's own ssh, which looks it up. The
  runbook named a fixed IP in three places and all three went stale the first
  time the lab was put away properly.

Reaching it is gated on a firewall that allows exactly one source address, so a
changed network — VPN on or off, a different wifi — makes the connection **time
out rather than refuse**. That is the firewall, not a dead VM, and re-pointing
it at your current address is a one-liner. Expect to need it after a restore.

Host details, the commands and the cost table are in
`docs-local/release/windows-qa-runbook.md`.

## Getting the code there

The VM has an **extracted tarball at `C:\build\src`, not a clone** — no `.git`,
so no git command works there. Ship with `git archive | gzip` → `scp` → untar.
Roughly 9 MB. No push is involved, so this is fine on any day and does not
touch the weekend-publishing rule.

Always read the provenance line the extract script prints. It is the only thing
between you and confidently building last week's code.

## Building

```powershell
npm ci --no-audit --no-fund
npm run build:electron -- --publish never
```

- **`npm ci`, NEVER `npm install`.** `npm install` rewrites
  `package-lock.json` on Windows, which makes `THIRD-PARTY-NOTICES.md` stale
  against it and correctly fails `prebuild:electron`. The gate is right; the
  install is wrong. Restore the committed lockfile and use `npm ci`.
- **Never set `CI=1`.** electron-builder then tries to publish to GitHub
  Releases and aborts on a missing `GH_TOKEN`, before producing an installer.
  `--publish never` states the intent.
- Budget ~11 min for `npm ci` and ~13 min for `build:electron`. Skip `npm ci`
  on a rebuild whose lockfile has not changed.
- **You usually should not hand-build here at all any more.** Since the release
  workflows were split (2026-08-28), `release-electron.yml` with
  `dry_run: true` builds the real macOS and Windows shells in CI and uploads
  their artifacts **without publishing anything**, on any day of the week. That
  gives you the actual release artifact — the thing users get — rather than a
  local approximation of it, so prefer downloading it from the run over
  building on this box. Hand-building is still the right move when you are
  testing an unpushed branch.
- The `dry_run` escape applies to the CI path only. Run
  `scripts/release-electron.js` **on a machine that configures a local window**
  (`scripts/release-window.local.json`) outside Fri/Sat and
  `assertReleaseWindow()` refuses, by design. Runners carry no such file, which
  is why the workflow's own window job is what enforces the rule there — and
  why a dry run is exempt from it.

## Installing

The silent install takes **~12 minutes**. Everything here follows from that.

- **Launch it as an S4U scheduled task.** A process started with
  `Start-Process` over SSH dies when the SSH session ends — one attempt left
  nothing installed at all because of this.
- **`Start-Process -Wait` hangs the SSH session** on the NSIS installer. Poll
  from separate short calls.
- **Never kill it for looking stuck.** Giving up after 3 minutes and killing it
  left every file copied but no Start-menu shortcut, no desktop shortcut and no
  uninstall registry entry — an app that cannot be launched and reads to the
  user as "it was removed".
- **The completion signal is the INSTALLER PROCESS EXITING — nothing else.**
  The exe, the Start-menu shortcut AND the uninstall registry entry are all
  written EARLY, before the installer extracts the app bundle, which is the
  bulk of the payload. Polling for any of them reports "installed" on a build
  whose packaged `.next` does not exist yet — and a provenance check against
  that comes back all zeros, looking exactly like a bad build.
- **Verify all five together, after the process exits:** no installer running,
  the exe, the shortcut, the registry entry, and `.next` present inside the
  bundle — the last being what the shell actually loads. Report readiness on
  that set, never on any single one.
- **Whether the installer launches the app when it finishes is NOT
  guaranteed.** It usually does — landing in session 0 with nowhere to render,
  but still creating its data directory, so clear that afterwards or the
  "fresh install" being handed over is not one. On the 0.1.16 run it did
  NOT: no `Libi.exe` process and no `%APPDATA%\Libi` existed right after the
  installer process exited. Check for the data directory rather than assuming
  either behaviour, and only clear it if it is actually there.

## Proving the build is the code you think it is

Search the INSTALLED tree, at
`<install>\resources\libi-bundle\node_modules\@nagellabs\libi\.next`:

- Use `Get-ChildItem … -Recurse -Filter *.js` piped to `Select-String`. A `**`
  glob handed to `Select-String -Path` matches nothing SILENTLY and reports a
  good build as empty.
- Search **both `static` and `server`**. Client components put their strings in
  `.next/static`; searching only `server` finds zero and looks like a bad build.
- Assert absences too — a testing-only flag that should be gone, and any
  simulation module that must never ship.
- **Match the needle to where the code actually lives.** The MCP server ships
  as SOURCE under the package's `mcp/` folder and is never compiled by Next, so
  a string from `mcp/server.ts`, `mcp/tools/*.ts`, `mcp/instructions-core.md`
  or `mcp/skills/**` can never appear in `.next`. A checker that searches only
  `.next` reports a perfectly good build as missing the fix — it happened on
  2026-09-17 and the false negative is indistinguishable from a real one.
- Any path filter must be relative to the PACKAGE. The package lives under
  `node_modules`, so excluding `\node_modules\` by full path excludes
  everything and reports zero files.

### Proving the `runAsNode` fuse holds (reusable check)

Electron's `runAsNode: false` fuse is what stops `Libi.exe` from being
invoked as a bare Node process — reachable to check WITHOUT a screen. Do this
under an S4U task (**never bare over SSH — see the warning below**), on any
build:

```powershell
$env:ELECTRON_RUN_AS_NODE = "1"
& "<install>\Libi.exe" C:\path\to\probe.js
Remove-Item Env:\ELECTRON_RUN_AS_NODE
```

Read the result the right way round — it is easy to get backwards, and
getting it backwards sends the next QA pass chasing the wrong cause:

- **The fuse HOLDS (the shipped, correct configuration) when NO marker/output
  from `probe.js` appears and the whole APP boots instead** — confirmed on the
  0.1.16 build via `electron-main-sync.log`: `module loaded … app.on ready …
  about to createSplash`, the ordinary boot sequence, not the probe script's
  output. This is EXACTLY why closing a terminal launches a second `Libi.exe`
  (F1/EL-3, pre-fix): node-pty's Windows kill path `fork`s a helper script
  under `ELECTRON_RUN_AS_NODE=1` expecting it to run as a harmless Node
  process, and with the fuse holding, Electron ignores that variable and runs
  the fork as the full app instead — refused by the single-instance lock
  (`second-instance: another launch was refused`). Seeing that log line after
  closing a terminal means the fuse is fine; it is a relaunch problem
  (EL-3's fix — `useConptyDll: true` — is what actually stops the fork), not a
  fuse regression.
- **The fuse is OFF (a security regression) when `probe.js`'s marker/output
  DOES appear.** That means the signed binary just ran ARBITRARY JavaScript
  because it was asked to — the real security problem this check exists to
  catch, on a build that should never allow it.

**Never run this bare over SSH.** With the fuse holding it boots the whole
app: that creates `%APPDATA%\Libi`, which destroys a prepared "genuine
first-run" snapshot, and leaves `Libi.exe` running after the SSH session
ends (the app is not killed with it, unlike a plain script). Launch it as an
S4U task, wait for it to finish, then kill `Libi.exe`/`node.exe` and check
whether `%APPDATA%\Libi` now exists (F1's `fusetest.ps1` does exactly this) —
clean up the same way the install section's reset does.

## Resetting to a first run

The app keeps everything under `%APPDATA%\Libi` — the database, the persisted
UI state and ~700 MB of downloaded binaries. **The uninstaller leaves it**, so
uninstall + reinstall drops you back into the app you already set up while
looking like a fresh install.

**A full uninstall is not fast either — budget ~11.5 minutes** (0.1.14,
40465 files removed as an S4U task, same reasoning as the install: launch it
that way, don't `-Wait` over SSH, poll from separate short calls). A naive
"is the app still there" PowerShell probe run in a loop can itself cost
~1 minute per check on this box, which stacks up and reads like the uninstall
is hung when it is the probing that is slow. Don't kill a long-running
uninstall on a hunch; verify with a cheap check (the install directory or the
uninstall registry entry gone) rather than a heavy one run repeatedly.

```powershell
# Fast: back at the first screen in seconds, keeps the downloads.
Remove-Item "$env:APPDATA\Libi\libi.sqlite","$env:APPDATA\Libi\Local Storage" -Recurse -Force

# Full: a genuine new user. Uninstall first; first boot re-downloads ~700 MB.
Remove-Item "$env:APPDATA\Libi" -Recurse -Force
```

## Seeing the UI

**You cannot.** Nobody is logged in, so a GUI app launched over SSH lands in
session 0, finds nowhere to render, and exits leaving no window and no logs —
which looks exactly like a crash and is not. RDP is the only way, and it needs
a password, so that step belongs to the operator. Say so plainly rather than
reporting a headless check as if it were a launch.

**When RDP "cannot connect", suspect the CLIENT before the box.** Windows App
keeps saved PCs forever, so its tile routinely points at a recycled IP from a
VM deleted weeks ago — and error `0x204` reads as "the PC is off" when it only
means "nothing answered". Rule the box out in two commands before touching
anything: `az network public-ip list -o table` for the real address, then an
X.224 negotiation probe (in the runbook) — a healthy box answers `NEG_RSP
protocol = 2`. Confirm from the other side too: with zero RdpCoreTS events
since boot and zero 4625s, the attempt never arrived and Windows is not the
problem. Prefer writing an `.rdp` file over editing Windows App's Core Data
store; if you must edit it, quit the app properly first (closing the window
does not), and keep the backup where a later copy cannot overwrite it.

Server-side paths can be exercised headlessly by running the packaged runtime
without Electron under an S4U task.

## The VM is an ADMINISTRATOR. Real users are not.

Added 2026-08-28, after this cost a shipped release.

`libiqa` is in `Administrators`, an SSH session on the box is **elevated**, and
its token holds `SeCreateSymbolicLinkPrivilege`. A real user's token does not:
UAC strips it from administrators and ordinary accounts never had it, unless
Developer Mode is on (it is off there, and off by default everywhere).

v0.1.8 shipped a Windows build that **did not start for any ordinary user** and
passed every QA pass on this box, because the whole product's boot depends on
creating a directory symlink and this account is allowed to. The app died
between two log lines with no error: a splash screen that never resolved.

So: **an elevated pass is not evidence for anything privilege-sensitive** —
symlinks, junctions, writes outside the user profile, service registration,
firewall rules. Re-run those under a genuinely unprivileged principal:

```powershell
schtasks /create /tn Probe /tr "C:\path\to\probe.exe" /sc once /st 00:00 ^
  /ru "NT AUTHORITY\LOCAL SERVICE" /f
schtasks /run /tn Probe
schtasks /query /tn Probe /v /fo LIST | findstr "Last Result"
```

LOCAL SERVICE needs no password, so this puts no credential on the box. Give it
a working directory it can actually read — it cannot see `libiqa`'s profile, so
copy any binary it needs to something like `C:\junctest\`.

**Do NOT use `runas /trustlevel:0x20000` for this.** It is a SAFER *restricted*
token, not a UAC-filtered one, and it fails in ways real users never see —
junctions it creates are structurally perfect and permanently unreadable, which
looked exactly like a second product bug and cost an hour to disprove.

**An S4U task run as `libiqa` with `-RunLevel Limited` (requesting no
elevation) is not a stand-in either.** Confirmed 2026-09-26: it still measured
High IL. The account is still `libiqa`, still an administrator — asking the
task not to elevate does not change which account it runs as. Only
`NT AUTHORITY\LOCAL SERVICE` (above) or an actual RDP session under a
genuinely non-administrator account is a real stand-in for an ordinary user.

The GUI half still needs a screen. Launching over SSH gives an elevated token,
so "it booted over SSH" says nothing about a real first run; do that check at
an RDP session.

## Two ways to hurt yourself on this box

**Never `icacls` a directory you are connected through.** Granting rights on
`%USERPROFILE%` (or its parents) to test something as another principal resets
the SSH session mid-command — sshd runs as that user. It happened on 2026-08-28,
left the ACLs half-applied and the app tree without its farm, and cost two
recovery passes. If you need another principal to reach the install, copy what
it needs to `C:\<something>` instead and grant there.

**Getting a CI artifact onto the box: hand it the signed URL, don't relay it.**
The artifacts API 302s to a signed Azure blob URL, and the VM is itself in
Azure — so it pulls 795 MB in about 25 seconds, against several minutes of
uplink if you download locally and `scp` it up.

```bash
curl -s -o /dev/null -w "%{redirect_url}" \
  -H "Authorization: token $(gh auth token)" \
  -H "Accept: application/vnd.github+json" \
  "https://api.github.com/repos/Nagellabs/libi/actions/artifacts/<id>/zip"
```

Write that URL to a file, `scp` the file (it is a few hundred bytes), and have a
`.ps1` on the box `Invoke-WebRequest` it. The URL is short-lived, so fetch it
immediately before use.

## Driving a setup/chat terminal over its WebSocket (a probe harness)

Exercising a Providers/Agents setup terminal or a chat terminal headlessly (no
RDP) means talking to `lib/terminal/ws-server.ts` directly rather than through
the editor UI. **Send the first `resize` only AFTER the snapshot frame
arrives, never at `open`.** A `resize` sent immediately on connect races the
server's own initial snapshot and is dropped; the terminal then never applies
the size the held input assumed, the command typed against it is never
actually typed into the PTY, and the `Enter` that was meant to submit it sits
as a pending keystroke with nothing to apply to. This cost one probe run on
2026-09-26. Wait for the snapshot message, THEN resize, THEN type, THEN send
`Enter`.

## Two habits that prevent most of the above

1. **Write a `.ps1`, `scp` it, run it with `-File`.** Quoting through
   bash → ssh → PowerShell mangles reliably. Keep those scripts **ASCII-only** —
   one em dash arrives as mojibake and breaks the parser.
2. **Trust the status file, not the SSH exit code.** A wrapper piping to `tail`
   exits 0 while the build inside it failed. Have every script write
   START/OK/FAIL/DONE lines to a status file, and read that.