Test Cli Application

---
name: test-cli-application
locale: caveman
source_locale: en
source_commit: 82c77053
fence_basis_commit: 82c77053
translator: "Julius Brussee homage — caveman"
translation_date: "2026-05-03"
description: >
  Write integration tests for a Node.js CLI application using the built-in
  node:test module. Covers the exec helper pattern, output assertions,
  filesystem state verification, cleanup hooks, JSON output parsing, error
  case testing, and state restoration after destructive tests. Use when
  adding tests to an existing CLI, testing a new command, verifying adapter
  behavior across frameworks, or setting up CI for a CLI tool.
license: MIT
allowed-tools: Read Write Edit Bash Grep Glob
metadata:
  author: Philipp Thoss
  version: "1.0"
  domain: cli
  complexity: intermediate
  language: TypeScript
  tags:
    - cli
    - testing
    - nodejs
    - node-test
    - integration
---

# Test CLI Application

Write integration tests for Node.js CLI using built-in `node:test` module with `execSync`.

## When Use

- Add tests to existing CLI application
- Test newly created command
- Verify adapter/plugin behavior across target frameworks
- Set up CI that validates CLI correctness
- Catch regressions after refactoring CLI internals

## Inputs

- **Required**: Path to CLI entry point (e.g., `cli/index.js`)
- **Required**: Commands to test
- **Optional**: Framework adapters to test (dry-run mode)
- **Optional**: Cleanup requirements (files/symlinks created by tests)

## Steps

### Step 1: Set Up Test Infrastructure

```javascript
import { describe, it, before, after } from 'node:test';
import assert from 'node:assert/strict';
import { execSync } from 'child_process';
import { existsSync, rmSync } from 'fs';
import { resolve } from 'path';

const CLI = 'node cli/index.js';
const ROOT = process.cwd();

function run(args) {
  return execSync(`${CLI} ${args}`, {
    cwd: ROOT,
    encoding: 'utf8',
    timeout: 10000,
  });
}
```

Key design decisions:
- `node:test` is built-in — no test runner dependency needed
- `execSync` runs the CLI as a subprocess — tests the actual binary, not internal functions
- 10-second timeout prevents hanging on interactive prompts
- `encoding: 'utf8'` gives string output for regex matching
- All paths relative to `ROOT` for reproducibility

**Got:** Test file imports from `node:test`, has working `run()` helper.

**If fail:** `node:test` not available? Node.js version below 18. Upgrade or use polyfill.

### Step 2: Write Smoke Tests

Smoke tests verify CLI starts, parses arguments, produces expected output shapes:

```javascript
describe('meta', () => {
  it('shows version', () => {
    const out = run('--version');
    assert.match(out, /\d+\.\d+\.\d+/);
  });

  it('shows help with all commands', () => {
    const out = run('--help');
    assert.match(out, /install/);
    assert.match(out, /list/);
    assert.match(out, /detect/);
  });
});

describe('registry', () => {
  it('list shows expected counts', () => {
    const out = run('list --domains');
    assert.match(out, /\d+ domains/);
  });

  it('search finds known items', () => {
    const out = run('search "docker"');
    assert.match(out, /result\(s\) for "docker"/);
  });

  it('search returns 0 for nonsense', () => {
    const out = run('search "xyzzy-nonexistent"');
    assert.match(out, /0 result/);
  });
});
```

Smoke test patterns:
- `--version` and `--help` always work
- Registry loading validates data integrity
- Search with known and unknown terms

**Got:** Smoke tests confirm CLI functional and data loaded.

**If fail:** Registry counts change frequent? Use `\d+` instead of hardcoded numbers.

### Step 3: Write Lifecycle Tests

Lifecycle tests verify create → verify → delete sequences with cleanup:

```javascript
describe('install', () => {
  const testPath = resolve(ROOT, '.agents/skills/commit-changes');

  after(() => {
    // Always clean up, even if tests fail
    try { rmSync(testPath); } catch {}
    try { rmSync(resolve(ROOT, '.agents/skills'), { recursive: true }); } catch {}
    try { rmSync(resolve(ROOT, '.agents'), { recursive: true }); } catch {}
  });

  it('dry-run does not create files', () => {
    const out = run('install commit-changes --dry-run');
    assert.match(out, /DRY RUN/);
    assert.ok(!existsSync(testPath));
  });

  it('installs creates the target', () => {
    run('install commit-changes');
    assert.ok(existsSync(testPath));
  });

  it('skips already installed', () => {
    const out = run('install commit-changes');
    assert.match(out, /skipped/);
  });

  it('uninstall removes the target', () => {
    run('uninstall commit-changes');
    assert.ok(!existsSync(testPath));
  });
});
```

Cleanup rules:
- Use `after()` hooks, not `afterEach()` — lifecycle tests build on each other
- Wrap cleanup in `try/catch` — cleanup must not fail the test suite
- Clean from leaf to root (file → parent dir → grandparent dir)
- If the test modifies shared state (symlinks, config files), restore it

**Got:** Tests run in sequence within describe block, cleanup runs even on failure.

**If fail:** Tests run in parallel (non-default in node:test)? Force sequential with `{ concurrency: 1 }`.

### Step 4: Write Dry-Run Tests for Each Adapter

Test each adapter's target path without making changes:

```javascript
describe('adapter: cursor (dry-run)', () => {
  it('targets .cursor/skills/ path', () => {
    const out = run('install commit-changes --framework cursor --dry-run');
    assert.match(out, /\.cursor\/skills/i);
  });
});

describe('adapter: copilot (dry-run)', () => {
  it('targets .github/ path', () => {
    const out = run('install commit-changes --framework copilot --dry-run');
    assert.match(out, /\.github/i);
  });
});
```

This pattern scales to any number of adapters. Each test:
- Uses `--framework` to bypass auto-detection
- Uses `--dry-run` so no files are created
- Asserts the target path appears in output

**Got:** One describe block per adapter, each with at least path assertion.

**If fail:** Adapter doesn't exist in project? Test will fail with "Unknown framework." Correct — adapter tests should only exist for implemented adapters.

### Step 5: Write Error Case Tests

```javascript
describe('errors', () => {
  it('rejects unknown items', () => {
    assert.throws(
      () => run('install nonexistent-skill-xyz'),
      /No matching items|Unknown/,
    );
  });

  it('rejects unknown framework', () => {
    assert.throws(
      () => run('install commit-changes --framework nonexistent'),
      /Unknown framework/,
    );
  });

  it('handles missing state gracefully', () => {
    assert.throws(
      () => run('scatter nonexistent-team'),
      /not burning|Unknown/,
    );
  });
});
```

Error testing patterns:
- `assert.throws` catches non-zero exit codes from `execSync`
- Regex match on the error message (captured from stderr)
- Test both "item not found" and "invalid option" errors
- Verify error messages suggest corrective actions

**Got:** All error paths produce non-zero exit codes and helpful messages.

**If fail:** `execSync` throws on non-zero exit. Error's `stderr` or `stdout` contains message. Check `error.stdout` if `assert.throws` regex doesn't match.

### Step 6: Write JSON Output Tests

```javascript
describe('json output', () => {
  it('campfire --json outputs valid JSON', () => {
    const out = run('campfire --json');
    const data = JSON.parse(out);
    assert.ok(typeof data.totalTeams === 'number');
    assert.ok(Array.isArray(data.fires));
  });

  it('gather --dry-run --json outputs structured data', () => {
    const out = run('gather tending --dry-run --json');
    // JSON may follow a DRY RUN header — extract from first '{'
    const jsonStart = out.indexOf('{');
    assert.ok(jsonStart >= 0, 'Should contain JSON');
    const data = JSON.parse(out.slice(jsonStart));
    assert.equal(data.team, 'tending');
  });
});
```

JSON testing gotchas:
- Some commands prefix JSON with human-readable text (e.g., DRY RUN header)
- Extract JSON by finding the first `{` character
- Validate structure (key presence, types), not exact values
- Values like counts may change as content is added

**Got:** JSON output parseable, contains expected keys.

**If fail:** `JSON.parse` fails? Command may be mixing human text with JSON. Either fix command to output pure JSON in `--json` mode, or extract JSON substring.

### Step 7: Handle Cleanup and State Restoration

```javascript
describe('stateful commands', () => {
  const stateDir = resolve(ROOT, '.agent-almanac');

  after(() => {
    // Remove state file created by tests
    try { rmSync(stateDir, { recursive: true }); } catch {}
  });

  // Tests that create/modify state...
});

// Restore symlinks that destructive tests may remove
describe('destructive tests', () => {
  after(() => {
    // Restore symlinks that scatter/uninstall removed
    const skills = ['heal', 'meditate', 'remote-viewing'];
    for (const skill of skills) {
      const link = resolve(ROOT, `.claude/skills/${skill}`);
      if (!existsSync(link)) {
        try {
          execSync(`ln -s ../../skills/${skill} ${link}`, { cwd: ROOT });
        } catch {}
      }
    }
  });
});
```

State restoration rules:
- State files (`.agent-almanac/state.json`) must be cleaned after tests
- Symlinks removed by `scatter`/`uninstall` must be restored
- Manifest files (`agent-almanac.yml`) created by `init` must be removed
- Order: `after()` hooks run in reverse declaration order — declare restore hooks last

**Got:** Test suite leaves project in same state it found it.

**If fail:** CI reports leftover files after test runs? Add cleanup to `after()`. Use `git status` after test runs to detect leaked state.

## Checks

- [ ] Test file runs with `node --test cli/test/cli.test.js`
- [ ] All tests pass (0 failures)
- [ ] Smoke tests cover `--version`, `--help`, registry loading
- [ ] Lifecycle tests verify create → verify → delete with cleanup
- [ ] At least one adapter dry-run test exists per implemented adapter
- [ ] Error cases test non-zero exit codes with message matching
- [ ] JSON output tests parse actual output (not mocked)
- [ ] After hooks restore all state modified by tests

## Pitfalls

- **Hardcoded counts that break**: Registry totals change as content added. Use `\d+` regex or read count dynamic instead of asserting `329 skills`.
- **Tests depend on execution order**: `node:test` runs suites in declaration order default, but tests within suite may not. Use lifecycle suites (create → verify → delete) within single `describe` to guarantee order.
- **Missing cleanup on test failure**: Test fails mid-lifecycle, `after()` still runs. But throw in `before()`? Subsequent tests and `after()` may not run. Keep `before()` minimal.
- **Interactive prompts hang tests**: Commands with confirmation prompts will hang `execSync`. Either pipe `echo y |` or ensure `--yes` always passed in tests.
- **Test with real installs in CI**: Tests that create files in `.claude/skills/` or `.agents/skills/` modify working tree. CI may fail on "dirty working directory" checks. Always clean up.

## See Also

- `scaffold-cli-command` — build commands that these tests verify
- `build-cli-plugin` — build adapters tested in Step 4
- `design-cli-output` — output patterns that tests assert against