Property Mutation Testing Expert

---
name: property-mutation-testing-expert
description: "Property-based testing (fast-check) and mutation testing (Stryker) / Pengujian berbasis properti (fast-check) dan mutasi (Stryker)."
author: "Roedy Rustam"
version: "4.8.0"

---

# property-mutation-testing-expert — vibes-plug Skill

[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)

---

<a name="english"></a>
## English

### Orchestration & Integration
- **TDD & Test Runners**: `autonomous-tdd-debugger`, `e2e-testing-expert`
- **Formal Verification & Invariants**: `formal-spec-z3-verifier`, `spec-driven-development-expert`
- **Quality Gates & Anti-Slop**: `anti-slop`, `ai-code-review-autonomous`, `production-ready-hardener`, `zero-tech-debt-auditor`
- **Language Type Systems**: `typescript-expert`, `python-programming-expert`, `rust-programming-expert`
- **Lifecycle & Governance**: `brainstorming`, `zero-to-prod-orchestrator`, `session-memory-manager`

### Description
Production guide for two advanced testing techniques that frontier AI models unlock at unprecedented scale: (1) Property-Based Testing — generating thousands of randomized test inputs guided by algebraic properties and invariants, catching edge cases that example-based tests miss; and (2) Mutation Testing — systematically introducing bugs into the codebase and verifying that the test suite catches every one of them, measuring the actual "kill rate" of your tests. Together, these techniques expose false confidence in code coverage percentages.

### Trigger Conditions
Activate this skill when:
- Code coverage is high (>80%) but bugs still escape to production (false confidence in tests).
- Building financial, medical, or safety-critical logic where correctness is non-negotiable.
- The AI agent is generating test suites and needs to verify they actually catch real bugs.
- Implementing data transformations, serialization/deserialization, or mathematical computations.
- Auditing test quality during `production-ready-hardener` or `zero-tech-debt-auditor` phases.

---

### Core Concepts & Patterns

#### 1. Property-Based Testing with fast-check (TypeScript)

Instead of writing example-based tests (`expect(add(2, 3)).toBe(5)`), define universal properties that must hold for ALL inputs:

```typescript
import fc from 'fast-check';
import { describe, it, expect } from 'vitest';

describe('Array sorting properties', () => {
  it('output has the same length as input', () => {
    fc.assert(
      fc.property(fc.array(fc.integer()), (arr) => {
        const sorted = [...arr].sort((a, b) => a - b);
        expect(sorted.length).toBe(arr.length);
      })
    );
  });

  it('output is ordered (monotonically non-decreasing)', () => {
    fc.assert(
      fc.property(fc.array(fc.integer()), (arr) => {
        const sorted = [...arr].sort((a, b) => a - b);
        for (let i = 1; i < sorted.length; i++) {
          expect(sorted[i]).toBeGreaterThanOrEqual(sorted[i - 1]);
        }
      })
    );
  });

  it('output is a permutation of input (same elements)', () => {
    fc.assert(
      fc.property(fc.array(fc.integer()), (arr) => {
        const sorted = [...arr].sort((a, b) => a - b);
        const inputCounts = countElements(arr);
        const outputCounts = countElements(sorted);
        expect(outputCounts).toEqual(inputCounts);
      })
    );
  });

  it('sorting is idempotent (sorting twice = sorting once)', () => {
    fc.assert(
      fc.property(fc.array(fc.integer()), (arr) => {
        const sortOnce = [...arr].sort((a, b) => a - b);
        const sortTwice = [...sortOnce].sort((a, b) => a - b);
        expect(sortTwice).toEqual(sortOnce);
      })
    );
  });
});

function countElements(arr: number[]): Map<number, number> {
  const counts = new Map<number, number>();
  for (const item of arr) {
    counts.set(item, (counts.get(item) ?? 0) + 1);
  }
  return counts;
}
```

#### 2. Common Property Patterns

| Pattern | Property | Example |
| :--- | :--- | :--- |
| **Roundtrip** | `decode(encode(x)) === x` | JSON serialize/deserialize, URL encode/decode |
| **Idempotency** | `f(f(x)) === f(x)` | Sorting, normalization, deduplication |
| **Commutativity** | `f(a, b) === f(b, a)` | Set union, addition, merging configs |
| **Invariant** | `property(f(x))` always holds | Sorted array is ordered, hash length is constant |
| **Equivalence** | `f_optimized(x) === f_reference(x)` | Optimized vs. naive implementation |

#### 3. Mutation Testing with Stryker (TypeScript/JavaScript)

```json
// stryker.config.json
{
  "$schema": "https://raw.githubusercontent.com/stryker-mutator/stryker/master/packages/core/schema/stryker-core.schema.json",
  "mutate": ["src/**/*.ts", "!src/**/*.test.ts", "!src/**/*.spec.ts"],
  "testRunner": "vitest",
  "reporters": ["html", "progress", "dashboard"],
  "thresholds": {
    "high": 90,
    "low": 70,
    "break": 60
  },
  "concurrency": 4,
  "timeoutMS": 10000
}
```

**What mutation testing does:**

```
ORIGINAL CODE                    MUTANT (auto-generated)
─────────────                    ───────────────────────
if (age >= 18) {          →      if (age > 18) {           // boundary mutation
  return 'adult';                  return 'adult';
}                                }

price * quantity          →      price / quantity           // arithmetic mutation
price * quantity          →      price + quantity           // arithmetic mutation

items.length > 0          →      items.length >= 0          // relational mutation
items.length > 0          →      items.length < 0           // relational mutation
items.length > 0          →      false                      // boolean mutation
```

If your test suite **still passes** after a mutation, that mutant "survived" — meaning your tests have a gap.

#### 4. Stateful Model-Based Property Testing (`fc.commands` Ledger Invariant)

Verify stateful systems (financial ledgers, shopping carts, rate limiters) against a simplified reference model across arbitrary sequences of operations:

```typescript
import { describe, it, expect } from 'vitest';
import * as fc from 'fast-check';

class ProductionWallet {
  private balanceCents = 0;
  deposit(amountCents: number): void {
    if (amountCents <= 0) throw new Error('Deposit must be positive');
    this.balanceCents += amountCents;
  }
  withdraw(amountCents: number): boolean {
    if (amountCents <= 0 || amountCents > this.balanceCents) return false;
    this.balanceCents -= amountCents;
    return true;
  }
  getBalanceCents(): number {
    return this.balanceCents;
  }
}

interface WalletModel {
  expectedBalanceCents: number;
}

class DepositCommand implements fc.Command<WalletModel, ProductionWallet> {
  constructor(readonly amountCents: number) {}
  check = () => this.amountCents > 0;
  run(model: WalletModel, real: ProductionWallet): void {
    real.deposit(this.amountCents);
    model.expectedBalanceCents += this.amountCents;
    expect(real.getBalanceCents()).toBe(model.expectedBalanceCents);
  }
  toString = () => `Deposit(${this.amountCents})`;
}

class WithdrawCommand implements fc.Command<WalletModel, ProductionWallet> {
  constructor(readonly amountCents: number) {}
  check = () => this.amountCents > 0;
  run(model: WalletModel, real: ProductionWallet): void {
    const succeeded = real.withdraw(this.amountCents);
    if (this.amountCents <= model.expectedBalanceCents) {
      expect(succeeded).toBe(true);
      model.expectedBalanceCents -= this.amountCents;
    } else {
      expect(succeeded).toBe(false);
    }
    expect(real.getBalanceCents()).toBe(model.expectedBalanceCents);
    expect(real.getBalanceCents()).toBeGreaterThanOrEqual(0);
  }
  toString = () => `Withdraw(${this.amountCents})`;
}

describe('Stateful Wallet Conservation Property', () => {
  it('preserves non-negative integer balance across arbitrary command sequences', () => {
    const commandArb = fc.commands(
      [
        fc.integer({ min: 1, max: 500_000 }).map((n) => new DepositCommand(n)),
        fc.integer({ min: 1, max: 500_000 }).map((n) => new WithdrawCommand(n)),
      ],
      { maxCommands: 50 }
    );

    fc.assert(
      fc.property(commandArb, (cmds) => {
        const setup = () => ({
          model: { expectedBalanceCents: 0 },
          real: new ProductionWallet(),
        });
        fc.modelRun(setup, cmds);
      }),
      { numRuns: 250 }
    );
  });
});
```

#### 5. Mutation Score Targets

| Context | Minimum Kill Rate | Rationale |
| :--- | :--- | :--- |
| **Financial calculations** | ≥95% | Any surviving mutant = potential money loss |
| **Authentication/Authorization** | ≥95% | Security-critical logic |
| **Core business domain** | ≥85% | High-impact business logic |
| **Utility functions** | ≥80% | Moderate risk |
| **UI rendering logic** | ≥70% | Lower risk, visual regression tests supplement |

---

### Best Practices

1. **Properties Over Examples**: For any pure function, write property-based tests first. Example tests are supplementary, not primary.
2. **Target Mutation Score, Not Coverage**: 100% line coverage with 50% mutation score means half your tests are worthless.
3. **Use AI to Discover Properties**: Ask frontier models to identify algebraic properties of your functions that you may not have considered.
4. **Run Stryker on Critical Paths Only**: Mutation testing is computationally expensive. Target `src/domain/`, `src/auth/`, `src/billing/` — not the entire codebase.
5. **Shrink Failed Cases**: fast-check automatically shrinks failing inputs to the minimal reproducing case. Always include the shrunk value in bug reports.

---

### Common Pitfalls to Avoid

| Anti-Pattern | Consequence | Remedy |
| :--- | :--- | :--- |
| Relying solely on line coverage metrics | Tests pass with mutants alive (false confidence) | Run mutation testing to measure real test quality |
| Writing only example-based tests | Edge cases missed (empty arrays, negative numbers, Unicode) | Use property-based tests with random generation |
| Running mutation testing on entire codebase | Hours of CI time, developer frustration | Scope to critical domain and business logic paths |
| Ignoring surviving mutants | Known gaps in test quality left unaddressed | Treat surviving mutants as P1 bugs to fix |

### Hard Guardrails (Declarative Invariants)
- **Universal Algebraic Invariants Over Single Examples**: MUST verify pure domain and financial functions with `fc.assert(fc.property(...))` (roundtrip, idempotency, monotonicity, conservation) across at least `200` generated runs.
- **Explicit IEEE-754 `noNaN` Constraints**: MUST set `{ noNaN: true, noDefaultInfinity: true }` on `fc.double()` / `fc.float()` arbitraries so numeric property tests are deterministic.
- **Strict Mutation Break Thresholds**: MUST configure Stryker (`thresholds.break >= 75` overall and `>= 95` on billing/auth paths) so surviving critical-path mutants break CI.
- **Stateful Command-Sequence Verification**: MUST test stateful domain aggregates using `fc.commands` and `fc.modelRun` against a reference model to catch multi-step race or state-corruption bugs.
- **Minimal Counterexample Shrinking**: MUST preserve and log the minimal shrunk counterexample emitted by `fast-check` or Hypothesis as a permanent regression test.

### Sovereign Execution, Token Efficiency & Invariants
| Dimension | Sovereign Mandate & Deterministic Contract |
| :--- | :--- |
| **Frontier Model Cascade** | High Fast / High (`Grok 4.7 High Fast`, `Claude Sonnet 5.5 High`, `Gemini 4 Pro`) |
| **Availability Fallback** | `Gemini 4 Pro` -> `Gemini 3.8 Flash`; `Claude Opus 5.5` -> `Claude Opus 4.6`; `Grok 4.7` -> `Grok 4.6 High Fast` (Zero unhandled missing-model errors) |
| **KV-Cache & Token Protocol** | Pin static instructions & schemas at prompt prefix (`kv-cache-prefix-optimizer`); pass surgical `CONTEXT_MAP.md` slices (<250 tokens/call) during Swarm Fan-Out |
| **SaaT CLI & MCP Dispatch** | CLI: `vibes skill exec property-mutation-testing-expert <audit\|eval\|verify\|generate\|blueprint>` / MCP: `vibes_skill_dispatch`, `vibes_critic_validate`, `vibes_self_heal`, `vibes_eval_sandbox`, `vibes_verify_invariants` |
| **NFR & Anti-Slop Invariants** | Zero OWASP Top 10 vectors, 0 hardcoded secrets, >= 90% branch coverage on domain logic, 0 Anti-Slop violations; 100% complete zero-placeholder output verified via `node scripts/check-anti-slop.js --strict` |

---


---

<a name="bahasa-indonesia"></a>
## Bahasa Indonesia

### Integrasi Orkestrasi
- **TDD & Runner Pengujian**: `autonomous-tdd-debugger`, `e2e-testing-expert`
- **Verifikasi Formal & Invarian**: `formal-spec-z3-verifier`, `formal-theorem-verifier-lean4`, `spec-driven-development-expert`
- **Gerbang Kualitas & Anti-Slop**: `anti-slop`, `ai-code-review-autonomous`, `production-ready-hardener`, `zero-tech-debt-auditor`
- **Sistem Tipe Bahasa**: `typescript-expert`, `python-programming-expert`, `rust-programming-expert`
- **Siklus Hidup & Tata Kelola**: `brainstorming`, `zero-to-prod-orchestrator`, `session-memory-manager`

### Deskripsi
Panduan produksi untuk dua teknik testing lanjutan yang model frontier AI buka pada skala belum pernah ada: (1) Property-Based Testing — menghasilkan ribuan input tes acak yang dipandu oleh properti aljabar dan invarian, menangkap edge case yang tes berbasis contoh lewatkan; dan (2) Mutation Testing — secara sistematis menyisipkan bug ke codebase dan memverifikasi bahwa suite tes menangkap setiap bug, mengukur "kill rate" aktual tes Anda.

### Kondisi Pemicu
Aktifkan skill ini ketika:
- Coverage kode tinggi (>80%) tetapi bug masih lolos ke produksi (kepercayaan palsu pada tes).
- Membangun logika kritis (keuangan, medis, keamanan) di mana kebenaran tidak bisa dinegosiasikan.
- Agen AI menghasilkan suite tes dan perlu memverifikasi bahwa tes benar-benar menangkap bug nyata.

---

### Konsep Inti & Pola Praktik

#### 1. Property-Based Testing
Alih-alih menulis tes contoh, definisikan properti universal yang harus berlaku untuk SEMUA input: Roundtrip (`decode(encode(x)) === x`), Idempotency (`f(f(x)) === f(x)`), Commutativity (`f(a,b) === f(b,a)`), Invariant (properti selalu berlaku), dan Equivalence (implementasi optimal = implementasi referensi).

#### 2. Mutation Testing
Secara sistematis memperkenalkan mutasi kecil pada kode (mengubah `>=` menjadi `>`, `*` menjadi `/`, `true` menjadi `false`) dan memverifikasi suite tes mendeteksi setiap mutasi. Jika tes masih lolos setelah mutasi, ada celah di kualitas tes Anda.

#### 3. Target Skor Mutasi
Kalkulasi keuangan: ≥95%. Autentikasi/otorisasi: ≥95%. Domain bisnis inti: ≥85%. Fungsi utilitas: ≥80%.

---

### Praktik Terbaik

1. **Properti di Atas Contoh**: Untuk fungsi pure, tulis property-based tests lebih dulu.
2. **Targetkan Skor Mutasi, Bukan Coverage**: 100% line coverage dengan 50% mutation score berarti separuh tes Anda tidak berguna.
3. **Gunakan AI untuk Menemukan Properti**: Minta model frontier mengidentifikasi properti aljabar dari fungsi Anda.
4. **Jalankan Stryker pada Jalur Kritis Saja**: Mutation testing mahal secara komputasi.

### Matriks Eksekusi Berdaulat, Efisiensi Token & Invarian
| Dimensi | Mandat Berdaulat & Kontrak Deterministik |
| :--- | :--- |
| **Kaskade Model Frontier** | High Fast / High (`Grok 4.7 High Fast`, `Claude Sonnet 5.5 High`, `Gemini 4 Pro`) |
| **Invarian Fallback Ketersediaan** | `Gemini 4 Pro` -> `Gemini 3.8 Flash`; `Claude Opus 5.5` -> `Claude Opus 4.6`; `Grok 4.7` -> `Grok 4.6 High Fast` (Bebas gagal saat model belum tersedia/kuota penuh) |
| **Protokol Hemat Token & KV-Cache** | Kunci instruksi & skema statis di prefix awal (`kv-cache-prefix-optimizer`); kirim potongan `CONTEXT_MAP.md` (<250 token/panggilan) saat Fan-Out |
| **Eksekusi SaaT CLI & MCP** | CLI: `vibes skill exec property-mutation-testing-expert <audit\|eval\|verify\|generate\|blueprint>` / MCP: `vibes_skill_dispatch`, `vibes_critic_validate`, `vibes_self_heal`, `vibes_eval_sandbox`, `vibes_verify_invariants` |
| **Invarian NFR & Anti-Slop** | Nol celah OWASP Top 10, 0 kredensial hardcoded, cakupan pengujian >= 90% pada logika domain, 0 pelanggaran Anti-Slop; kode 100% tuntas tanpa placeholder diverifikasi melalui `node scripts/check-anti-slop.js --strict` |