Kv Cache Prefix Optimizer

---
name: kv-cache-prefix-optimizer
description: "Deterministic prompt prefix engineering & KV-cache hit optimization / Rekayasa prefix prompt deterministik & optimasi cache KV."
author: "Roedy Rustam"
version: "4.8.0"

---

# kv-cache-prefix-optimizer — vibes-plug Skill

[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)

---

<a name="english"></a>
## English

### Orchestration & Integration
- **Upstream & Orchestration**: `zero-to-prod-orchestrator` (Phase 4 AI agent integration), `brainstorming`, `session-memory-manager`.
- **Frontier LLM & Model Caching**: `frontier-ai-models-expert` (Gemini 4 Argon/Pro, Claude 5.5 Sonnet/Opus, Claude Fable 5.1, GPT-5.6 Sol, Grok 4.7 caching specs), `gemini-agent-booster` (Google 2M+ native context caching), `vercel-ai-sdk-expert` (provider cache controls).
- **FinOps & Adaptive Execution**: `llm-finops-router` (cost calculation & cache routing), `adaptive-model-cascade` (cascade tier prefixing), `thinking-budget-orchestrator` (prefix pinning with thinking tokens).
- **Prompt Engineering & Context Windows**: `ai-prompt-engineering-expert` (prompt canonicalization), `context-window-engineer` (large context sliding windows).

### Description
Production guide for engineering deterministic prompt prefixes, stabilizing KV-cache boundaries, and optimizing context reuse across late-2026 frontier models (Google Gemini 4 Argon / Pro context caching, Anthropic Claude 5.5 / 5.1 prompt caching, OpenAI GPT-5.6 Sol prefix caching, and xAI Grok 4.7). Delivers architectures that guarantee 90%+ cache hit rates through canonical tool definition sorting, stable JSON serialization, and dynamic context partitioning.

### Trigger Conditions
Activate this skill when:
- Operating large-context agents (>100k tokens) where recurring prompt costs dominate operational budgets.
- Experiencing high Time-to-First-Token (TTFT) latencies in interactive chat or agentic coding loops.
- Configuring model caching primitives (`cache_control: { type: 'ephemeral' }` or Gemini explicit cache tokens).
- Architecting system instructions, tool schemas, and workspace context maps for multi-turn sessions.

---

### Core Concepts & Patterns

#### 1. The Anatomy of a High-Cache Prompt
Modern frontier models utilize prefix-matching algorithms. Even a single changed byte (such as a timestamp, random ID, or altered tool order) invalidates all subsequent cached tokens:

```
[STATIC PREFIX: CACHE PINNED (90%+ HIT RATE)]
  ├── System Directives & Behavioral Guardrails (Immutable)
  ├── 171-Skill Ecosystem Registry (Canonical Alphabetical Order)
  ├── Tool Definitions (Strictly Sorted Keys & Parameters)
  └── Workspace Schema & Architecture Map (Immutable per Session)
------------------------ [CACHE BOUNDARY PIN] ------------------------
[DYNAMIC SUFFIX: VOLATILE PER TURN]
  ├── Ephemeral Turn Timestamp & User Message
  └── Real-Time Tool Execution Results
```

#### 2. Deterministic Prompt & Cache Pinning (TypeScript Implementation)

```typescript
export interface ToolDefinition {
  name: string;
  description: string;
  parameters: Record<string, unknown>;
}

export function serializeCanonicalJson(obj: unknown): string {
  if (obj === null || typeof obj !== 'object') {
    return JSON.stringify(obj);
  }
  if (Array.isArray(obj)) {
    return `[${obj.map((item) => serializeCanonicalJson(item)).join(',')}]`;
  }
  const sortedKeys = Object.keys(obj as Record<string, unknown>).sort();
  const entries = sortedKeys.map(
    (key) => `${JSON.stringify(key)}:${serializeCanonicalJson((obj as Record<string, unknown>)[key])}`
  );
  return `{${entries.join(',')}}`;
}

export function sortToolsCanonically(tools: ToolDefinition[]): ToolDefinition[] {
  return [...tools].sort((a, b) => a.name.localeCompare(b.name));
}

export interface PromptPayload {
  systemPrompt: string;
  cachedContextBlock: {
    type: 'text';
    text: string;
    cache_control: { type: 'ephemeral' };
  };
  messages: Array<{ role: 'user' | 'assistant'; content: string }>;
}

export function constructCacheOptimizedPayload(
  systemDirectives: string,
  staticWorkspaceContext: string,
  tools: ToolDefinition[],
  userTurns: Array<{ role: 'user' | 'assistant'; content: string }>
): PromptPayload {
  const sortedTools = sortToolsCanonically(tools);
  const canonicalToolSpecs = serializeCanonicalJson(sortedTools);

  const staticPinnedBlock = [
    '=== IMMUTABLE SYSTEM CONTEXT ===',
    systemDirectives,
    '=== CANONICAL TOOL REGISTRY ===',
    canonicalToolSpecs,
    '=== PROJECT ARCHITECTURAL MAP ===',
    staticWorkspaceContext
  ].join('\n\n');

  return {
    systemPrompt: 'You are an advanced sovereign AI coding agent operating under strict anti-slop guidelines.',
    cachedContextBlock: {
      type: 'text',
      text: staticPinnedBlock,
      cache_control: { type: 'ephemeral' }
    },
    messages: userTurns
  };
}
```

#### 3. Cache Performance & Telemetry Targets

| Metric | Unoptimized Baseline | KV-Cache Optimized Target | Improvement Factor |
| :--- | :--- | :--- | :--- |
| **Cache Hit Ratio** | 0% - 20% | **90% - 98%** | **4.5x - 5x higher** |
| **Time-to-First-Token (TTFT)** | 3,200ms - 5,500ms | **250ms - 450ms** | **~10x faster response** |
| **Prompt Token Cost** | 100% full rate | **20% - 25% discounted rate** | **75% - 80% cost reduction** |

---

### Best Practices

1. **Never Insert Timestamps in System Prompts**: Place dates or timestamps exclusively in the final volatile user message, never in the static system prompt.
2. **Sort Tool Definitions Alphabetically**: Always sort tool names and their JSON schema parameter keys canonically before sending requests.
3. **Partition Dynamic Context to the Tail**: Ensure all large documentation and code context maps appear before volatile conversation history.
4. **Monitor Cache Read Discrepancies**: Log `usage.prompt_tokens_details.cached_tokens` in application telemetry to catch accidental prefix cache busts immediately.
5. **Cache Prefix Portability & Availability Fallback**: Preserve identical system prefix structures across model fallback pairs (Gemini 4 Argon/Pro <-> Gemini 3.8 Flash, Claude Opus 5.5 <-> Claude Opus 4.6) so that if an availability switch occurs, subsequent turns immediately re-warm the KV-cache without format discrepancies.

---

### Common Pitfalls to Avoid

| Anti-Pattern | Operational Consequence | Engineering Remedy |
| :--- | :--- | :--- |
| Dynamic session IDs embedded in the system prompt | 100% cache invalidation on every user session | Relocate session IDs to application state headers |
| Randomized order of tools from `Object.values(tools)` | Non-deterministic JSON payload busts KV cache | Sort tools alphabetically by name and schema keys |
| Modifying instruction wording midway through conversation | Re-reads entire 100k+ token context at full price | Keep static prompt templates strictly versioned and immutable |

---

#### Gemini 4 Argon / Flash Fallback Prefix Cache Manager (TypeScript)
```typescript
export interface PrefixCacheDescriptor {
  prefixHash: string;
  modelId: 'gemini-4-argon-medium' | 'gemini-3.8-flash';
  estimatedTokens: number;
}

export function computePrefixDescriptor(
  staticPrefixText: string,
  isGemini4Available: boolean
): PrefixCacheDescriptor {
  let hash = 2166136261;
  for (let i = 0; i < staticPrefixText.length; i++) {
    hash ^= staticPrefixText.charCodeAt(i);
    hash = Math.imul(hash, 16777619);
  }
  return {
    prefixHash: (hash >>> 0).toString(16).padStart(8, '0'),
    modelId: isGemini4Available ? 'gemini-4-argon-medium' : 'gemini-3.8-flash',
    estimatedTokens: Math.ceil(staticPrefixText.length / 4)
  };
}
```

### Hard Guardrails (Declarative Invariants)
- **Deterministic Canonical Prefix Invariant**: The prompt prefix (system instructions, tool definitions, architecture maps) MUST be byte-identical and canonically sorted across turns to maintain $\ge 90\%$ KV-cache hit rates.
- **Prohibition of Timestamps & Session IDs in System Prompt**: Dynamic timestamps, session UUIDs, and volatile turn state MUST NEVER be placed within the static system prompt; they MUST reside exclusively at the tail in user turn messages.
- **Alphabetical Tool & Schema Serialization**: Tool specifications and JSON Schema property keys MUST be sorted alphabetically before sending API payloads to guarantee identical byte representations.
- **Fallback Model Prefix Portability**: System prompt prefix structures MUST remain byte-compatible between primary and availability fallback models (Gemini 4 $\leftrightarrow$ Gemini 3.8 Flash, Claude Opus 5.5 $\leftrightarrow$ Claude Opus 4.6).

### Non-Functional Requirements (NFR)
- **Cache Hit Target**: Achieve $\ge 90\%$ prompt KV-cache hit ratio on multi-turn conversations.
- **TTFT Acceleration**: Sub-500ms Time-to-First-Token on cached prompt evaluations.
- **Cost Reduction**: Minimum 70% prompt token expenditure savings through KV-cache reuse.

### Sovereign Execution, Token Efficiency & Invariants
| Dimension | Sovereign Mandate & Deterministic Contract |
| :--- | :--- |
| **Frontier Model Cascade** | Medium / High (`GPT-5.6 Sol Medium`, `Gemini 4 Argon`, `Claude Sonnet 5.5 High`, `Grok 4.7 High Fast`) |
| **Availability Fallback** | `Gemini 4 Argon/Pro` -> `Gemini 3.8 Flash`; `Claude Opus 5.5` -> `Claude Opus 4.6` (Zero unhandled missing-model errors) |
| **KV-Cache & Token Protocol** | Pin static instructions & schemas at prompt prefix (`kv-cache-prefix-optimizer`); pass surgical `CONTEXT_MAP.md` slices (<250 tokens/call) during Swarm Fan-Out |
| **SaaT CLI & MCP Dispatch** | CLI: `vibes skill exec kv-cache-prefix-optimizer <audit\|eval\|verify\|generate\|blueprint>` / MCP: `vibes_skill_dispatch`, `vibes_ast_query`, `vibes_compile_cache`, `vibes_optimize_budget`, `vibes_verify_gate` |
| **NFR & Anti-Slop Invariants** | P99 API Gateway Latency < 85ms, OpenAPI 3.1 / TypeSpec strict schema parity, RFC 9457 Problem Details error contracts, Idempotency-Key enforcement; 100% complete zero-placeholder output verified via `node scripts/check-anti-slop.js --strict` |

---


---

<a name="bahasa-indonesia"></a>
## Bahasa Indonesia

### Integrasi Orkestrasi
- **Hulu & Orkestrasi**: `zero-to-prod-orchestrator` (Integrasi agen AI Fase 4), `brainstorming`, `session-memory-manager`.
- **LLM Frontier & Caching Model**: `frontier-ai-models-expert` (spesifikasi caching Gemini 4 Argon/Pro, Claude 5.5 Sonnet/Opus, Claude Fable 5.1, GPT-5.6 Sol, Grok 4.7), `gemini-agent-booster` (caching konteks native Google 2M+), `vercel-ai-sdk-expert` (kontrol cache provider).
- **FinOps & Eksekusi Adaptif**: `llm-finops-router` (kalkulasi biaya & routing cache), `adaptive-model-cascade` (prefix tingkat cascade), `thinking-budget-orchestrator` (pinning prefix dengan token penalaran).
- **Rekayasa Prompt & Context Window**: `ai-prompt-engineering-expert` (kanonikalisasi prompt), `context-window-engineer` (sliding window konteks besar).

### Deskripsi
Panduan produksi untuk merekayasa awalan (*prefix*) prompt yang deterministik, mengunci batas KV-cache, dan memaksimalkan penggunaan ulang konteks pada model frontier akhir 2026 (Google Gemini 4 Argon / Pro context caching, Anthropic Claude 5.5 / 5.1 prompt caching, OpenAI GPT-5.6 Sol prefix caching, dan xAI Grok 4.7). Memberikan arsitektur yang menjamin tingkat keberhasilan cache (*cache hit rate*) di atas 90% melalui pengurutan definisi tool kanonikal, serialisasi JSON stabil, dan partisi konteks dinamis.

### Kondisi Pemicu
Aktifkan skill ini ketika:
- Mengelola agen berkonteks besar (>100k token) dengan biaya input token berulang yang signifikan.
- Mengalami latensi tinggi pada Time-to-First-Token (TTFT) saat percakapan atau coding loop interaktif.
- Mengonfigurasi parameter caching model (`cache_control: { type: 'ephemeral' }` atau Gemini explicit caching).
- Merancang instruksi sistem, skema tool, dan peta repositori untuk sesi multi-turn.

---

### Konsep Inti & Pola Praktik

#### 1. Anatomi Prompt Ber-Cache Tinggi
Model frontier mencocokkan token awalan secara persis (*prefix matching*). Perubahan satu karakter saja di awal prompt akan merusak seluruh cache setelahnya:
- **Blok Statis (Terkunci)**: Instruksi sistem, daftar 171 skill, skema tool terurut, dan peta arsitektur proyek.
- **Titik Kunci Cache (*Cache Pin*)**: Menandai akhir blok statis dengan header kontrol cache.
- **Blok Volatil (Dinamis)**: Riwayat percakapan terkini dan hasil eksekusi tool giliran terakhir.

#### 2. Serialisasi JSON Kanonikal
Kunci objek JSON wajib diurutkan secara alfabetis sebelum dikirimkan ke model agar representasi string biner tidak berubah antar permintaan.

---

### Praktik Terbaik

1. **Jangan Taruh Timestamp di System Prompt**: Masukkan waktu atau tanggal hanya pada pesan user paling akhir, bukan pada blok instruksi utama.
2. **Urutkan Definisi Tool Secara Alfabetis**: Selalu lakukan *sorting* pada daftar nama fungsi dan parameter tool.
3. **Posisikan Data Dinamis di Bagian Akhir**: Taruh seluruh dokumen referensi dan kode sebelum giliran pesan pengguna.
4. **Pantau Metrik Cache**: Pantau token yang terbaca dari cache melalui telemetri guna mendeteksi jika terjadi *cache bust* yang tidak disengaja.
5. **Portabilitas Prefix Saat Fallback**: Pertahankan struktur prefix sistem yang identik saat pengalihan ketersediaan model terjadi (Gemini 4 <-> Gemini 3.8 Flash, Claude Opus 5.5 <-> Claude Opus 4.6) agar pemanasan ulang (*re-warming*) KV-cache berlangsung instan tanpa inkonsistensi format.

---

### Jebakan Umum yang Harus Dihindari

| Praktik Buruk | Dampak Buruk | Solusi Rekayasa |
| :--- | :--- | :--- |
| Menyisipkan ID sesi acak ke dalam system prompt | Cache batal 100% pada setiap sesi baru | Pindahkan ID sesi ke header aplikasi di luar prompt |
| Urutan tool berubah-ubah dari `Object.values(tools)` | Representasi string berubah dan membatalkan KV cache | Urutkan tool secara alfabetis berdasarkan namanya |
| Mengubah redaksi instruksi di tengah percakapan | Membaca ulang 100k+ token dengan harga penuh | Kunci template instruksi dalam keadaan statis dan terversi |

### Hard Guardrails (Invarian Deklaratif)
- **Invarian Awalan Kanonikal Deterministik**: Awalan prompt (instruksi sistem, definisi tool, peta arsitektur) WAJIB identik biner dan diurutkan secara kanonikal antar giliran untuk mempertahankan rasio cache hit $\ge 90\%$.
- **Larangan Timestamp & ID Sesi di System Prompt**: Nilai dinamis (waktu tanggal, ID sesi UUID, state giliran volatil) DILARANG KERAS disisipkan ke dalam system prompt statis; wajib ditaruh di pesan user paling akhir.
- **Serialisasi Skema & Tool Alfabetis**: Spesifikasi tool dan kunci objek JSON Schema wajib diurutkan secara alfabetis sebelum dikirimkan ke model agar representasi string biner tidak berubah.
- **Portabilitas Prefix Antar Model Fallback**: Struktur prefix prompt sistem wajib kompatibel biner antara model utama dan model pengalihan ketersediaan (Gemini 4 $\leftrightarrow$ Gemini 3.8 Flash, Claude Opus 5.5 $\leftrightarrow$ Claude Opus 4.6).

### Non-Functional Requirements (NFR)
- **Target Cache Hit**: Rasio keberhasilan KV-cache prompt $\ge 90\%$ pada sesi percakapan multi-turn.
- **Akselerasi TTFT**: Time-to-First-Token $\le 500\text{ms}$ pada eksekusi prompt yang ter-cache.
- **Penghematan Biaya Token**: Penghematan biaya input token minimal 70% melalui penggunaan ulang cache.

### Matriks Eksekusi Berdaulat, Efisiensi Token & Invarian
| Dimensi | Mandat Berdaulat & Kontrak Deterministik |
| :--- | :--- |
| **Kaskade Model Frontier** | Medium / High (`GPT-5.6 Sol Medium`, `Gemini 4 Argon`, `Claude Sonnet 5.5 High`, `Grok 4.7 High Fast`) |
| **Invarian Fallback Ketersediaan** | `Gemini 4 Argon/Pro` -> `Gemini 3.8 Flash`; `Claude Opus 5.5` -> `Claude Opus 4.6` (Bebas gagal saat model belum tersedia/kuota penuh) |
| **Protokol Hemat Token & KV-Cache** | Kunci instruksi & skema statis di prefix awal (`kv-cache-prefix-optimizer`); kirim potongan `CONTEXT_MAP.md` (<250 token/panggilan) saat Fan-Out |
| **Eksekusi SaaT CLI & MCP** | CLI: `vibes skill exec kv-cache-prefix-optimizer <audit\|eval\|verify\|generate\|blueprint>` / MCP: `vibes_skill_dispatch`, `vibes_ast_query`, `vibes_compile_cache`, `vibes_optimize_budget`, `vibes_verify_gate` |
| **Invarian NFR & Anti-Slop** | Latensi P99 API < 85ms, paritas skema ketat OpenAPI 3.1 / TypeSpec, kontrak error RFC 9457 Problem Details, penegakan Idempotency-Key; kode 100% tuntas tanpa placeholder diverifikasi melalui `node scripts/check-anti-slop.js --strict` |