Kb Ready Extraction

---
name: kb-ready-extraction
description: Use to transform raw transcripts, lectures, articles, and unstructured text into high-signal, RAG-optimized, Knowledge Base-ready Markdown documents (for Open-Notebook, Obsidian, NotebookLM, Notion, Vector DBs).
---

# Knowledge Base-Ready Extraction (KB-Ready Extraction) Skill

This skill implements the **KB-Ready Knowledge Engineer** persona and architectural framework. It is specifically designed to transform raw speech-to-text transcripts (ASR), lecture recordings, messy meeting notes, or unstructured web content into structured, high-density, **Knowledge Base (KB) and RAG-ready Markdown documents**.

> **Scope**: Knowledge distillation, conversational denoising, information structuring, and RAG chunk optimization.
> **Target Systems**: Open-Notebook, Obsidian, Logseq, NotebookLM, Notion, Local RAG Vector DBs (Chroma, Qdrant, SurrealDB).
> **Complementary to SMART POLE**: While  structures *prompts & tasks* (Task Specification),  structures *learning & factual content* (Knowledge Representation).

---

## How to Load This Skill

1. **Set system prompt**: Load `references/system-prompt.md` as the agent's system prompt (paste into Custom GPT Instructions, Gemini Gem, Claude Project, or automated extraction pipeline).
2. **Provide reference files** *(optional but recommended)*: Make `references/schema-specification.md`, `references/denoising-rules.md`, and `references/rag-optimization-guide.md` available as context.
3. **Execute extraction**: Feed raw transcript text along with metadata (Course name, Module, Lesson title). The agent outputs a single, clean, valid Markdown document with YAML Frontmatter.

---

## Reference Files

| File | Purpose |
|------|---------|
| `references/system-prompt.md` | 🔴 **Required** — Full Knowledge Engineer system prompt (v1.0). Load this as system instructions. |
| `references/schema-specification.md` | Detailed schema specs: YAML frontmatter fields, Markdown heading hierarchy, Callouts, and FAQ format. |
| `references/denoising-rules.md` | Strict cleaning heuristics: removing spoken fillers, mic checks, small talk, ASR repetition loops. |
| `references/rag-optimization-guide.md` | RAG architecture guide: semantic chunking, HyDE reverse QA retrieval, vector cosine distance optimization. |
| `references/examples.md` | Comprehensive Before/After demonstration: from messy raw ASR transcript to polished KB document. |

---

## The 4-Stage Knowledge Engineering Pipeline

```text
┌────────────────────────────────────────────────────────┐
│ 1. INGESTION & CONVERSATIONAL DENOISING               │
│    - Strip filler words (à, ừm, thì, là, mà)           │
│    - Strip mic checks, greetings, ASR repetition loops │
└────────────────────────┬───────────────────────────────┘
                         ▼
┌────────────────────────────────────────────────────────┐
│ 2. SEMANTIC DISTILLATION                               │
│    - Extract definitions, frameworks, formulas, steps  │
│    - Capture Case Studies & Anti-Patterns (Mistakes)   │
└────────────────────────┬───────────────────────────────┘
                         ▼
┌────────────────────────────────────────────────────────┐
│ 3. PEDAGOGICAL STRUCTURING                             │
│    - Inject YAML Frontmatter (Metadata & Summary)      │
│    - Organize into H1, H2 (Key Takeaways), H3 sections │
└────────────────────────┬───────────────────────────────┘
                         ▼
┌────────────────────────────────────────────────────────┐
│ 4. RAG & SEARCH OPTIMIZATION                           │
│    - Synthesize 3-5 Reverse QA Pairs (HyDE alignment)  │
│    - Validate YAML delimiters & Heading boundaries     │
└────────────────────────────────────────────────────────┘
```

---

## The Golden Document Structure

Every document generated by this skill strictly follows this canonical layout:

```markdown
---
title: "{Clear, descriptive lesson/article title}"
course: "{Course / Category / Book title}"
module: "{Module or Topic cluster, or empty}"
instructor: "{Instructor / Author name, or Unknown}"
difficulty: "{Cơ bản | Trung cấp | Nâng cao}"
target_audience: "{Who this knowledge is for}"
tags:
  - "{tag 1}"
  - "{tag 2}"
summary: "{Concise 2-3 sentence summary of the core knowledge}"
---

# {Lesson / Note Title}

## 💡 Tóm Tắt Cốt Lõi (Key Takeaways)
- {Key takeaway 1}
- {Key takeaway 2}
- {Key takeaway 3}

## 📖 Nội Dung Chi Tiết
### 1. {Main Section 1}
{In-depth academic/practical explanation with bullet points and bold technical terms.}

> **Ví dụ / Case Study:** {Real-world examples, numbers, or practical applications}

> ⚠️ **Lỗi Sai Thường Gặp (Anti-Pattern):** {Common mistakes, misconceptions, or what to avoid}

### 2. {Main Section 2}
...

## ❓ Câu Hỏi & Trả Lời Trọng Tâm (Q&A for Search)
- **Hỏi:** {High-intent natural question matching user search intent?}
  - **Đáp:** {Direct, comprehensive, self-contained answer.}
- **Hỏi:** {Question 2}
  - **Đáp:** {Answer 2}
- **Hỏi:** {Question 3}
  - **Đáp:** {Answer 3}
```

---

## Why This Structure Maximizes RAG & Knowledge Bases

1. **High Signal-to-Noise Ratio**: Eliminates 100% conversational debris, maximizing embedding semantic density.
2. **Deterministic Chunk Boundaries**: Splitters split cleanly on `##` and `###` without severing contextual sentences.
3. **HyDE Reverse Retrieval**: The `## ❓ Q&A` section matches user query vectors with high cosine similarity.
4. **Actionable Anti-Patterns**: Capturing mistakes/pitfalls prevents downstream AI agents from suggesting bad advice.
5. **Metadata Filtering**: Frontmatter fields enable hybrid search (metadata filtering + dense vector search) in Open-Notebook, Chroma, and Qdrant.