Build and navigate the genai-graph Document Graph — the Folder/Document/MarkdownSection provenance hierarchy, DocumentGraphFactory, tree_parser, ingest_document_graph/drop_document_graph, and the vectorless agentic RAG navigation tools. Use when editing genai_graph/kg/document_graph, writing a document-ingestion pipeline, or giving an agent tools to walk a document corpus by heading.
Installs into .claude/skills of the current project.
Sponsors
SSkills DirectorySkills Directory
Know which skills are safe — weekly.
SSkills DirectorySkills Directory
Is that skill safe to install?
Are you the author of Kg Document Graph?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/tclatos-kg-document-graph)
---
name: kg-document-graph
description: Build and navigate the genai-graph Document Graph — the Folder/Document/MarkdownSection provenance hierarchy, DocumentGraphFactory, tree_parser, ingest_document_graph/drop_document_graph, and the vectorless agentic RAG navigation tools. Use when editing genai_graph/kg/document_graph, writing a document-ingestion pipeline, or giving an agent tools to walk a document corpus by heading.
---
# GenAI Graph Document Graph
## Read First
- `docs/document-graph.md` — full schema, factories, CLI, querying
- `genai_graph/kg/document_graph/ingest.py` — `ingest_document_graph`, `drop_document_graph`
- `genai_graph/kg/document_graph/build.py` — the single build pipeline (`build_document_graph`, `warm_outline_cache`) wrapped by the CLI, workflow engine, and bench
- `genai_graph/kg/document_graph/tree_parser.py` — heading-hierarchy parser
- `genai_graph/kg/document_graph/repository.py` — section CRUD
- `genai_graph/kg/factories/document_graph_factory.py` — `DocumentGraphFactory`, `DocumentGraphBundle`
- `genai_graph/kg/factories/document_factory.py` — `DocumentDirectoryFactory`
- `genai_graph/kg/query/document_graph_tools.py` — navigation functions + LangChain tools
- `genai_graph/kg/nodes/document.py`, `genai_graph/kg/nodes/document_section.py` — node models
## What it is
A generic, **vectorless** representation of a document corpus: which folder files came
from, file metadata, and — for Markdown — the heading hierarchy. It is the provenance layer
every entity-extraction factory attaches to, and a substrate for agentic RAG where an agent
walks a document's table of contents and reads section text directly (no embeddings).
```
Folder ──CONTAINS──▶ Document ──HAS_SECTION──▶ MarkdownSection ──HAS_SUBSECTION──▶ MarkdownSection ──…
```
| Node | Key | Notes |
|---|---|---|
| `Folder` | `folder_id` | A directory, `.zip` archive, or a single file's parent. |
| `Document` | `content_hash` (xxHash of raw bytes) | Provenance anchor; carries `filename`, `relative_path`, `path`, `mime_type`, `token_count`, `section_count`. |
| `MarkdownSection` | `section_id` = `{content_hash}::{sequence}` | One heading-delimited section (heading + body up to the next heading). A synthetic level-0 root section captures heading-less documents/preambles. |
Sections form a flat table with `parent_section_id`; the hierarchy is materialized as
`HAS_SUBSECTION` edges. A section's `text` is its own content only (non-overlapping), so
concatenating every section in `sequence` order reconstructs the original Markdown exactly.
**Identity/dedup:** `Document` and `MarkdownSection` are keyed by content hash, so
re-ingesting unchanged files is a no-op MERGE. When an entity factory
(`MarkdownBamlFactory`/`JsonFileBackedFactory` via `DocumentMixin`) also produces a
`Document` node for the same file, it MERGEs into the *same* node — provenance and extracted
entities share one graph node.
There are **no `Chunk` nodes** — no chunking or embeddings. Chunking/embedding RAG is a
separate path (`DocumentDirectoryFactory` as a base for custom pipelines).
## Factories
| Factory | Produces | Use when |
|---|---|---|
| `DocumentGraphFactory` | `Folder → Document → MarkdownSection` | You want the navigable heading hierarchy over Markdown. |
| `DocumentDirectoryFactory` | Plain `Document` nodes | File-level provenance only (no sections), or a base for custom RAG. |
| `MarkdownBamlFactory` | Your entity nodes + a provenance `Document` (`MENTIONS`) | Inline BAML entity extraction from Markdown. |
| `JsonFileBackedFactory` | Your entity nodes + a provenance `Document` | Entities already extracted as JSON files. |
`DocumentGraphFactory(sources=[...], include=["*.md"], exclude=[])` — `sources` accepts a
mix of directories, files, and `.zip` archives; each becomes a `Folder`.
## Ingest
```python
from genai_graph.kg.backend import KuzuBackend
from genai_graph.kg.document_graph.ingest import ingest_document_graph, drop_document_graph
from genai_graph.kg.factories.document_graph_factory import DocumentGraphFactory
backend = KuzuBackend()
backend.connect("./data/kg/tree.db")
factory = DocumentGraphFactory(sources=["./docs"], include=["*.md"])
result = ingest_document_graph(backend, factory) # MERGEs everything
result = ingest_document_graph(backend, factory, force=True) # rebuild sections for existing docs
drop_document_graph(backend) # drop Section/Document/Folder tables
```
`ingest_document_graph` builds the schema, parses each file's heading hierarchy
(`tree_parser`), and MERGEs in one call. `force=True` rebuilds sections for documents
already in the graph (e.g. after a heading edit).
## End-to-end via the workflow engine
`docgraph_build_step` (in `genai_graph/orchestration/workflow_steps.py`) ties
markdownization, entity extraction, and the document graph into **one** database:
1. Optionally dispatch `sources` through the ingest route table (`routes=`):
rule-selected workflows (markdownize for documents, web-page fetching for
URLs, …) run in parallel and converge to Markdown.
2. Run each configured entity `factories` (e.g. a `MarkdownBamlFactory` subclass) into one
KG named `kg_name`.
3. Optionally ingest the `Folder → Document → MarkdownSection` graph over the same Markdown
into the *same* database (`build_document_graph`, default `true`) — `Document` nodes
MERGE by content hash with the ones the entity factories create.
This backs both `cli docgraph run` (ad-hoc sources) and a project's named profiles via
`cli kg create <name>` — see `kg-workflows`.
## Navigation tools (vectorless agentic RAG)
`genai_graph.kg.query.document_graph_tools` exposes read-only Cypher-backed helpers used by
the CLI, an agent's tools, and the Textual TUI:
```python
from genai_graph.kg.query.document_graph_tools import (
list_documents,
get_document_toc,
get_section_content,
reconstruct_document,
reconstruct_section,
search_sections,
create_document_graph_tools,
)
```
Documents are addressed by content hash (full or prefix), filename, or
source path. `create_document_graph_tools(db_path)` wraps these as LangChain `BaseTool`s:
`list_documents`, `get_document_toc`, `get_section_content`, `search_sections`. Wire them
into an agent profile (see `kg-query` and `genai-tk/add-tool`).
The agentic RAG loop: `list_documents` → `get_document_toc` (the map) →
`get_section_content` (only the sections worth reading) → answer. No embeddings, no chunk
retrieval — the agent reads exactly the section text it navigated to.
## CLI
```bash
cli docgraph build ./docs --db ./data/kg/tree.db # markdownize + build document graph
cli docgraph build ./RFQ.zip --db ./data/kg/tree.db --routes default
cli docgraph run --workflow rainbow_extract -s ./some_file.pptx # project entity-extraction workflow
cli docgraph list --db ./data/kg/tree.db
cli docgraph toc <filename-or-hash> --db ./data/kg/tree.db
cli docgraph cat <filename-or-hash> --db ./data/kg/tree.db
cli docgraph search "keyword" --db ./data/kg/tree.db
cli docgraph tui --db ./data/kg/tree.db
```
`cli docgraph build` always markdownizes first then ingests; `cli docgraph run` runs a
project's `docgraph_build`-based workflow against ad-hoc or configured sources. See `kg-cli`.
## Change Workflow
1. Schema changes to `Folder`/`Document`/`MarkdownSection` live in
`kg/nodes/document.py` and `kg/nodes/document_section.py`; keep the content-hash keys —
they are what make re-ingest a no-op and what let entity factories MERGE into the same
`Document`.
2. Heading-parsing changes go in `tree_parser.py`; verify the "concatenate sections in
`sequence` order == original Markdown" invariant with a round-trip test.
3. New navigation helpers go in `document_graph_tools.py`; expose them both as raw
functions and inside `create_document_graph_tools` so agents and the CLI share them.
4. Re-ingest with `force=True` (or `--force graph`) after a parser/section change.
## Commands
```bash
cli docgraph build ./docs --db ./data/kg/tree.db --force md # re-run markdown conversion
cli docgraph build ./docs --db ./data/kg/tree.db --force graph # re-ingest, reuse markdown cache
uv run just test
```
## Avoid
- Do not add `Chunk` nodes or embeddings to the Document Graph — it is deliberately
vectorless; chunking RAG is a separate path.
- Do not key `Document`/`MarkdownSection` by anything but content hash — that breaks the
MERGE-with-entity-factories guarantee.
- Do not hand-roll navigation Cypher in agent code — reuse `document_graph_tools`.
- Do not markdownize manually before `cli docgraph build` — it markdownizes internally.
## Complements
- `kg-factories` — `DocumentGraphFactory`/`MarkdownBamlFactory`/`DocumentDirectoryFactory`.
- `kg-query` — `create_document_graph_tools` and the broader Cypher/Text-to-Cypher story.
- `kg-workflows` — `docgraph_build_step` and `cli docgraph run`.
- `genai-tk/workflow-engine` — the ingest route table and `markdownize_flow` used in step 1.