Customization And Structured Outputs

---
name: customization-and-structured-outputs
description: "Customize LlamaIndex Settings, prompts, callbacks,
  instrumentation, evaluation signals, and Pydantic structured outputs without
  relying on external APIs during local checks."
disable-model-invocation: true
metadata:
  disco-role: operating
license: MIT
---

# Customization and Structured Outputs

## Use This Sub-Skill When

- You need to configure `Settings.llm`, `Settings.embed_model`, `Settings.node_parser`, `Settings.transformations`, `Settings.callback_manager`, tokenizer, chunk sizes, or prompt helpers.
- You need custom `PromptTemplate` or `ChatPromptTemplate` behavior, prompt inspection, prompt updates, or output-parser formatting instructions.
- You need Pydantic structured output through `PydanticOutputParser`, `LLMTextCompletionProgram`, `llm.as_structured_llm(...)`, or `get_response_synthesizer(output_cls=...)`.
- You need callbacks, instrumentation, token counting, or evaluator result shapes to debug query behavior.
- You need deterministic tests using `MockLLM` and `MockEmbedding` while avoiding external provider calls.

## Route Elsewhere

- Use `../indexing-and-querying/` for index construction, retrievers, query engines, node parsing as part of ingestion, and response modes in normal RAG flows.
- Use `../integrations-and-storage/` for provider packages, credentials, vector stores, readers, and external backend setup.
- Use `../agents-and-workflows/` for `AgentWorkflow`, `FunctionAgent`, `ReActAgent`, handoffs, tools, and agent orchestration; return here for schemas, parsers, prompts, and structured-output validation.

## Core Workflow

1. Decide scope: prefer per-object constructor arguments (`llm=`, `embed_model=`, `callback_manager=`, prompt templates) when isolation matters; use global `Settings` only for app-wide defaults.
2. For tests, set `Settings.llm = MockLLM(...)` and `Settings.embed_model = MockEmbedding(embed_dim=...)`, then restore prior `Settings` fields after the test.
3. Customize prompts with `PromptTemplate`, `ChatPromptTemplate`, `partial_format`, `template_var_mappings`, `function_mappings`, or component prompt APIs such as `get_prompts()` / `update_prompts()`.
4. Add structured outputs with a Pydantic model, then choose the surface: parser-attached prompt, `LLMTextCompletionProgram.from_defaults`, structured LLM wrapper, or response synthesizer `output_cls`.
5. Add callbacks or instrumentation before constructing the component that should emit events, then verify events with a local query or explicit callback-manager context.
6. Evaluate with evaluator classes that return `EvaluationResult` fields (`query`, `contexts`, `response`, `passing`, `score`, `feedback`, invalid metadata), making sure input shapes match the evaluator.

## Bundled References

- `references/customization-recipes.md` contains copy-adapt recipes for Settings isolation, prompts, structured outputs, callbacks, instrumentation, and evaluation.
- `references/api-reference.md` summarizes verified import paths, signatures, and object responsibilities for this sub-skill.
- `references/troubleshooting.md` maps common customization and structured-output failures to targeted fixes.
- `scripts/inspect_settings_and_prompts.py` prints Settings, prompt, parser, callback, evaluator, and mock-model checks without requiring external APIs.

## Safety Notes

- Avoid deprecated `ServiceContext`; current code uses `Settings` and explicit constructor arguments.
- Never let global `Settings` mutations leak between tests; snapshot and restore modified attributes.
- Do not hard-code provider API keys in skill examples; provider installation and credentials are integration concerns.
- Treat Pydantic validation errors as schema/prompt/model-contract feedback, not as retriever failures.