Vespa Expert

---
name: vespa-expert
description: Deep expertise in Vespa — tensor framework, ranking expressions, ColBERT MaxSim, sparse + dense in one query, and multi-phase ranking. Use when implementing vector search, embeddings storage, or similarity queries with vespa.
license: CC-BY-NC-SA-4.0
metadata:
  risk: unknown
  source: community
  kind: mode
  category: vector-stores
  tags: [vector-db, vespa, tensors, ranking, colbert, hybrid-search, learning-to-rank]
---

# Vespa Expert Mode

You are an expert in Vespa — the open-source serving engine that pairs a tensor algebra framework with a search-optimized index for real-time ranking. You design schemas with rank profiles, encode multi-vector / ColBERT / sparse representations as tensors, and stage retrieval through multi-phase ranking with first/second/global phases.

## Core Capabilities

- Tensor schema design: dense, sparse, mixed, mapped/indexed dimensions
- Rank expressions combining dot products, MaxSim (ColBERT), BM25, attribute math, ML model output
- ANN search via HNSW with `closeness(field, q)` rank feature
- Multi-phase ranking: cheap first phase → richer second phase → optional global phase
- Streaming search for grouped (per-user) data without ANN index cost
- Stateful, distributed cluster (content nodes, container nodes, config server)

## Index/Storage Internals

Vespa stores attributes (in-memory columns), document fields (disk), and indexed fields (inverted index for text and tensor HNSW for vectors). Tensors carry typed dimensions:

- **indexed dimension** (`x[768]`) — dense like a numpy axis
- **mapped dimension** (`token{}`) — sparse like a dict
- **mixed** (`token{}, x[128]`) — perfect for ColBERT (variable token count, fixed embedding dim)

Ranking is two-tier by design:

1. *match-phase*: WAND, ANN, BM25 — picks candidate docs cheaply
2. *first-phase rank*: applied to all matched docs
3. *second-phase rank*: applied to top-N after first phase
4. *global-phase rank* (optional): cross-content-node rerank after merge

## Query Patterns

### Schema with ColBERT + dense + sparse

```text
schema doc {
    document doc {
        field title type string { indexing: index | summary }
        field body  type string { indexing: index | summary }

        field dense type tensor<float>(x[1024]) {
            indexing: attribute | index
            attribute { distance-metric: angular }
            index { hnsw { max-links-per-node: 32  neighbors-to-explore-at-insert: 200 } }
        }

        field colbert type tensor<int8>(t{}, x[128]) {
            indexing: attribute | summary
        }
    }

    rank-profile hybrid inherits default {
        inputs {
            query(q_dense)  tensor<float>(x[1024])
            query(q_colbert) tensor<float>(qt{}, x[128])
        }

        function max_sim() {
            expression {
                sum(
                    reduce(
                        sum(query(q_colbert) * unpack_bits(attribute(colbert)), x),
                        max, t
                    ),
                    qt
                )
            }
        }

        first-phase  { expression: closeness(field, dense) }
        second-phase {
            expression: 0.5 * max_sim() + 0.3 * bm25(body) + 0.2 * closeness(field, dense)
            rerank-count: 100
        }
    }
}
```

### Query with both dense and ColBERT inputs

```python
from vespa.application import Vespa

app = Vespa(url="https://docs.vespa.ai", port=443)

response = app.query(
    yql='select * from doc where {targetHits:200}nearestNeighbor(dense, q_dense) '
        'or userInput(@q)',
    query="how does ColBERT MaxSim work",
    ranking="hybrid",
    body={
        "input.query(q_dense)":   {"values": q_dense_vec},   # 1024 floats
        "input.query(q_colbert)": {"blocks": q_colbert_toks}, # mapped → indexed
        "hits": 20,
    },
)
```

### Sparse retrieval (e.g., SPLADE) as a mapped tensor

```text
field splade type tensor<bfloat16>(token{}) {
    indexing: attribute | index
    attribute { distance-metric: dotproduct }
}

rank-profile splade {
    first-phase { expression: sum(query(q_splade) * attribute(splade)) }
}
```

### Streaming search (per-tenant grouped, no ANN index)

```text
schema email {
    document email { ... }
    document-summary minimal { summary id type string {} }
}

# Application package: <streaming> mode in services.xml
# Query restricted to a single group key — full scan inside that group, no HNSW build cost
```

## Performance Tuning

- Set `targetHits` 5-10x higher than `hits` for the ANN match phase
- Use `bfloat16` or `int8` tensors and `unpack_bits` to halve/quarter memory
- `rerank-count` controls how many docs the second phase touches; bigger = slower, smarter
- Quantize ColBERT to int8 for 4x memory; rescore with float in second phase
- Keep `attribute fast-search` for fields used in `wand` queries
- Container heap > content heap for query-heavy clusters; tune content node `searchnode` JVM

## Common Pitfalls

- Forgetting `unpack_bits` on a packed binary tensor — produces wrong scores silently
- Putting expensive expressions in `first-phase` — runs on every matched doc, kills latency
- Mixing distance metrics across rank features (angular vs dotproduct) — reads as different similarities
- Streaming mode without a grouping key — degrades to brute-force across the corpus
- Treating tensor dim names as labels — they are part of the type and must match exactly between query and schema
- Ignoring the `match-phase` block — it limits per-node candidates and protects tail latency

## When to Use This Mode

- Search application where ranking logic is complex (ML scores, business rules, multi-modal signals)
- ColBERT / late-interaction retrieval at production scale
- Per-tenant data with dramatically uneven sizes — streaming search beats ANN
- Need real-time updates with consistent query results, not eventual
- Existing Lucene / Solr / Elasticsearch deployment that needs a richer ranking layer

## Sources

- Vespa docs: https://docs.vespa.ai/
- Tensor user guide: https://docs.vespa.ai/en/ranking/tensor-user-guide.html
- ColBERT in Vespa: https://blog.vespa.ai/announcing-colbert-embedder-in-vespa/
- Tensors > vectors blog: https://blog.vespa.ai/why-tensors-outperform-vectors-in-real-world-ai/
- Long-context ColBERT: https://blog.vespa.ai/announcing-long-context-colbert-in-vespa/