Reasoning Path Confidence

---
name: reasoning-path-confidence
title: "A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning"
version: 0.0.2
engine: skillxiv-v0.0.2-claude-opus-4.6
license: MIT
url: "https://arxiv.org/abs/2510.15444"
keywords: [LLM reasoning, test-time scaling, self-consistency, perplexity, probabilistic sampling]
description: "Reduce LLM sampling costs by 50% while maintaining reasoning performance through Reasoning Path Confidence (RPC), which combines perplexity-guided pruning with self-consistency sampling."
---

# Technique: Reasoning Path Confidence — Efficient Test-Time Scaling

LLMs achieve strong performance on complex reasoning tasks through test-time sampling strategies like self-consistency, but this approach requires extensive sampling from the full probability distribution. However, sampling introduces high estimation error while perplexity-based filtering suffers from high modeling error. Most reasoning paths are ineffective noise.

RPC bridges these approaches by recognizing that self-consistency and perplexity capture complementary signals. Rather than choosing between them, the technique combines perplexity consistency to detect unreliable reasoning branches with reasoning pruning to eliminate low-probability paths before ensemble voting.

## Core Concept

RPC operates on two key insights:
- **Perplexity Consistency**: Path perplexity correlates with reasoning quality—high perplexity indicates uncertain or incoherent reasoning steps
- **Reasoning Pruning**: Eliminating low-probability reasoning paths reduces noise without degrading the signal from high-quality paths

The combination achieves exponential convergence improvement (from linear to exponential) compared to self-consistency alone while reducing sampling costs by 50%.

## Architecture Overview

- **Probability Distribution Analysis**: Track per-step token probabilities during chain-of-thought generation
- **Perplexity Scoring**: Compute Shannon entropy over next-token distributions to detect uncertainty spikes
- **Path Filtering**: Prune reasoning paths where cumulative perplexity exceeds a learned threshold
- **Ensemble Voting**: Apply standard self-consistency voting only on surviving high-quality paths
- **Convergence Acceleration**: Fewer but higher-quality samples accelerate accuracy improvement

## Implementation Steps

The core algorithm computes perplexity per step and filters paths before ensemble aggregation.

```python
import numpy as np
from collections import defaultdict

def reasoning_path_confidence(
    reasoning_paths,
    step_log_probs,
    perplexity_threshold=2.5,
    top_k_paths=None
):
    """
    RPC filtering: prune low-confidence reasoning paths before ensemble voting.

    Args:
        reasoning_paths: list of reasoning strings (complete chains-of-thought)
        step_log_probs: list of lists, log probabilities per step in each path
        perplexity_threshold: entropy cutoff for path acceptance
        top_k_paths: if set, keep only top-k paths by score

    Returns:
        filtered_paths: paths surviving confidence filtering
        path_scores: confidence scores for each surviving path
    """
    # Compute perplexity (entropy) per step for each path
    perplexity_scores = []
    for i, log_probs in enumerate(step_log_probs):
        # Average entropy across steps in this reasoning path
        step_entropies = [-lp for lp in log_probs]  # Higher negLP = higher entropy
        path_perplexity = np.mean(step_entropies)
        perplexity_scores.append(path_perplexity)

    # Filter paths by perplexity threshold
    surviving_indices = [
        i for i, score in enumerate(perplexity_scores)
        if score <= perplexity_threshold
    ]

    filtered_paths = [reasoning_paths[i] for i in surviving_indices]
    filtered_scores = [perplexity_scores[i] for i in surviving_indices]

    # Optional: keep only top-k highest-confidence paths
    if top_k_paths and len(filtered_paths) > top_k_paths:
        top_indices = np.argsort(filtered_scores)[:top_k_paths]
        filtered_paths = [filtered_paths[i] for i in top_indices]
        filtered_scores = [filtered_scores[i] for i in top_indices]

    return filtered_paths, filtered_scores


def ensemble_reasoning_with_rpc(
    model,
    prompt,
    num_samples=20,
    perplexity_threshold=2.5
):
    """
    Generate multiple reasoning paths, apply RPC filtering, then ensemble.
    """
    paths = []
    log_probs_list = []

    for _ in range(num_samples):
        # Generate path and collect per-step log probabilities
        path, step_lps = model.generate_with_logprobs(
            prompt, max_steps=50
        )
        paths.append(path)
        log_probs_list.append(step_lps)

    # Apply RPC filtering
    filtered_paths, scores = reasoning_path_confidence(
        paths, log_probs_list, perplexity_threshold
    )

    # Extract answers and vote
    answers = extract_answers(filtered_paths)
    final_answer = majority_vote(answers)

    return final_answer, len(filtered_paths), num_samples
```

Perplexity threshold tuning: Start at 2.5 and adjust based on validation set. Higher threshold = more paths survive = more computation, lower threshold = stronger filtering = fewer samples needed.

## Practical Guidance

| Scenario | Threshold | Num Samples | Expected Reduction |
|----------|-----------|-------------|-------------------|
| Math reasoning (AIME) | 2.0-2.5 | 16-20 | 40-50% fewer samples |
| Commonsense QA | 2.5-3.0 | 8-12 | 30-40% fewer samples |
| Code generation | 1.8-2.2 | 12-16 | 45-55% fewer samples |

**When to Use:**
- You have a large sampling budget and want to reduce inference cost without losing accuracy
- Reasoning tasks have clear right/wrong answers for ensemble voting
- You can log per-step token probabilities from your model

**When NOT to Use:**
- Open-ended generation without verifiable correct answers
- Tasks where perplexity doesn't correlate with answer quality
- Online inference with strict latency constraints (pruning adds overhead)

**Common Pitfalls:**
- Setting perplexity threshold too low → filters out valid paths, degrades accuracy
- Not normalizing entropy across variable-length reasoning chains
- Applying threshold uniformly to all task difficulties (easier tasks may have lower perplexity)

## Reference

[NeurIPS 2025 | A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning](https://arxiv.org/abs/2510.15444)