Verifier Free Test Time Sampling

---
name: verifier-free-test-time-sampling
title: "Verifier-Free Test-Time Sampling for Vision Language Action Models"
version: 0.0.2
engine: skillxiv-v0.0.2-claude-opus-4.6
license: MIT
url: "https://arxiv.org/abs/2510.05681"
keywords: [Test-Time Sampling, Vision Language Models, Action Prediction, Self-Verification, Confidence]
description: "Use model confidence and prediction consistency to select high-quality actions at test time without external verifiers, enabling reliable action selection in VLA systems."
---

# Technique: Self-Verified Action Selection via Confidence Scoring

Vision-Language Action models (VLAs) predict actions for embodied tasks, but their confidence can be poorly calibrated. External verifiers add latency and complexity. Verifier-Free Test-Time Sampling uses intrinsic model signals—confidence scores and prediction consistency across multiple generations—to identify reliable actions without extra models.

The key insight is that models show different confidence levels for different predictions. High-confidence predictions tend to be more accurate. By generating multiple action predictions and selecting based on confidence and agreement, the system achieves reliable action selection without verification overhead.

## Core Concept

The method operates through three stages:

1. **Multi-Sampling**: Generate K action predictions with model confidence scores
2. **Consistency Filtering**: Keep only actions appearing frequently or with high average confidence
3. **Calibration-Aware Selection**: Account for model's confidence-accuracy relationship

## Architecture Overview

- **Input**: Visual observation (image/video) and task instruction
- **Multi-Action Generation**: Sample K actions with confidence scores
- **Confidence Aggregation**: Pool confidence signals across samples
- **Selection**: Choose action maximizing confidence-weighted frequency
- **Output**: Single high-confidence action for execution

## Implementation Steps

Implement multi-sampling with confidence score extraction.

```python
def sample_actions_with_confidence(model, observation, instruction, num_samples=5):
    """
    Generate multiple action predictions and extract confidence scores.

    Args:
        model: Vision-language action model
        observation: Image or video observation
        instruction: Task instruction string
        num_samples: Number of action samples to generate

    Returns:
        samples: List of (action, confidence) tuples
    """

    samples = []

    for _ in range(num_samples):
        # Generate action
        output = model.generate(
            observation, instruction,
            return_confidence=True,
            temperature=0.8  # Temperature > 0 for diversity
        )

        action = output['action']
        confidence = output['confidence']  # Range [0, 1]

        samples.append((action, confidence))

    return samples
```

Implement consistency-based filtering.

```python
def filter_by_consistency(samples, consistency_threshold=0.5,
                         confidence_threshold=0.3):
    """
    Filter action samples by agreement and confidence.

    Args:
        samples: List of (action, confidence) tuples
        consistency_threshold: Min agreement rate for action to pass
        confidence_threshold: Min average confidence threshold

    Returns:
        filtered_samples: High-quality (action, avg_confidence) tuples
    """

    from collections import Counter

    # Count action occurrences
    action_counts = Counter([action for action, _ in samples])
    action_freq = {action: count / len(samples) for action, count in action_counts.items()}

    # Compute average confidence per action
    action_confidences = {}
    for action, confidence in samples:
        if action not in action_confidences:
            action_confidences[action] = []
        action_confidences[action].append(confidence)

    action_avg_conf = {action: sum(confs) / len(confs)
                       for action, confs in action_confidences.items()}

    # Filter by consistency and confidence
    filtered = []
    for action in action_counts:
        freq = action_freq[action]
        avg_conf = action_avg_conf[action]

        # Accept if agreement rate high OR average confidence high
        if freq >= consistency_threshold or avg_conf >= confidence_threshold:
            filtered.append((action, avg_conf))

    return filtered
```

Implement confidence-calibrated selection.

```python
def select_action_by_confidence(model, samples):
    """
    Select action using confidence-weighted scoring.

    Args:
        model: VLA model with calibration information
        samples: List of (action, avg_confidence) tuples

    Returns:
        selected_action: Action to execute
        selection_confidence: Confidence in selection
    """

    if not samples:
        return None, 0.0

    # Get model's confidence calibration curve
    # (from validation data: binned confidence vs. accuracy)
    calibration = model.get_confidence_calibration()

    # Score each action candidate
    action_scores = []
    for action, confidence in samples:
        # Calibrated confidence: confidence * empirical_accuracy_at_confidence
        calibrated = confidence * calibration.get_accuracy_at_confidence(confidence)
        action_scores.append((action, calibrated))

    # Select highest-scored action
    selected_action, selection_score = max(action_scores,
                                          key=lambda x: x[1])

    return selected_action, selection_score
```

Implement dynamic confidence calibration using validation data.

```python
class ConfidenceCalibrator:
    def __init__(self):
        self.bins = 10  # Number of confidence bins
        self.bin_data = {}  # confidence_bin -> [accuracies]

    def calibrate_from_validation(self, validation_predictions, validation_labels):
        """
        Build confidence-accuracy relationship from validation data.

        Args:
            validation_predictions: List of (action, confidence) tuples
            validation_labels: List of ground-truth actions
        """

        for (pred_action, confidence), true_action in \
                zip(validation_predictions, validation_labels):

            # Bin confidence
            bin_idx = int(confidence * self.bins)
            bin_idx = min(bin_idx, self.bins - 1)

            if bin_idx not in self.bin_data:
                self.bin_data[bin_idx] = []

            # Record if prediction was correct
            is_correct = (pred_action == true_action)
            self.bin_data[bin_idx].append(is_correct)

    def get_accuracy_at_confidence(self, confidence):
        """
        Get empirical accuracy for given confidence level.

        Args:
            confidence: Confidence score in [0, 1]

        Returns:
            accuracy: Empirical accuracy at this confidence
        """

        bin_idx = int(confidence * self.bins)
        bin_idx = min(bin_idx, self.bins - 1)

        if bin_idx not in self.bin_data or not self.bin_data[bin_idx]:
            return 0.5  # Default if no data

        accuracies = self.bin_data[bin_idx]
        return sum(accuracies) / len(accuracies)
```

Implement the full test-time sampling pipeline.

```python
def test_time_action_selection(model, observation, instruction,
                              calibrator=None, num_samples=5):
    """
    Full pipeline for verifier-free action selection.

    Args:
        model: Vision-language action model
        observation: Visual observation
        instruction: Task instruction
        calibrator: Optional confidence calibrator
        num_samples: Number of action samples

    Returns:
        action: Selected action for execution
        confidence: Confidence score
        selection_info: Debug information
    """

    # Stage 1: Multi-sample
    samples = sample_actions_with_confidence(
        model, observation, instruction, num_samples
    )

    # Stage 2: Filter by consistency
    filtered_samples = filter_by_consistency(
        samples,
        consistency_threshold=0.4,
        confidence_threshold=0.5
    )

    if not filtered_samples:
        # Fallback: select by raw confidence
        selected_action, confidence = max(samples, key=lambda x: x[1])
        return selected_action, confidence, {'fallback': True}

    # Stage 3: Select with calibration
    if calibrator:
        selected_action, confidence = select_action_by_confidence(
            model, filtered_samples
        )
    else:
        # Without calibrator, just use average confidence
        selected_action, confidence = max(filtered_samples,
                                         key=lambda x: x[1])

    selection_info = {
        'num_samples': num_samples,
        'num_filtered': len(filtered_samples),
        'selection_confidence': confidence
    }

    return selected_action, confidence, selection_info
```

## Practical Guidance

| Aspect | Recommendation | Notes |
|--------|---------------|-------|
| Number of samples | 3-10 per action | More samples improve reliability; trade-off with latency |
| Consistency threshold | 0.3-0.6 | Higher = more selective; lower = more lenient |
| Confidence threshold | 0.4-0.7 | Domain-dependent; tune on validation data |
| Calibration data | 500-2000 examples | Build confidence-accuracy mapping from validation |
| When to use | Embodied tasks with action prediction | Robot control, navigation, web interaction |
| When NOT to use | Tasks where single-sample sufficient | Overhead of multi-sampling not justified |
| Common pitfall | Miscalibrated confidence scores | Validate calibration on held-out test data |

### When to Use Verifier-Free Test-Time Sampling

- Embodied AI/robotics where action reliability is critical
- Deployments without access to external verification
- Tasks with natural action diversity where consensus is informative
- Cost-sensitive settings where external verifiers are expensive

### When NOT to Use Verifier-Free Test-Time Sampling

- Low-latency requirements where multi-sampling is prohibitive
- Tasks with single deterministic correct action
- Scenarios where confidence scores are poorly calibrated

### Common Pitfalls

- **Confidence miscalibration**: Models may be overconfident; always validate on held-out data
- **Insufficient samples**: Too few samples make frequency-based filtering unreliable
- **Action diversity**: If model samples same action repeatedly, consistency filtering fails
- **Validation data leakage**: Use separate validation set for calibration, not training data

## Reference

Paper: https://arxiv.org/abs/2510.05681