Heterogeneous Neural Predictivity Lm

---
name: heterogeneous-neural-predictivity-lm
description: "Analysis framework for evaluating language model neural predictivity during naturalistic comprehension. Identifies heterogeneous brain-language alignment patterns across participants and brain regions, separating predictive usefulness from shared neural organization claims (arXiv: 2606.26880)"
metadata:
  arxiv_id: "2606.26880"
  authors: ["Xiao Jia"]
  published: "2026-06-25"
  source_url: "https://arxiv.org/abs/2606.26880"
  tags: ["neuroscience", "language models", "brain-language alignment", "naturalistic comprehension", "neural predictivity", "EEG", "MEG", "ECoG"]
  categories: ["cs.CL", "q-bio.NC", "cs.LG"]
  methodology: "Multi-dataset frozen LM analysis with controlled feature ablations and participant-level matching"
  datasets: ["Brain Treebank", "MEG-MASC", "Podcast ECoG"]
  models_tested: ["8 frozen language models", "blocked encoding models"]
  key_findings: "67/432 evaluable rows met controlled predictive criterion; participant-level advantages localized not uniform"
---

# Heterogeneous Neural Predictivity from Language Models

## Core Contribution

Demonstrates that language model representations can annotate neural activity during natural speech/text comprehension, but with critical caveats about interpretation.

## Methodology

### Multi-Dataset Framework
- Analyzed locked derived data from 3 datasets: Brain Treebank, MEG-MASC, Podcast ECoG
- Used 8 frozen language models with matched controls
- Controlled for temporal, nuisance, and representation-capacity confounds

### Evaluation Strategy
- Source-level summaries for held-out prediction
- Comparison against low-level baselines
- Feature ablations to test component-level sensitivity

### Control Conditions
- Brain-derived timing-linked controls
- Acoustic controls
- Implanted-signal controls
- Matched-capacity controls

## Key Findings

### Predictive Success
- 67 of 432 evaluable rows met controlled predictive-only criterion
- Positive held-out prediction widespread in source-level summaries
- Model-side feature ablations changed prediction scores in most evaluable rows

### Heterogeneity Pattern
- Participant-level matched-control advantages were **localized**, not uniform
- Response-profile and feature-specificity contrasts bounded interpretations
- Complete co-indexed integrated interpretation requires future jointly indexed coverage

## Critical Distinction

**Separates two claims:**
1. **Predictive usefulness**: LM features can predict neural activity ✓
2. **Shared organization**: LM features reveal neural computation mechanisms ✗ (not proven)

The framework explicitly distinguishes between using LMs as neural predictors vs. claiming they share neural computational principles.

## Implementation Guidance

### When to Use
- Evaluating LM neural predictivity in naturalistic settings
- Designing controlled brain-language alignment studies
- Interpreting heterogeneous alignment patterns across participants

### Methodology Steps
1. Select multiple frozen LMs across architecture families
2. Establish matched temporal and capacity controls
3. Test held-out prediction against low-level baselines
4. Perform feature ablations to verify component sensitivity
5. Analyze participant-level heterogeneity (don't assume uniform effects)
6. Separate predictive utility from mechanistic claims

### Pitfalls
- **Over-interpretation**: Predictive success ≠ shared computational mechanism
- **Uniformity assumption**: Effects are localized, not uniform across participants
- **Insufficient controls**: Must control for temporal, acoustic, and capacity confounds
- **Single-dataset claims**: Multi-dataset validation required for robust conclusions

## Applications

- Naturalistic neuroscience experimental design
- Brain-computer interface feature selection
- Language model evaluation for cognitive modeling
- Cross-participant neural pattern analysis

## Related Concepts

- Brain-language alignment
- Neural encoding models
- Naturalistic stimulus processing
- Frozen feature analysis
- Participant heterogeneity