Neural Surrogate Affective Neuromodulation Design

---
name: neural-surrogate-affective-neuromodulation-design
description: AI-driven neural surrogate framework for in silico design of cognitive-affective neuromodulation targets. Combines fMRI decoding, deep generative modeling (VDVAE), and constrained latent-space steering with closed-form first-order perturbation solutions. Activation: neuromodulation target design, fMRI latent steering, representational perturbation, affective valence modulation, Good Regulator Theorem, inverse design.
license: MIT
metadata:
  arxiv_id: "2609.27729"
  published: "2026-09-23"
  authors: "Marco Rothermel, Madleen Stenger, Soroush Daftarian, Svenja Jule Francke, Bita Shariatpanahi, José C. García Alanis, Mohammad-Ali Nikouei Mahani, Stefan G. Hofmann, Tim Hahn, Hamidreza Jamalabadi"
  tags: [neuromodulation, fmri-decoding, generative-model, latent-steering, cognitive-affective, control-theory]
---

# Neural Surrogate Framework for In Silico Neuromodulation Target Design

Methodology from arXiv:2609.27729 — "AI-Driven Neural Surrogates for In Silico Design of Cognitive-Affective Neuromodulation Targets" (Rothermel et al., University of Marburg, Sep 2026).

## Overview and Key Innovation

Moves beyond brain **decoding** toward brain **intervention design**: an AI surrogate that proposes candidate representational changes (perturbation directions in a generative latent space) and tests their predicted perceptual consequences behaviorally — *before* any physical stimulation. This is the upstream "target identification" step of the neuromodulation control problem.

**Core insight**: Prediction ≠ intervention. Grounded in Ashby's Good Regulator Theorem — a predictive model becomes relevant to regulation only when it preserves the task-relevant relationships needed to propose interventions and anticipate consequences. The pipeline makes candidate targets explicit, dose-dependent, and falsifiable.

**Pipeline**: 7T fMRI (Natural Scenes Dataset) → subject-specific ridge decoder → VDVAE latent space → empirical attribute-axis steering → CLIP-conditioned diffusion reconstruction → automated + independent human evaluation.

## Core Methodology

### 1. Problem Formulation (Static Inverse Design)

Given latent z ∈ R^n with decoder D and assessor A, define objective f(z) = (A ∘ D)(z). Seek perturbation u maximizing predicted attribute within a norm budget:

```
maximize f(z_org + u)  subject to ||u||_2 ≤ ε
```

This is deliberately **static and local** — no neural dynamics, no stimulation forward model. It is a simplified component of a future closed-loop control system, not a demonstration of neural control.

### 2. Closed-Form First-Order Solution

First-order Taylor expansion + Cauchy–Schwarz gives the locally optimal direction:

```
u*_lin = ε · ∇f(z_org) / ||∇f(z_org)||_2
```

Valid only for small ε (trust-region reasoning); global optimality for the nonlinear assessor–decoder pipeline is NOT guaranteed.

### 3. Empirical Population-Level Steering Direction (black-box compatible)

When exact gradients are unavailable (assessor/decoder as black box), estimate an attribute axis from scored examples:

```
θ = z̄_high − z̄_low    (centroids of top/bottom quartile latents by score)
u_θ = α · θ / ||θ||_2   (sign of α = direction, magnitude = strength)
```

Held-out latent projections along θ tracked assessor scores (valence r=0.812, memorability r=0.834). Caveat: population axis ≠ per-image locally optimal gradient — exact gradients were nearly orthogonal to it, and iterative per-image optimization yields larger assessor gains (Appendix C.1 of paper).

### 4. fMRI-Space Projection via Decoder Adjoint

Subject-specific ridge decoder R maps voxelwise fMRI beta patterns x to latents: z = Rx. The candidate fMRI-pattern direction is:

```
u_x,θ ∝ R^T θ
```

R^T acts as the **adjoint** of the learned decoder — it pulls the latent attribute direction back into measured fMRI feature space for cortical-map visualization. It is NOT an inverse causal model, NOT a stimulation control law. The perturbation itself is applied in latent space (Eq. 16: z_α = ẑ0 + α·θ/||θ||, α ∈ {−4,−2,0,2,4}).

### 5. Multi-Stage Evaluation Protocol

- **Data**: 4 deeply-sampled NSD participants, 8,859 train / 982 test images each; fMRI standardized per subject (divide by 300, z-score with train stats)
- **Surrogate**: frozen ImageNet-64 VDVAE (latent n=91,168), ridge λ=50,000; no fine-tuning
- **Refinement**: frozen Versatile Diffusion (CLIP text+vision conditioning; baseline recon = fMRI-predicted embeddings; perturbed α-series = original-image embeddings to isolate VDVAE perturbation)
- **Assessors**: EmoNet (valence), MemNet (memorability), frozen inference-only
- **Human validation**: 18 independent raters, 7,200 trials, 400 image presentations, 350 ms exposure; leave-one-participant-out linear fatigue correction (block order confound); ICC reliability + binomial/sign-flip directional tests with Holm adjustment
- **Fidelity**: PixCorr + CLIP ViT-L/14 embedding similarity vs α=0 reconstruction

## Key Empirical Findings (honest results)

| Finding | Value | Interpretation |
|---|---|---|
| fMRI→latent decoding (2-way ident.) | 0.79–0.88 (chance 0.5) | Coarse generative structure recovered |
| VDVAE valence steering range | −0.61 → +1.03 SD | Strong graded modulation |
| VDVAE memorability range | −1.34 → +1.45 SD | Strong, slight non-monotonicity at negatives |
| Versatile Diffusion valence | −0.10 → +0.11 SD (n.s.) | **Diffusion compresses valence steering** |
| Versatile Diffusion memorability | −0.67 → +0.54 SD | Attenuated but monotonic |
| Human valence slope | 0.038 SD/unit α (CI 0.003–0.074); 16/18 positive | Directionally confirmed, small effect |
| Human memorability slope | −0.011 (n.s.); 5/18 positive | **Null — perceived memorability did not shift** |
| Baseline human–assessor agreement | valence r=0.30 (p=.058), memorability r=0.10 | Suggestive / weak |
| Fidelity at α=±4 | PixCorr 0.17–0.37 | Extreme perturbations degrade stimuli |

**Stage-dependent transmission**: diffusion models preserve rank ordering of assessor scores while regularizing/overwriting strong mean-level perturbations — sharper images are NOT evidence of stronger neural fidelity. Evaluate steering at each generative stage separately.

**Fidelity-vs-effect trade-off**: practical validity region is moderate α; α=±4 enters degraded stimulus regime where the first-order approximation breaks down.

## Implementation Guidance

1. **Decoder fitting**: per-subject ridge regression (no cross-subject alignment; voxelwise features don't align across individuals). Standardize fMRI with train-set statistics only.
2. **Attribute axis**: compute quartile centroids on training scores; validate with held-out projection correlation before steering.
3. **Dose series**: symmetric α grid around 0; treat extremes as boundary conditions, not extrapolation targets.
4. **Behavioral testing**: independent raters, blind to targets/strengths; pre-registered fatigue correction because fixed block order confounds condition with time.
5. **Denser sampling near zero** (α ∈ {−2,−1,−0.5,0,0.5,1,2}) recommended for future work — behaviorally valid linear regime may be narrower than the automated-response regime.

## Pitfalls and Best Practices

- **Do not claim causal neuromodulation**: no stimulation was delivered; cortical maps are surrogate-derived predictions, not activation effects or validated targets.
- **R^T is an adjoint, not an inverse**: it cannot certify that a pattern is physically inducible. Hardware-alignment gap: inferred directions are high-dimensional and spatially distributed vs limited specificity of non-invasive stimulation.
- **Population axis ≠ local gradient**: single-direction steering extrapolates poorly; combine with trust-region constraints or iterative feedback for large perturbations.
- **Perceived memorability ≠ memory**: use incidental-encoding + delayed recognition designs to test actual memory effects.
- **Diffusion-stage confound**: perturbed refined images used fixed original-image CLIP conditioning — not end-to-end fMRI-only reconstructions. Report which conditioning source was used.
- **Generative hallucination risk**: realistic outputs can reflect category inference rather than faithful reconstruction (Shirakawa et al. caution).

## Applications and Extensions

- **Upstream target identification for TMS/tDCS/DBS research**: falsifiable candidate representational targets, dose-dependent, before hardware commitment
- **Psychiatric intervention design**: negative valence bias (MDD), intrusive memory salience (PTSD) as attribute targets
- **Model–human alignment auditing**: pipeline exposes where automated score changes diverge from human perception
- **Dynamical control extension**: upgrade to xt+1 = f(xt, ut), yt = g(xt) trajectory optimization once stimulation-to-brain forward models exist
- **Generalizes to other assessable attributes**: arousal, food preference, face trustworthiness — any scalar with a pretrained scorer or human ratings (θ is black-box compatible)

## Related Skills

- `mirage-fmri-mental-imagery` — multimodal fMRI encoding/decoding
- `visual-imagery-decoding-fmri` — latent alignment decoding
- `brain-network-controllability` — stimulation control energy
- `ultrasound-neuromodulation-prediction-framework` — biophysical forward modeling

## Source

- arXiv:2609.27729v1 [q-bio.NC, cs.AI, eess.SY], submitted 2026-09-23
- Data: Natural Scenes Dataset (naturalscenesdataset.org); code: neuralsurrogate.com (upon publication)
- Funding: DFG SFB/TRR 393, von Behring-Röntgen Stiftung, ERA-NET NEURON JTC 2024