Monroe Molecular Foundation Model

---
name: monroe-molecular-foundation-model
description: "Monroe molecular foundation model for in-context probabilistic inference using prior-data-fitted models (TabPFN). Use when performing bioassay activity prediction with data-limited scenarios requiring general-purpose chemical knowledge."
metadata:
  arxiv_id: "2608.18982"
  authors: "Various Authors"
  published: "2026-08-21"
  tags: [molecular-foundation-model, in-context-learning, bioassay-prediction, TabPFN, quantum-chemistry]
license: Complete terms in LICENSE.txt
---

# Monroe: Molecular Foundation Model

## Overview
Monroe is a molecular foundation model (MFM) designed for bioassay activity prediction in data-limited scenarios. It combines large-scale pre-training on quantum chemistry data with innovative downstream adaptation using prior-data-fitted models (TabPFN) for in-context probabilistic inference.

## Key Innovations

### Scale and Pre-training
- **81+ million molecules** from PM6 quantum chemistry dataset
- **Improved graph representation** of stereochemistry
- **Enhanced training losses**: conformer denoising and embedding decorrelation
- **Advanced multi-task learning** framework

### Downstream Adaptation
- **Prior-data-fitted model (TabPFN)** for in-context prediction
- Enables few-shot learning without fine-tuning
- **Probabilistic inference** with uncertainty quantification
- **Generalizes beyond Monroe** to improve other MFMs (MiniMol_PFN, CheMeleon_PFN)

## Architecture Components

### Molecular Graph Representation
Monroe uses an enhanced graph neural network architecture with:

1. **Stereochemistry-aware encoding**:
   - Explicit representation of chiral centers
   - Geometric constraints for 3D structure preservation
   - Conformer-aware message passing

2. **Multi-scale feature extraction**:
   - Atom-level features (element, hybridization, charge)
   - Bond-level features (type, stereochemistry, conjugation)
   - Global molecular properties (logP, molecular weight, etc.)

3. **Conformer denoising objective**:
   ```python
   # Denoise 3D conformer coordinates during pre-training
   def conformer_denoising_loss(predicted_coords, true_coords, mask):
       return mse_loss(predicted_coords[mask], true_coords[mask])
   ```

4. **Embedding decorrelation loss**:
   ```python
   # Reduce redundancy in learned representations
   def decorrelation_loss(embeddings):
       cov_matrix = torch.cov(embeddings.T)
       off_diag_sum = torch.sum(torch.abs(cov_matrix)) - torch.sum(torch.abs(torch.diag(cov_matrix)))
       return off_diag_sum
   ```

### In-Context Probabilistic Inference with TabPFN

#### TabPFN Integration
TabPFN (Prior-Data-Fitted Network) enables in-context learning by treating the training set as part of the input:

```python
def monroe_predict_with_tabpfn(query_molecule, support_set, support_labels):
    """
    Perform in-context prediction using Monroe embeddings + TabPFN
    
    Args:
        query_molecule: Molecule to predict (SMILES or graph)
        support_set: List of molecules with known activities
        support_labels: Corresponding activity labels
        
    Returns:
        Predicted activity distribution (mean, variance)
    """
    # Get Monroe embeddings
    query_emb = monroe.encode(query_molecule)
    support_embs = [monroe.encode(mol) for mol in support_set]
    
    # Format for TabPFN
    tabpfn_input = {
        'X': torch.stack(support_embs),
        'y': torch.tensor(support_labels),
        'x': query_emb
    }
    
    # TabPFN inference
    prediction = tabpfn.predict(**tabpfn_input)
    return prediction.mean, prediction.variance
```

#### Pairwise Comparison Framework
Monroe evaluation uses statistically rigorous pairwise comparisons:

```python
def pairwise_significance_test(model_a, model_b, test_set, num_trials=1000):
    """Statistically significant performance comparison."""
    scores_a, scores_b = [], []
    
    for _ in range(num_trials):
        subset = random_subset(test_set, size=100)
        score_a = evaluate(model_a, subset)
        score_b = evaluate(model_b, subset)
        scores_a.append(score_a)
        scores_b.append(score_b)
    
    # Wilcoxon signed-rank test
    p_value = wilcoxon_test(scores_a, scores_b)
    effect_size = cliff_delta(scores_a, scores_b)
    
    return {
        'p_value': p_value,
        'effect_size': effect_size,
        'significant': p_value < 0.05,
        'model_a_better': np.mean(scores_a) > np.mean(scores_b)
    }
```

## Implementation Workflow

### Step 1: Pre-training Setup
```python
# Load PM6 quantum chemistry dataset
dataset = load_pm6_dataset('path/to/pm6_81m_molecules/')

# Initialize Monroe model
model = Monroe(
    hidden_dim=1024,
    num_layers=12,
    dropout=0.1,
    use_stereochemistry=True
)

# Multi-task training objectives
objectives = [
    'mask_atom_prediction',
    'mask_bond_prediction', 
    'conformer_denoising',
    'embedding_decorrelation',
    'multi_task_property_prediction'
]
```

### Step 2: Pre-training Execution
```python
optimizer = AdamW(model.parameters(), lr=1e-4)
scheduler = CosineAnnealingLR(optimizer, T_max=num_epochs)

for epoch in range(num_epochs):
    for batch in dataloader:
        # Forward pass with multiple objectives
        outputs = model(batch)
        
        # Calculate combined loss
        total_loss = 0
        for obj in objectives:
            loss = calculate_loss(obj, outputs, batch)
            total_loss += loss_weight[obj] * loss
            
        # Backward pass
        optimizer.zero_grad()
        total_loss.backward()
        optimizer.step()
        scheduler.step()
```

### Step 3: Downstream Adaptation with TabPFN
```python
# Prepare support set for in-context learning
support_molecules = get_few_shot_support_set(task='kinase_inhibition')
support_labels = get_support_labels(support_molecules)

# Encode with Monroe
support_embeddings = []
for mol in support_molecules:
    emb = monroe.encode(mol)
    support_embeddings.append(emb)

# Initialize TabPFN
tabpfn = TabPFN(
    input_dim=monroe.hidden_dim,
    output_dim=1,
    num_layers=4,
    hidden_dim=512
)

# In-context prediction
def predict_activity(query_smiles):
    query_emb = monroe.encode(query_smiles)
    prediction = tabpfn.predict(
        X=torch.stack(support_embeddings),
        y=torch.tensor(support_labels),
        x=query_emb
    )
    return prediction
```

### Step 4: Evaluation on Benchmarks
```python
# Polaris benchmarks
polaris_results = evaluate_on_polaris_benchmarks(monroe_tabpfn)

# Activity cliff benchmarks (molecular discovery focus)
cliff_results = evaluate_on_activity_cliffs(monroe_tabpfn)

# Statistical significance testing
baseline_models = [minimol, chemeleon, existing_mfm]
for baseline in baseline_models:
    comparison = pairwise_significance_test(monroe_tabpfn, baseline, cliff_benchmarks)
    print(f"vs {baseline.name}: p={comparison.p_value:.4f}, "
          f"effect={comparison.effect_size:.3f}, "
          f"better={comparison.model_a_better}")
```

## Performance Results

### Polaris Benchmarks
- **Matches or exceeds** existing molecular foundation models
- Strong performance across diverse assay types
- Robust to distribution shifts

### Activity Cliff Benchmarks
- **Significant improvements** over prior methods
- Better generalization to novel molecular scaffolds
- Improved hit identification for drug discovery

### Transfer Learning Results
- **MiniMol_PFN**: MiniMol + TabPFN downstream adaptation
- **CheMeleon_PFN**: CheMeleon + TabPFN downstream adaptation
- Both achieve **new state-of-the-art performance**
- Demonstrates **generalizability** of TabPFN approach

## Best Practices

### Data Preparation
- **Quality control**: Filter molecules with problematic functional groups
- **Scaffold splitting**: Ensure proper train/test separation
- **Activity thresholding**: Use consistent activity definitions across datasets

### Pre-training Considerations
- **Batch size**: Use large batches (≥4096 molecules) for stable training
- **Learning rate**: Start with 1e-4, warmup for first 5% of training
- **Regularization**: Apply dropout (0.1) and gradient clipping (1.0)

### TabPFN Configuration
- **Support set size**: 16-64 examples typically sufficient
- **Uncertainty calibration**: Validate predictive uncertainty on held-out data
- **Ensemble predictions**: Average multiple TabPFN runs for robustness

### Evaluation Protocol
- **Multiple random seeds**: Report mean ± std across ≥5 seeds
- **Statistical testing**: Use pairwise comparisons with significance testing
- **Domain-specific metrics**: Include both general and task-specific metrics

## Activation Keywords
- monroe
- molecular foundation model
- in-context learning
- TabPFN
- bioassay prediction
- activity cliffs
- quantum chemistry pre-training

## References
- Original paper: https://arxiv.org/abs/2608.18982
- Related skills: molecular-qubit-vibronic-engineering, quantum-pave-chemistry