Log Odds Mutation Scoring

---
name: log-odds-mutation-scoring
description: Evaluates a protein language model's zero-shot capability to score single-site amino acid mutations by comparing contextual likelihoods of wild-type versus mutant residues. It probes how well masked language modeling objectives capture evolutionary and structural constraints for mutant effect prediction. Use when the user has predictions and gold and needs to compute log-odds ratio.
metadata:
  skill_kind: metric
  source_arxiv: 2307.12682
  bibtex_key: jiang2023proprime
  confidence: low
---

# log-odds-mutation-scoring

> Pro-PRIME: A general Temperature-Guided Language model to engineer enhanced Stability and Activity in Proteins — Jiang et al. (2023) (arXiv:2307.12682, 2023)

## What this evaluates

Evaluates a protein language model's zero-shot capability to score single-site amino acid mutations by comparing contextual likelihoods of wild-type versus mutant residues. It probes how well masked language modeling objectives capture evolutionary and structural constraints for mutant effect prediction.

## Datasets

- **Unspecified protein sequences** — total ?; splits: test (-1)

## Metrics

- `log-odds ratio` **(primary)** — range: other
  - Computed as the difference between the log-likelihood of the mutated amino acid and the wild-type amino acid at a specific position, conditioned on the surrounding sequence context.

## Input / output format

**Input**: A protein sequence containing a single-site mutation (wild-type and mutant amino acids at the target position).

**Output**: A scalar log-odds score representing the relative likelihood of the mutation.

## Scoring recipe

```python
def compute_log_odds(wt_seq, mut_seq, model):
    wt_log_prob = model.log_prob(wt_seq)
    mut_log_prob = model.log_prob(mut_seq)
    return mut_log_prob - wt_log_prob
```

## Common pitfalls

- Zero-shot log-odds scores may not directly correlate with experimental stability or activity without calibration.
- Fine-tuning on homologous sequences is unsupervised and can introduce phylogenetic bias if not controlled.

## Evidence (verbatim from paper)

> The mutations are then scored using the log-odds ratio at the mutated position. (See Fig. 1C; the details can be found in the Methods).

## Citation

```bibtex
@misc{jiang2023proprime,
  title={Pro-PRIME: A general Temperature-Guided Language model to engineer enhanced Stability and Activity in Proteins},
  author={Jiang et al. (2023)},
  year={2023},
  note={arXiv:2307.12682}
}
```

- arXiv: 2307.12682