Musals Eval

---
name: musals-eval
description: Evaluates the computational efficiency and alignment quality of a multiple sequence alignment algorithm across genomic and protein datasets. It probes the trade-off between runtime scalability and evolutionary accuracy metrics like distance distortion and gap percentage. Use when the user wants to benchmark on Greengenes 12.10, Greengenes 13.5, PDB, PFam-10k, PFam-100k, PFam-1M, or asks about evaluating this task. Reports runtime, distance distortion.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.15458
  bibtex_key: light2026musals
  confidence: high
---

# musals-eval

> MuSAlS: A Fast Multiple Sequence Alignment Approach Using Hierarchical Clustering — Light et al. (2026) (arXiv:2601.15458, 2026)

## What this evaluates

Evaluates the computational efficiency and alignment quality of a multiple sequence alignment algorithm across genomic and protein datasets. It probes the trade-off between runtime scalability and evolutionary accuracy metrics like distance distortion and gap percentage.

## Datasets

- **Greengenes 12.10** — total ?; splits: test (-1)
- **Greengenes 13.5** — total ?; splits: test (-1)
- **PDB** — total ?; splits: test (-1)
- **PFam-10k** — total 10000; splits: test (10000)
- **PFam-100k** — total 100000; splits: test (100000)
- **PFam-1M** — total 1000000; splits: test (1000000)

## Metrics

- `runtime` **(primary)** — range: other
  - Wall-clock time in seconds required to complete the alignment process.
- `alignment width` — range: other
  - Total number of columns in the resulting multiple sequence alignment.
- `alignment stretch` — range: other
  - Ratio measuring the expansion of the alignment relative to the original sequences.
- `percentage of gaps` — range: percent
  - Proportion of gap characters in the alignment matrix.
- `distance distortion` **(primary)** — range: other
  - Metric quantifying the deviation of pairwise evolutionary distances in the alignment from reference distances.
- `minimum p distance` — range: [0, 1]
  - Minimum pairwise p-distance (proportion of differing sites) across all sequence pairs.
- `mean p distance` — range: [0, 1]
  - Average pairwise p-distance across all sequence pairs.
- `maximum p distance` — range: [0, 1]
  - Maximum pairwise p-distance across all sequence pairs.

## Input / output format

**Input**: Raw nucleotide or protein sequence datasets for benchmarking.

**Output**: Aligned multiple sequence alignment (MSA) output, plus computed statistics for runtime, width, stretch, gap percentage, distance distortion, and p-distances.

## Scoring recipe

```python
def score_msa(alignments, runtimes):
    results = {}
    for alg, (msa, time) in alignments.items():
        results[alg] = {
            'runtime': time,
            'width': msa.num_columns,
            'stretch': msa.width / msa.max_seq_length,
            'gap_pct': msa.count_gaps() / msa.total_cells * 100,
            'distortion': compute_distance_distortion(msa),
            'p_min': min(pairwise_pdist(s1, s2) for s1, s2 in msa.pairs),
            'p_avg': mean(pairwise_pdist(s1, s2) for s1, s2 in msa.pairs),
            'p_max': max(pairwise_pdist(s1, s2) for s1, s2 in msa.pairs)
        }
    return results
```

## Common pitfalls

- Several baseline algorithms fail silently or abort on specific datasets (e.g., MAGUS on proteins, KAlign on non-standard PDB residues), so missing table entries indicate compatibility limits rather than poor performance.
- Optimizing for compactness (low width and gap percentage) can trade off with evolutionary accuracy, as seen when MuSAlS achieves tighter alignments but higher distance distortion on Greengenes datasets.

## Evidence (verbatim from paper)

> We measured performance across various datasets (Greengenes 12.10, Greengenes 13.5, PDB, and PFam subsets) using the evaluation metrics runtime, alignment width, alignment stretch, percentage of gaps, distance distortion, minimum p distance, mean p distance, and maximum p distance.

## Citation

```bibtex
@misc{light2026musals,
  title={MuSAlS: A Fast Multiple Sequence Alignment Approach Using Hierarchical Clustering},
  author={Light et al. (2026)},
  year={2026},
  note={arXiv:2601.15458}
}
```

- arXiv: 2601.15458