Prism Eval

---
name: prism-eval
description: Evaluates fine-grained, multi-aspect-aware paper-to-paper retrieval by decomposing long-form query papers into aspect-specific views and segmenting candidate papers into section-level representations for targeted retrieval. Use when the user wants to benchmark on SciFullBench, PatentFullBench, or asks about evaluating this task. Reports Recall@K.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2507.10057
  bibtex_key: park2025prism
  confidence: high
---

# prism-eval

> PRISM: Fine-Grained Paper-to-Paper Retrieval with Multi-Aspect-Aware Query Optimization — Park et al. (2025) (arXiv:2507.10057, 2025)

## What this evaluates

Evaluates fine-grained, multi-aspect-aware paper-to-paper retrieval by decomposing long-form query papers into aspect-specific views and segmenting candidate papers into section-level representations for targeted retrieval.

## Datasets

- **SciFullBench** — total ?; splits: ICLR-References (-1), ICLR-Citations (-1), NeurIPS-Citations (-1), ACL-Citations (-1); repo https://github.com/psw0021/PRISM
- **PatentFullBench** — total ?; splits: References (-1), Citations (-1); repo https://github.com/psw0021/PRISM

## Metrics

- `Recall@K` **(primary)** — range: [0, 1]
  - The proportion of ground-truth relevant documents that appear in the top-K retrieved results. Calculated as |Top-K ∩ Gold| / |Gold|.
- `CHV Ratio` — range: other
  - Convex Hull Volume Ratio measuring semantic coverage diversity in the latent embedding space, computed as the ratio of coverage between two experimental configurations over the same test set.

## Input / output format

**Input**: Query paper (abstract or full text/chunks) and a corpus of candidate papers (abstracts or full text/chunks).

**Output**: Ranked list of candidate papers.

## Scoring recipe

```python
def recall_at_k(predictions, gold, k):
    top_k = predictions[:k]
    relevant = sum(1 for p in top_k if p in gold)
    return relevant / len(gold)
```

## Common pitfalls

- Full papers are truncated to the maximum context length of the embedding model.
- Iterative retrieval explicitly skips already-selected candidates to prevent redundancy.
- Aspect-aware cache is required to avoid redundant exploration within each aspect branch.

## Evidence (verbatim from paper)

> Table 1 presents the main results, where CoR outperforms all baselines across various settings, validating the effectiveness of our proposed framework for full paper-to-paper retrieval. Notably, when using the same domain-agnostic retriever, CoR surpasses abstract-to-abstract (A2A) baselines by an average of 5.16% in Recall, demonstrating that simply relying on abstracts is suboptimal compared to our aspect-driven approach.

## Citation

```bibtex
@misc{park2025prism,
  title={PRISM: Fine-Grained Paper-to-Paper Retrieval with Multi-Aspect-Aware Query Optimization},
  author={Park et al. (2025)},
  year={2025},
  note={arXiv:2507.10057}
}
```

- arXiv: 2507.10057