Diversity Preference Link Rec Eval

---
name: diversity-preference-link-rec-eval
description: Evaluates link recommendation models on social networks by measuring how well they predict future connections while respecting individual users' diversity preferences across profile dimensions. Use when the user wants to benchmark on Large-scale social network datasets, or asks about evaluating this task. Reports F1 Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2205.10689
  bibtex_key: yin2022linkrec
  confidence: medium
---

# diversity-preference-link-rec-eval

> Diversity Preference-Aware Link Recommendation for Online Social Networks — Yin et al. (2022) (arXiv:2205.10689, 2022)

## What this evaluates

Evaluates link recommendation models on social networks by measuring how well they predict future connections while respecting individual users' diversity preferences across profile dimensions.

## Datasets

- **Large-scale social network datasets** — total ?; splits: test (-1)

## Metrics

- `DPMS` — range: [0, 1]
  - Custom Diversity Preference Matching Score; exact formula not provided in the excerpt.
- `Precision` — range: [0, 1]
  - Fraction of recommended friends in the top-k list that are actual positive links.
- `Recall` — range: [0, 1]
  - Fraction of actual positive links successfully recovered in the top-k recommendations.
- `F1 Score` **(primary)** — range: [0, 1]
  - Harmonic mean of Precision and Recall: 2 * (Precision * Recall) / (Precision + Recall).

## Input / output format

**Input**: User profile attributes and existing social graph edges.

**Output**: Top-k ranked candidate friends for recommendation.

## Scoring recipe

```python
def compute_metrics(predictions, gold, k):
    hits = len(set(predictions[:k]) & set(gold))
    prec = hits / k
    rec = hits / len(gold) if len(gold) > 0 else 0
    f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
    return prec, rec, f1
```

## Common pitfalls

- Metrics are reported for varying k (6 to 14), so precision/recall values are not directly comparable across different k settings.
- DPMS is a custom metric; its exact calculation is not defined in the provided text.

## Evidence (verbatim from paper)

> Table A.6: Comparison between DPA-LR and State-of-the-art Link Recommendation Methods:  $k = 6$  to  $k = {14}$  

<table><tr><td>Metric</td><td>Methods</td><td>k=6</td><td>k=8</td><td>k=10</td><td>k=12</td><td>k=14</td></tr><tr><td rowspan=7>DPMS</td><td>DPA-LR</td><td>0.3998</td><td>0.4100</td><td>0.4559</td><td>0.4595</td><td>0.4612</td></tr><tr><td rowspan=2>GCN-LR</td><td>0.1993</td><td>0.2210</td><td>0.2389</td><td>0.2540</td><td>0.2663</td></tr><tr><td>(100.59%)</td><td>(85.52%)</td><td>(90.89%)</td><td>(80.91%)</td><td>(73.19%)</td></tr><tr><td rowspan=2>GraphSage-LR</td><td>0.1577</td><td>0.1751</td><td>0.1895</td><td>0.2018</td><td>0.2124</td></tr><tr><td>(153.58%)</td><td>(134.16%)</td><td>(140.59%)</td><td>(127.69%)</td><td>(117.14%)</td></tr><tr><td rowspan=2>GAT-LR</td><td>0.1642</td><td>0.1833</td><td>0.1992</td><td>0.2126</td><td>0.2243</td></tr><tr><td>(143.53%)</td><td>(123.68%)</td><td>(128.89%)</td><td>(116.10%)</td><td>(105.59%)</td></tr><tr><td rowspan=7>Precision</td><td>DPA-LR</td><td>0.1619</td><td>0.1577</td><td>0.1541</td><td>0.1509</td><td>0.1479</td></tr><tr><td rowspan=2>GCN-LR</td><td>0.1455</td><td>0.1411</td><td>0.1380</td><td>0.1354</td><td>0.1329<

## Citation

```bibtex
@misc{yin2022linkrec,
  title={Diversity Preference-Aware Link Recommendation for Online Social Networks},
  author={Yin et al. (2022)},
  year={2022},
  note={arXiv:2205.10689}
}
```

- arXiv: 2205.10689