Inductive Link Prediction Eval

---
name: inductive-link-prediction-eval
description: Evaluates a model's ability to predict missing links in knowledge graphs using only topological path information, without relying on entity embeddings. It tests inductive generalization by training on one graph and testing on a disjoint graph with unseen entities. Use when the user wants to benchmark on WN18RR, FB15K-237, NELL-995 (inductive versions v1-v4), or asks about evaluating this task. Reports Hits@1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2312.10293
  bibtex_key: zhang2023inductive
  confidence: high
---

# inductive-link-prediction-eval

> Inductive Link Prediction in Knowledge Graphs using Path-based Neural Networks — Zhang et al. (2023) (arXiv:2312.10293, 2023)

## What this evaluates

Evaluates a model's ability to predict missing links in knowledge graphs using only topological path information, without relying on entity embeddings. It tests inductive generalization by training on one graph and testing on a disjoint graph with unseen entities.

## Datasets

- **WN18RR, FB15K-237, NELL-995 (inductive versions v1-v4)** — total ?; splits: train (-1), test (-1)

## Metrics

- `AUC-PR` — range: percent
  - Replace the source or target entity of each test triple with a random entity to form negative triples. Score the positive test triples and an equal number of negative triples, then compute the area under the precision-recall curve.
- `Hits@10` — range: percent
  - Rank each test triple against corrupted candidates. In entity-corrupted ranking, candidates are formed by replacing the source or target entity with a random entity (50 negatives per test triple). The metric is the percentage of true triples ranked in the top-10 positions.
- `Hits@1` **(primary)** — range: percent
  - Rank each test triple against corrupted candidates where the relation is replaced by other relations in the graph. The metric is the percentage of true triples ranked in the top-1 position.
- `Hits@3` — range: percent
  - Same as Hits@1 but measures the percentage of true triples ranked in the top-3 positions under relation-corrupted ranking.

## Input / output format

**Input**: Path sequences connecting source and target entities (connection-based) or out-reaching paths from source/target entities (subgraph-based), optionally paired with relation embeddings.

**Output**: A scalar similarity score (inner product) for each candidate triple, used to rank candidates.

## Scoring recipe

```python
def evaluate(dataset, model):
    hits_at_10 = []
    auc_pr_pairs = []
    for triple in dataset.test_triples:
        # Entity-corrupted negatives (50 fixed)
        negatives = generate_entity_corrupted(triple, k=50)
        candidates = [triple] + negatives
        scores = [model.score(c) for c in candidates]
        true_score = scores[0]
        rank = sum(1 for s in scores[1:] if s > true_score) + 1
        hits_at_10.append(1 if rank <= 10 else 0)
        # AUC-PR uses 1:1 ratio
        neg_1 = generate_entity_corrupted(triple, k=1)[0]
        auc_pr_pairs.append((true_score, 1))
        auc_pr_pairs.append((model.score(neg_1), 0))
    return {
        'Hits@10': sum(hits_at_10) / len(hits_at_10) * 100,
        'AUC-PR': compute_auc(auc_pr_pairs) * 100
    }
```

## Common pitfalls

- Entity-corrupted and relation-corrupted ranking use fundamentally different negative sampling strategies (fixed 50 negatives vs. variable negatives based on relation count), making Hits@k values incomparable across settings.
- Inductive splits enforce strictly disjoint entity sets between training and inference graphs, so models must rely purely on topological paths rather than pre-trained entity embeddings.
- AUC-PR evaluation uses a strict 1:1 positive-to-negative ratio, which differs from standard open-world link prediction benchmarks and can inflate or deflate scores depending on dataset density.

## Evidence (verbatim from paper)

> We apply both classification metric and ranking metric to evaluate the performance of our model. For classification metric, we use the area under the precision-recall curve (AUC-PR) following GraIL. That is, we replace the source or target entity of each test triple with a random entity to form a negative triple. Then, we score the positive test triples with an equal number of negative triples to calculate AUC-PR. For the ranking metric, however, there seems to be two different settings. The first setting is purposed in GraIL... Accordingly, Hits@10 (the rate of true test triples ranked top-10 in all performed rankings) is calculated with respect to all test triples. We refer to this setting as entity-corrupted ranking.

## Citation

```bibtex
@misc{zhang2023inductive,
  title={Inductive Link Prediction in Knowledge Graphs using Path-based Neural Networks},
  author={Zhang et al. (2023)},
  year={2023},
  note={arXiv:2312.10293}
}
```

- arXiv: 2312.10293