Freshretailnet 50k Eval

---
name: freshretailnet-50k-eval
description: This benchmark evaluates models' ability to recover latent demand during stockout periods in perishable retail. It probes whether algorithms can disentangle true consumption patterns from supply-induced censoring using hourly temporal data and contextual covariates. Success is measured by prediction accuracy, bias mitigation, and the decoupling of recovered demand from stockout ratios. Use when the user wants to benchmark on FreshRetailNet-50K, or asks about evaluating this task. Reports WAPE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.16319
  bibtex_key: wang2025freshretailnet
  confidence: high
---

# freshretailnet-50k-eval

> FreshRetailNet-50K: A Stockout-Annotated Censored Demand Dataset for Latent Demand Recovery and Forecasting in Fresh Retail — Wang et al. (2025) (arXiv:2505.16319, 2025)

## What this evaluates

This benchmark evaluates models' ability to recover latent demand during stockout periods in perishable retail. It probes whether algorithms can disentangle true consumption patterns from supply-induced censoring using hourly temporal data and contextual covariates. Success is measured by prediction accuracy, bias mitigation, and the decoupling of recovered demand from stockout ratios.

## Datasets

- **FreshRetailNet-50K** — total 50000; splits: (unstated); repo https://github.com/Dingdong-Inc/frn-50k-baseline

## Metrics

- `WAPE` **(primary)** — range: percent
  - Weighted Absolute Percentage Error: sum(|d_t - y_t|) / sum(y_t). Measures magnitude accuracy of recovered demand.
- `WPE` — range: percent
  - Weighted Percentage Error: sum(d_t - y_t) / sum(y_t). Measures bias direction (over/under-prediction).
- `rho_DS` — range: [-1, 1]
  - Decoupling Score: weighted Pearson correlation between stockout ratios and recovered demand across store-product pairs, where weights are proportional to mean sales.

## Input / output format

**Input**: Hourly time-series sales data with contextual covariates (weather, promotions, holidays) for specific store-product pairs.

**Output**: Predicted true demand values (y_t) for each hourly time step t.

## Scoring recipe

```python
import numpy as np
def compute_metrics(preds, gold, stockout_ratios, mean_sales):
    wape = np.sum(np.abs(gold - preds)) / np.sum(gold)
    wpe = np.sum(gold - preds) / np.sum(gold)
    weights = mean_sales / np.sum(mean_sales)
    pearson_corr = np.corrcoef(stockout_ratios, preds)[0, 1]
    rho_ds = np.sum(weights * pearson_corr)
    return wape, wpe, rho_ds
```

## Common pitfalls

- Evaluating directly on observed sales during stockouts instead of recovering latent demand, which violates the MNAR simulation protocol.
- Focusing exclusively on WAPE while ignoring WPE, thereby missing systematic bias that causes inventory cascades.
- Treating all store-product pairs equally in the Decoupling Score without applying the mean-sales weighting (w_i), which skews results toward high-volume items.

## Evidence (verbatim from paper)

> Model performance is quantified through Weighted Absolute Percentage Error(WAPE) and Weighted Percentage Error(WPE): $$ \mathrm {W A P E} = \frac {\sum_ {t} \left| d _ {t} - y _ {t} \right|}{\sum_ {t} y _ {t}} \quad (\text {m a g n i t u d e a c c u r a c y}) \tag {4} $$ $$ \mathrm {W P E} = \frac {\sum_ {t} \left(d _ {t} - y _ {t}\right)}{\sum_ {t} y _ {t}} \quad (\text {b i a s d i r e c t i o n}) \tag {5} $$

## Citation

```bibtex
@misc{wang2025freshretailnet,
  title={FreshRetailNet-50K: A Stockout-Annotated Censored Demand Dataset for Latent Demand Recovery and Forecasting in Fresh Retail},
  author={Wang et al. (2025)},
  year={2025},
  note={arXiv:2505.16319}
}
```

- arXiv: 2505.16319