Missing Indicator Eval

---
name: missing-indicator-eval
description: Evaluates the performance of missing data preprocessing strategies (mean imputation, missForest, Gaussian Copula imputation, with and without Missing Indicator Method) across linear, tree-based, and neural network models. It probes how well these methods handle informative versus uninformative missingness patterns in both low and high-dimensional tabular settings. Use when the user wants to benchmark on Synthetic Low-Dimensional, Synthetic High-Dimensional, OpenML (12 subsets), or asks about evaluating this task. Reports RMSE, 1 - AUC, 1 - accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2211.09259
  bibtex_key: [REDACTED]
  confidence: high
---

# missing-indicator-eval

> The Missing Indicator Method: From Low to High Dimensions — Van Ness et al. (2022) (arXiv:2211.09259, 2022)

## What this evaluates

Evaluates the performance of missing data preprocessing strategies (mean imputation, missForest, Gaussian Copula imputation, with and without Missing Indicator Method) across linear, tree-based, and neural network models. It probes how well these methods handle informative versus uninformative missingness patterns in both low and high-dimensional tabular settings.

## Datasets

- **Synthetic Low-Dimensional** — total 10000; splits: train (-1), test (-1); repo https://github.com/mvanness354/missing_indicator_method
- **Synthetic High-Dimensional** — total 10000; splits: train (-1), test (-1); repo https://github.com/mvanness354/missing_indicator_method
- **OpenML (12 subsets)** — total ?; splits: train (-1), test (-1); repo https://github.com/mvanness354/missing_indicator_method

## Metrics

- `RMSE` **(primary)** — range: other
  - Root Mean Squared Error: sqrt(mean((y_true - y_pred)^2)). Used for regression tasks.
- `1 - AUC` **(primary)** — range: [0, 1]
  - One minus the Area Under the Receiver Operating Characteristic Curve. Used for binary classification tasks.
- `1 - accuracy` **(primary)** — range: [0, 1]
  - One minus classification accuracy. Used for multiclass classification tasks.

## Input / output format

**Input**: Tabular feature matrix X with missing values, where missingness is generated via a self-masking mechanism P(R_j=0|X_j) = 1/(1+exp(-λ_j X_j)). Features are standardized over observed entries.

**Output**: Continuous predictions for regression, or class probabilities/labels for binary and multiclass classification.

## Scoring recipe

```python
def compute_metric(y_true, y_pred, task_type):
    if task_type == 'regression':
        return np.sqrt(np.mean((y_true - y_pred) ** 2))
    elif task_type == 'binary':
        return 1 - roc_auc_score(y_true, y_pred)
    elif task_type == 'multiclass':
        return 1 - accuracy_score(y_true, y_pred)
```

## Common pitfalls

- Assuming MIM universally improves performance; it can cause overfitting in high dimensions if uninformative indicators are retained (SMIM is required to filter them).
- Misinterpreting tree-based model results; XGBoost with mean imputation already captures missingness signals via feature binning, so MIM provides little additional benefit unless non-constant imputation is used.
- Failing to standardize features over observed entries before imputation, which breaks the theoretical equivalence of mean imputation to 0-imputation used in the experiments.

## Evidence (verbatim from paper)

> For performance metrics, we use RMSE for regression tasks, 1 - AUC for binary classification tasks, and 1 - accuracy for multiclass classification tasks (so in all cases lower is better).

## Citation

```bibtex
@misc{vaness2022missing,
  title={The Missing Indicator Method: From Low to High Dimensions},
  author={Van Ness et al. (2022)},
  year={2022},
  note={arXiv:2211.09259}
}
```

- arXiv: 2211.09259