Localized Weather Prediction Eval

---
name: localized-weather-prediction-eval
description: Evaluates the ability of neural network architectures (KANs, TKANs, RNNs) to forecast localized weather variables (temperature, precipitation, pressure) one day ahead. Probes nonlinear time-series modeling and regression accuracy under varying data distributions, such as low precipitation versus high temperature variance. Use when the user wants to benchmark on Abidjan, Kigali, or asks about evaluating this task. Reports R².
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.22686
  bibtex_key: akazan2025localized
  confidence: high
---

# localized-weather-prediction-eval

> Localized Weather Prediction Using Kolmogorov-Arnold Network-Based Models and Deep RNNs — Akazan et al. (2025) (arXiv:2505.22686, 2025)

## What this evaluates

Evaluates the ability of neural network architectures (KANs, TKANs, RNNs) to forecast localized weather variables (temperature, precipitation, pressure) one day ahead. Probes nonlinear time-series modeling and regression accuracy under varying data distributions, such as low precipitation versus high temperature variance.

## Datasets

- **Abidjan** — total ?; splits: unspecified (-1); repo https://github.com/AngeClementAkazan/Localized-Weather-Prediction-Using-KAN-and-DeepRNNs
- **Kigali** — total ?; splits: unspecified (-1); repo https://github.com/AngeClementAkazan/Localized-Weather-Prediction-Using-KAN-and-DeepRNNs

## Metrics

- `MSE` — range: other
  - Mean Squared Error: average of the squares of the errors between predicted and actual values.
- `RMSE` — range: other
  - Root Mean Squared Error: square root of MSE, providing error in the same units as the target variable.
- `MAE` — range: other
  - Mean Absolute Error: average of the absolute differences between predicted and actual values.
- `R²` **(primary)** — range: [0, 1]
  - Coefficient of Determination: 1 - (SS_res / SS_tot), representing the proportion of variance in the dependent variable predictable from the independent variables.
- `MAPE` — range: percent
  - Mean Absolute Percentage Error: average of absolute percentage errors between predicted and actual values. Note: highly sensitive to near-zero ground truth values.

## Input / output format

**Input**: Historical time-series weather data (temperature, precipitation, pressure) for a specific city.

**Output**: Predicted values for temperature (°C), precipitation (mm), and pressure (kPa) for the next day.

## Scoring recipe

```python
import numpy as np
def compute_metrics(y_true, y_pred):
    y_true, y_pred = np.array(y_true), np.array(y_pred)
    mse = np.mean((y_true - y_pred) ** 2)
    rmse = np.sqrt(mse)
    mae = np.mean(np.abs(y_true - y_pred))
    ss_res = np.sum((y_true - y_pred) ** 2)
    ss_tot = np.sum((y_true - np.mean(y_true)) ** 2)
    r2 = 1 - (ss_res / ss_tot)
    mape = np.mean(np.abs((y_true - y_pred) / y_true)) * 100
    return {'MSE': mse, 'RMSE': rmse, 'MAE': mae, 'R²': r2, 'MAPE': mape}
```

## Common pitfalls

- MAPE values are heavily inflated for precipitation due to near-zero ground truth values, making it misleading for low-rainfall conditions.
- Model performance varies drastically by variable: KAN dominates temperature (R² > 0.99) but underperforms on pressure compared to standard RNNs.
- No explicit train/validation/test splits or dataset sizes are reported, making reproducibility of split-dependent metrics difficult.

## Evidence (verbatim from paper)

> The notably high MAPE values across all models are a result of the inherently low precipitation levels in both cities, which inflate relative error calculations.

## Citation

```bibtex
@misc{akazan2025localized,
  title={Localized Weather Prediction Using Kolmogorov-Arnold Network-Based Models and Deep RNNs},
  author={Akazan et al. (2025)},
  year={2025},
  note={arXiv:2505.22686}
}
```

- arXiv: 2505.22686