Confer Eval

---
name: confer-eval
description: Evaluates continual learning methods for facial expression recognition under incremental, non-i.i.d. data settings. It probes a model's ability to learn new expressions sequentially while preserving prior knowledge, measuring both forward adaptation and backward forgetting. Use when the user wants to benchmark on CK+ (Extended Cohn-Kanade), or asks about evaluating this task. Reports Average Accuracy Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2305.06448
  bibtex_key: churamani2023confer
  confidence: high
---

# confer-eval

> Continual Facial Expression Recognition: A Benchmark — Churamani et al. (2023) (arXiv:2305.06448, 2023)

## What this evaluates

Evaluates continual learning methods for facial expression recognition under incremental, non-i.i.d. data settings. It probes a model's ability to learn new expressions sequentially while preserving prior knowledge, measuring both forward adaptation and backward forgetting.

## Datasets

- **CK+ (Extended Cohn-Kanade)** — total 1300; splits: test (-1)

## Metrics

- `Average Accuracy Score` **(primary)** — range: [0, 1]
  - Mean of per-task classification accuracies evaluated sequentially after each new task is introduced. Higher is better.
- `Catastrophic Forgetting` — range: other
  - Difference between accuracy on previous tasks immediately after learning them and accuracy after learning subsequent tasks. Lower values indicate less forgetting; negative values indicate positive transfer.

## Input / output format

**Input**: RGB facial expression images.

**Output**: Predicted facial expression class label.

## Scoring recipe

```python
def compute_avg_acc(preds, golds, task_ids):
    accs = []
    for t in sorted(set(task_ids)):
        mask = task_ids == t
        accs.append(np.mean(preds[mask] == golds[mask]))
    return np.mean(accs)

def compute_cf(preds, golds, task_ids, prev_accs):
    # CF at task t = avg_acc_on_prev_tasks_after_t - prev_accs[t-1]
    # Lower is better.
    pass
```

## Common pitfalls

- Task-IL assumes the model knows which task it is currently processing, while Class-IL requires predicting from all previously seen classes without task IDs.
- CF scores can be negative, indicating positive transfer or regularization effects rather than forgetting.
- Task ordering significantly impacts rehearsal-based methods; results should be averaged over multiple class orderings.

## Evidence (verbatim from paper)

> Table 3 presents model performances, in terms of the Acc (Average Accuracy Score) under the Task-IL settings with CK+, both without and with data augmentation. Table 4 shows CF (Catastrophic Forgetting) scores across all the methods.

## Citation

```bibtex
@misc{churamani2023confer,
  title={Continual Facial Expression Recognition: A Benchmark},
  author={Churamani et al. (2023)},
  year={2023},
  note={arXiv:2305.06448}
}
```

- arXiv: 2305.06448