Why Does Ctc Result In Peaky Behavior

# Why does CTC result in peaky behavior?

**arXiv ID:** 2105.14849
**Authors:** Albert Zeyer, Ralf Schlüter, Hermann Ney
**Published:** 2021-05-31T10:03:14Z
**Abstract:**
The peaky behavior of CTC models is well known experimentally. However, an understanding about why peaky behavior occurs is missing, and whether this is a good property. We provide a formal analysis of the peaky behavior and gradient descent convergence properties of the CTC loss and related training criteria. Our analysis provides a deep understanding why peaky behavior occurs and when it is suboptimal. On a simple example which should be trivial to learn for any model, we prove that a feed-forward neural network trained with CTC from uniform initialization converges towards peaky behavior with a 100% error rate. Our analysis further explains why CTC only works well together with the blank label. We further demonstrate that peaky behavior does not occur on other related losses including a label prior model, and that this improves convergence.

## Skill Description

This skill is generated from the arXiv paper: Why does CTC result in peaky behavior? (2105.14849).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2105.14849](http://arxiv.org/abs/2105.14849v2)