**arXiv ID:** 2105.14849 **Authors:** Albert Zeyer, Ralf Schlüter, Hermann Ney **Published:** 2021-05-31T10:03:14Z **Abstract:** The peaky behavior of CTC models is well known experimentally. However, an understanding about why peaky behavior occurs is missing, and whether this is a good property. We provide a formal analysis of the peaky behavior and gradient descent convergence properties of the CTC loss and related training criteria. Our analysis provides a deep understanding why peaky beh...
Installs into .claude/skills of the current project.
Sponsors
SSkills DirectorySkills Directory
Know which skills are safe — weekly.
SSkills DirectorySkills Directory
Is that skill safe to install?
Are you the author of Why Does Ctc Result In Peaky Behavior?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/hiyenwong-why-does-ctc-result-in-peaky-behavior)
# Why does CTC result in peaky behavior?
**arXiv ID:** 2105.14849
**Authors:** Albert Zeyer, Ralf Schlüter, Hermann Ney
**Published:** 2021-05-31T10:03:14Z
**Abstract:**
The peaky behavior of CTC models is well known experimentally. However, an understanding about why peaky behavior occurs is missing, and whether this is a good property. We provide a formal analysis of the peaky behavior and gradient descent convergence properties of the CTC loss and related training criteria. Our analysis provides a deep understanding why peaky behavior occurs and when it is suboptimal. On a simple example which should be trivial to learn for any model, we prove that a feed-forward neural network trained with CTC from uniform initialization converges towards peaky behavior with a 100% error rate. Our analysis further explains why CTC only works well together with the blank label. We further demonstrate that peaky behavior does not occur on other related losses including a label prior model, and that this improves convergence.
## Skill Description
This skill is generated from the arXiv paper: Why does CTC result in peaky behavior? (2105.14849).
## How to Use
[To be filled in by the user or by future automation]
## References
- [arXiv:2105.14849](http://arxiv.org/abs/2105.14849v2)
