Lattice Recurrent Unit Improving Convergence And Statistical Efficiency For Sequence Modeling

# Lattice Recurrent Unit: Improving Convergence and Statistical Efficiency for Sequence Modeling

**arXiv ID:** 1710.02254
**Authors:** Chaitanya Ahuja, Louis-Philippe Morency
**Published:** 2017-10-06T01:52:14Z
**Abstract:**
Recurrent neural networks have shown remarkable success in modeling sequences. However low resource situations still adversely affect the generalizability of these models. We introduce a new family of models, called Lattice Recurrent Units (LRU), to address the challenge of learning deep multi-layer recurrent models with limited resources. LRU models achieve this goal by creating distinct (but coupled) flow of information inside the units: a first flow along time dimension and a second flow along depth dimension. It also offers a symmetry in how information can flow horizontally and vertically. We analyze the effects of decoupling three different components of our LRU model: Reset Gate, Update Gate and Projected State. We evaluate this family on new LRU models on computational convergence rates and statistical efficiency. Our experiments are performed on four publicly-available datasets, comparing with Grid-LSTM and Recurrent Highway networks. Our results show that LRU has better empirical computational convergence rates and statistical efficiency values, along with learning more accurate language models.

## Skill Description

This skill is generated from the arXiv paper: Lattice Recurrent Unit: Improving Convergence and Statistical Efficiency for Sequence Modeling (1710.02254).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:1710.02254](http://arxiv.org/abs/1710.02254v2)