Switchbased Active Deep Dynaq Efficient Adaptive Planning For Taskcompletion Dialogue Policy Learning

# Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning

**arXiv ID:** 1811.07550
**Authors:** Yuexin Wu, Xiujun Li, Jingjing Liu, Jianfeng Gao, Yiming Yang
**Published:** 2018-11-19T08:23:34Z
**Abstract:**
Training task-completion dialogue agents with reinforcement learning usually requires a large number of real user experiences. The Dyna-Q algorithm extends Q-learning by integrating a world model, and thus can effectively boost training efficiency using simulated experiences generated by the world model. The effectiveness of Dyna-Q, however, depends on the quality of the world model - or implicitly, the pre-specified ratio of real vs. simulated experiences used for Q-learning. To this end, we extend the recently proposed Deep Dyna-Q (DDQ) framework by integrating a switcher that automatically determines whether to use a real or simulated experience for Q-learning. Furthermore, we explore the use of active learning for improving sample efficiency, by encouraging the world model to generate simulated experiences in the state-action space where the agent has not (fully) explored. Our results show that by combining switcher and active learning, the new framework named as Switch-based Active Deep Dyna-Q (Switch-DDQ), leads to significant improvement over DDQ and Q-learning baselines in both simulation and human evaluations.

## Skill Description

This skill is generated from the arXiv paper: Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning (1811.07550).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:1811.07550](http://arxiv.org/abs/1811.07550v1)