Unsupervised Learning And Exploration Of Reachable Outcome Space

# Unsupervised Learning and Exploration of Reachable Outcome Space

**arXiv ID:** 1909.05508
**Authors:** Giuseppe Paolo, Alban Laflaquière, Alexandre Coninx, Stephane Doncieux
**Published:** 2019-09-12T08:47:44Z
**Abstract:**
Performing Reinforcement Learning in sparse rewards settings, with very little prior knowledge, is a challenging problem since there is no signal to properly guide the learning process. In such situations, a good search strategy is fundamental. At the same time, not having to adapt the algorithm to every single problem is very desirable. Here we introduce TAXONS, a Task Agnostic eXploration of Outcome spaces through Novelty and Surprise algorithm. Based on a population-based divergent-search approach, it learns a set of diverse policies directly from high-dimensional observations, without any task-specific information. TAXONS builds a repertoire of policies while training an autoencoder on the high-dimensional observation of the final state of the system to build a low-dimensional outcome space. The learned outcome space, combined with the reconstruction error, is used to drive the search for new policies. Results show that TAXONS can find a diverse set of controllers, covering a good part of the ground-truth outcome space, while having no information about such space.

## Skill Description

This skill is generated from the arXiv paper: Unsupervised Learning and Exploration of Reachable Outcome Space (1909.05508).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:1909.05508](http://arxiv.org/abs/1909.05508v4)