Neural Constraint Satisfaction Hierarchical Abstraction For Combinatorial Generalization In Object Rearrangement

# Neural Constraint Satisfaction: Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement

**arXiv ID:** 2303.11373
**Authors:** Michael Chang, Alyssa L. Dayan, Franziska Meier, Thomas L. Griffiths, Sergey Levine, Amy Zhang
**Published:** 2023-03-20T18:19:36Z
**Abstract:**
Object rearrangement is a challenge for embodied agents because solving these tasks requires generalizing across a combinatorially large set of configurations of entities and their locations. Worse, the representations of these entities are unknown and must be inferred from sensory percepts. We present a hierarchical abstraction approach to uncover these underlying entities and achieve combinatorial generalization from unstructured visual inputs. By constructing a factorized transition graph over clusters of entity representations inferred from pixels, we show how to learn a correspondence between intervening on states of entities in the agent's model and acting on objects in the environment. We use this correspondence to develop a method for control that generalizes to different numbers and configurations of objects, which outperforms current offline deep RL methods when evaluated on simulated rearrangement tasks.

## Skill Description

This skill is generated from the arXiv paper: Neural Constraint Satisfaction: Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement (2303.11373).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2303.11373](http://arxiv.org/abs/2303.11373v1)