Position Leverage Foundational Models For Blackbox Optimization

# Position: Leverage Foundational Models for Black-Box Optimization

**arXiv ID:** 2405.03547
**Authors:** Xingyou Song, Yingtao Tian, Robert Tjarko Lange, Chansoo Lee, Yujin Tang, Yutian Chen
**Published:** 2024-05-06T15:10:46Z
**Abstract:**
Undeniably, Large Language Models (LLMs) have stirred an extraordinary wave of innovation in the machine learning research domain, resulting in substantial impact across diverse fields such as reinforcement learning, robotics, and computer vision. Their incorporation has been rapid and transformative, marking a significant paradigm shift in the field of machine learning research. However, the field of experimental design, grounded on black-box optimization, has been much less affected by such a paradigm shift, even though integrating LLMs with optimization presents a unique landscape ripe for exploration. In this position paper, we frame the field of black-box optimization around sequence-based foundation models and organize their relationship with previous literature. We discuss the most promising ways foundational language models can revolutionize optimization, which include harnessing the vast wealth of information encapsulated in free-form text to enrich task comprehension, utilizing highly flexible sequence models such as Transformers to engineer superior optimization strategies, and enhancing performance prediction over previously unseen search spaces.

## Skill Description

This skill is generated from the arXiv paper: Position: Leverage Foundational Models for Black-Box Optimization (2405.03547).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2405.03547](http://arxiv.org/abs/2405.03547v2)