-
Predicting from Strings: Language Model Embeddings for Bayesian Optimization
Tung Nguyen, Qiuyi Zhang, Bangding Yang, Chansoo Lee, Jorg Bornschein, Yingjie Miao, Sagi Perel, Yutian Chen, Xingyou Song
Google DeepMind Technical Report 2024
Paper:
[arXiv]
[Code]
External Media/References:
[MarkTechPost]
-
Position: Leverage Foundational Models for Black-Box Optimization
Xingyou Song, Yingtao Tian, Robert Tjarko Lange, Chansoo Lee, Yujin Tang, Yutian Chen
International Conference on Machine Learning (ICML) 2024
Paper:
[arXiv]
[Poster]
External Media/References:
[TuringPost]
[Eye on AI]
[TransferLab]
-
OmniPred: Language Models as Universal Regressors
Xingyou Song, Oscar Li, Chansoo Lee, Bangding (Jeffrey) Yang, Daiyi Peng, Sagi Perel, Yutian Chen
Paper:
[arXiv]
[Code]
[Poster]
External Media/References:
[HuggingFace Daily Papers]
[arXiv Papers (Video)]
[MarkTechPost]
[Prompt Engineering Institute]
[BNN Breaking]
[MultiPlatform AI]
[Linnk]
[Tunadorable Papers]
[CSDN (Chinese)]
Talks:
[Slides]
[AutoML Seminar]
[ML Collective]
-
Hardness of Low Rank Approximation of Entrywise Transformed Matrix Products
(a) Tamas Sarlos, Xingyou Song, David Woodruff, Qiuyi Zhang
Neural Information Processing Systems (NeurIPS) 2023
[arXiv]
[NeurIPS]
-
Discovering Adaptable Symbolic Algorithms from Scratch
Stephen Kelly, Daniel S. Park, Xingyou Song, Mitchell McIntire, Pranav Nashikkar, Ritam Guha, Wolfgang Banzhaf, Kalyanmoy Deb, Vishnu Naresh Boddeti, Jie Tan, Esteban Real
International Conference on Intelligent Robots and Systems (IROS) 2023 (Best Overall Paper Finalist, Top 12/2760 of Submissions)
Paper:
[arXiv]
[Video]
[Poster]
External Media/References:
[HuggingFace Daily Papers]
-
Towards Learning Universal Hyperparameter Optimizers with Transformers
Yutian Chen, Xingyou Song, Chansoo Lee, Zi Wang, Qiuyi Zhang, David Dohan, Kazuya Kawakami, Greg Kochanski, Arnaud Doucet, Marc'aurelio Ranzato, Sagi Perel, Nando de Freitas
Neural Information Processing Systems (NeurIPS) 2022
Paper:
[arXiv]
[Code]
[Google AI Blog]
[Poster]
External Media/References:
[Tensorflow Transformer Tutorial]
[Vevesta Labs]
[Accenture Federal Viewpoints]
[MarkTechPost]
[The Evolving Enterprise]
[LogicTech World]
[Analytics Vidhya]
[Same OG]
[Tech News Crypt]
[The Perfectech]
[IANoticas (Spanish)]
[MundoXDigital (Spanish)]
[Digizom (Spanish)]
[DataLearner (Chinese)]
[HubAI (Chinese)]
[AI Times (Korean)]
[Web Big Data (Japanese)]
Slides:
[OptFormer Slides]
[LLMs for Optimization Slides]
Talk Videos:
[ML Collective]
[AutoML Seminar]
[AutoML-Conf 2023 Tutorial]
-
Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
(a) Jack Parker-Holder*, Raghu Rajan*, Xingyou Song*, André Biedenkapp, Yingjie Miao, Theresa Eimer, Baohe Zhang, Vu Nguyen, Roberto Calandra, Aleksandra Faust, Frank Hutter, Marius Lindauer
Journal of Artificial Intelligence Research (JAIR) 2022
Automated Machine Learning Conference (AutoML-Conf) Journal Track 2023
-
Differentiable Architecture Search for Reinforcement Learning
Yingjie Miao*, Xingyou Song*, John D. Co-Reyes, Daiyi Peng, Summer Yue, Eugene Brevdo, Aleksandra Faust
Automated Machine Learning Conference (AutoML-Conf) 2022
-
ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search Spaces
Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder, Yunhao Tang, Qiuyi Zhang, Daiyi Peng, Deepali Jain, Wenbo Gao, Aldo Pacchiano, Tamas Sarlos, Yuxiang Yang
International Conference on Learning Representations (ICLR) 2020, Workshop on Neural Architecture Search
-
Unlocking Pixels for Reinforcement Learning via Implicit Attention
Krzysztof Choromanski*, Deepali Jain*, Wenhao Yu*, Xingyou Song, Jack Parker-Holder, Tingnan Zhang, Valerii Likhosherstov, Aldo Pacchiano, Anirban Santara, Yunhao Tang, Jie Tan, Adrian Weller.
-
Sub-Linear Memory: How to Make Performers SLiM
Valerii Likhosherstov, Krzysztof Choromanski, Jared Davis, Xingyou Song, Adrian Weller
Neural Information Processing Systems (NeurIPS) 2021
-
Debiasing First-order Heuristic for Approximate Bi-level Optimization
Valerii Likhosherstov*, Xingyou Song*, Krzysztof Choromanski, Jared Davis, Adrian Weller
Interntional Conference on Machine Learning (ICML) 2021
-
Rethinking Attention with Performers
Krzysztof Choromanski*, Valerii Likhosherstov*, David Dohan*, Xingyou Song*, Andreea Gane*, Tamas Sarlos*, Peter Hawkins*, Jared Davis*, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Colwell, Adrian Weller
International Conference on Learning Representations (ICLR) 2021 (Oral, Top 2% of Submissions)
-
An Ode to an ODE
Krzysztof Choromanski*, Jared Quincy Davis*, Valerii Likhosherstov*, Xingyou Song, Jean-Jacques Slotine, Jacob Varley, Honglak Lee, Adrian Weller, Vikas Sindhwani
Neural Information Processing Systems (NeurIPS) 2020
-
Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning
Xingyou Song*, Yuxiang Yang*, Krzysztof Choromanski, Ken Caluwaerts, Wenbo Gao, Chelsea Finn, Jie Tan
International Conference on Intelligent Robots and Systems (IROS) 2020
-
Robotic Table Tennis with Model-Free Reinforcement Learning
Wenbo Gao*, Laura Graesser*, Krzysztof Choromanski*, Xingyou Song, Nevena Lazic, Pannag Sanketi, Vikas Sindhwani, Navdeep Jaitly
International Conference on Intelligent Robots and Systems (IROS) 2020
-
Stochastic Flows and Geometric Optimization on the Orthogonal Group
Krzysztof Choromanski*, David Cheikhi*, Jared Davis*, Valerii Likhosherstov*, Achille Nazaret*, Achraf Bahamou*, Xingyou Song*, et al.
International Conference on Machine Learning (ICML) 2020
-
Observational Overfitting in Reinforcement Learning
Xingyou Song, Yiding Jiang, Stephen Tu, Yilun Du, Behnam Neyshabur
International Conference on Learning Representations (ICLR) 2020
-
Gradientless Descent: High-Dimensional Zeroth-Order Optimization
(a) Daniel Golovin, John Karro, Greg Kochanski, Chansoo Lee, Xingyou Song, Qiuyi Zhang
International Conference on Learning Representations (ICLR) 2020 (Spotlight, Top 6% of Submissions)
-
ES-MAML: Simple Hessian-Free Meta Learning
Xingyou Song*, Wenbo Gao*, Yuxiang Yang, Krzysztof Choromanski, Aldo Pacchiano, Yunhao Tang
International Conference on Learning Representations (ICLR) 2020
Neural Information Processing Systems (NeurIPS) 2019, Workshop on Meta-Learning (Spotlight)