Xingyou (Richard) Song

I am currently a Senior Research Scientist at Google DeepMind, currently working on Reinforcement Learning for Gemini Reasoning. Previously I worked on a variety of topics in Automated Machine Learning, Transformers, and Robotics.

I have also worked/interned at OpenAI, Citadel Securities, and Microsoft Research Asia.

Email  /  DBLP  /  Google Scholar  /  Linkedin  /  Github  /  Twitter  /  CV

Gemini Contributions

• Advanced version of Gemini with Deep Think officially achieves gold-medal standard at the International Mathematical Olympiad
  Thinking/Reasoning Contributor
  Google DeepMind Blog Post 2025
  [Blog]
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
  Thinking/Reasoning Contributor
  Gemini Technical Report 2025
  [arXiv] [Blog]

Text-to-Text Regression

• Performance Prediction for Large Systems via Text-to-Text Regression
  Yash Akhauri, Bryan Lewandowski, Cheng-Hsi Lin, Adrian N. Reyes, Grant C. Forbes, Arissa Wongpanich, Bangding Yang, Mohamed S. Abdelfattah, Sagi Perel, Xingyou Song
  Google Technical Report 2025
  Paper: [arXiv] [Google Research Blog] [Code]
  External Media/References: [MarkTechPost] [MG (Video)]
• OmniPred: Language Models as Universal Regressors
  Xingyou Song, Oscar Li, Chansoo Lee, Bangding (Jeffrey) Yang, Daiyi Peng, Sagi Perel, Yutian Chen
  Transactions on Machine Learning Research (TMLR) 2024
  Paper: [arXiv] [TMLR] [Code] [Poster]
  External Media/References: [HuggingFace Daily Papers] [arXiv Papers (Video)] [MarkTechPost] [Prompt Engineering Institute] [BNN Breaking] [MultiPlatform AI] [Linnk] [Tunadorable Papers] [CSDN (Chinese)]
  Talks: [Slides] [AutoML Seminar] [ML Collective]
• Decoding-based Regression
  Xingyou Song*, Dara Bahri*
  Transactions on Machine Learning Research (TMLR) 2025
  Paper: [arXiv] [TMLR] [Code]
  External Media/References: [MarkTechPost]
• Understanding LLM Embeddings for Regression
  Eric Tang, Bangding Yang, Xingyou Song
  Transactions on Machine Learning Research (TMLR) 2025
  Paper: [arXiv] [TMLR]
  External Media/References: [MarkTechPost] [BAAI]

Open Source Vizier: Google's Optimizer

• Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Blackbox Optimization
  Xingyou Song, Sagi Perel, Chansoo Lee, Greg Kochanski, Daniel Golovin
  Automated Machine Learning Conference (AutoML-Conf) Systems Track 2022
  Paper: [arXiv] [Google AI Blog] [AutoML-Conf] [PMLR] [Poster]     Code: [Github] [PyPI] [Docs]
  External Media/References: [Google Research 2022 Algorithms] [Deep Learning Tuning Playbook] [MarkTechPost] [Weights & Biases] [The Sequence] [Deep Learning Weekly] [Analytics India Magazine] [Electronic Smith] [gHacks] [ML News by Yannic Kilcher (Video)] [WebBigdata (Japanese)] [RandomAccess (Spanish)]
  Presentations: [Slides] [AutoML-Conf 2022 Video] [AutoML Seminar Video] [AutoML-Conf 2023 Tutorial Video]
• The Vizier Gaussian Process Bandit Algorithm
  Xingyou Song, Qiuyi Zhang, Chansoo Lee, Emily Fertig, Tzu-Kuo Huang, Lior Belenkil, Greg Kochanski, Setareh Ariafar, Srinivas Vasudevan, Sagi Perel, Daniel Golovin
  Google DeepMind Technical Report 2024
  Paper: [arXiv]
  External Media/References: [ChinaZ] [BAAI] [TuringPost] [DAIR.AI] [NLP Newsletter] [AIModels.FYI] [Open Data Science] [Appy Pie]
  Presentations: [Slides] [BAAI Talk]

OptFormer Project: Language Modeling for Optimization

• Towards Learning Universal Hyperparameter Optimizers with Transformers
  Yutian Chen, Xingyou Song, Chansoo Lee, Zi Wang, Qiuyi Zhang, David Dohan, Kazuya Kawakami, Greg Kochanski, Arnaud Doucet, Marc'aurelio Ranzato, Sagi Perel, Nando de Freitas
  Neural Information Processing Systems (NeurIPS) 2022
  Paper: [arXiv] [Code] [Google AI Blog] [Poster]
  External Media/References: [Tensorflow Transformer Tutorial] [Vevesta Labs] [Accenture Federal Viewpoints] [MarkTechPost] [The Evolving Enterprise] [LogicTech World] [Analytics Vidhya] [Same OG] [Tech News Crypt] [The Perfectech] [IANoticas (Spanish)] [MundoXDigital (Spanish)] [Digizom (Spanish)] [DataLearner (Chinese)] [HubAI (Chinese)] [AI Times (Korean)] [Web Big Data (Japanese)]
  Slides: [OptFormer Slides] [LLMs for Optimization Slides]
  Talk Videos: [ML Collective] [AutoML Seminar] [AutoML-Conf 2023 Tutorial]
• Predicting from Strings: Language Model Embeddings for Bayesian Optimization
  Tung Nguyen, Qiuyi Zhang, Bangding Yang, Chansoo Lee, Jorg Bornschein, Yingjie Miao, Sagi Perel, Yutian Chen, Xingyou Song
  Google DeepMind Technical Report 2024
  Paper: [arXiv] [Code]
  External Media/References: [MarkTechPost]
• Position: Leverage Foundational Models for Black-Box Optimization
  Xingyou Song, Yingtao Tian, Robert Tjarko Lange, Chansoo Lee, Yujin Tang, Yutian Chen
  International Conference on Machine Learning (ICML) 2024
  Paper: [arXiv] [Poster]
  External Media/References: [TuringPost] [Eye on AI] [TransferLab]

Reinforcement Learning and Robotics

• Discovering Adaptable Symbolic Algorithms from Scratch
  Stephen Kelly, Daniel S. Park, Xingyou Song, Mitchell McIntire, Pranav Nashikkar, Ritam Guha, Wolfgang Banzhaf, Kalyanmoy Deb, Vishnu Naresh Boddeti, Jie Tan, Esteban Real
  International Conference on Intelligent Robots and Systems (IROS) 2023 (Best Overall Paper Finalist, Top 12/2760 of Submissions)
  Paper: [arXiv] [Video] [Poster]
  External Media/References: [HuggingFace Daily Papers]
• Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning
  Xingyou Song*, Yuxiang Yang*, Krzysztof Choromanski, Ken Caluwaerts, Wenbo Gao, Chelsea Finn, Jie Tan
  International Conference on Intelligent Robots and Systems (IROS) 2020
  Paper: [arXiv] [IROS]     Code: [ES-MAML Code]
  Blogposts: [Google AI Blog] [Google Research 2020] [PyBullet Blog]
  Videos: [Experiment Video] [IROS Presentation Video] [AI Frontier Talk Video]
  Slides: [IROS 2020 Slides] [ML Collective Slides] [AI Frontier Slides]
• ES-MAML: Simple Hessian-Free Meta Learning
  Xingyou Song*, Wenbo Gao*, Yuxiang Yang, Krzysztof Choromanski, Aldo Pacchiano, Yunhao Tang
  International Conference on Learning Representations (ICLR) 2020
  Neural Information Processing Systems (NeurIPS) 2019, Workshop on Meta-Learning (Spotlight)
  Paper: [arXiv] [ICLR] [Slides] [Poster] [Code]
  External Media: [AITechBaseCamp (Chinese)]
• Observational Overfitting in Reinforcement Learning
  Xingyou Song, Yiding Jiang, Stephen Tu, Yilun Du, Behnam Neyshabur
  International Conference on Learning Representations (ICLR) 2020
  [arXiv] [ICLR] [Slides] [Slides (Old)] [Poster] [LQR Code]
• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
  (a) Jack Parker-Holder*, Raghu Rajan*, Xingyou Song*, André Biedenkapp, Yingjie Miao, Theresa Eimer, Baohe Zhang, Vu Nguyen, Roberto Calandra, Aleksandra Faust, Frank Hutter, Marius Lindauer
  Journal of Artificial Intelligence Research (JAIR) 2022
  Automated Machine Learning Conference (AutoML-Conf) Journal Track 2023
  [arXiv] [JAIR] [Tutorial Video] [Tutorial Slides]
• Differentiable Architecture Search for Reinforcement Learning
  Yingjie Miao*, Xingyou Song*, John D. Co-Reyes, Daiyi Peng, Summer Yue, Eugene Brevdo, Aleksandra Faust
  Automated Machine Learning Conference (AutoML-Conf) 2022
  [arXiv] [AutoML-Conf] [PMLR] [Slides] [Video] [Code] [Poster]
• ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search Spaces
  Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder, Yunhao Tang, Qiuyi Zhang, Daiyi Peng, Deepali Jain, Wenbo Gao, Aldo Pacchiano, Tamas Sarlos, Yuxiang Yang
  International Conference on Learning Representations (ICLR) 2020, Workshop on Neural Architecture Search
  [arXiv] [Code] [Poster (Old)]
• Robotic Table Tennis with Model-Free Reinforcement Learning
  Wenbo Gao*, Laura Graesser*, Krzysztof Choromanski*, Xingyou Song, Nevena Lazic, Pannag Sanketi, Vikas Sindhwani, Navdeep Jaitly
  International Conference on Intelligent Robots and Systems (IROS) 2020
  [arXiv] [Video]

Efficient Transformers and Attention

• Rethinking Attention with Performers
  Krzysztof Choromanski*, Valerii Likhosherstov*, David Dohan*, Xingyou Song*, Andreea Gane*, Tamas Sarlos*, Peter Hawkins*, Jared Davis*, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Colwell, Adrian Weller
  International Conference on Learning Representations (ICLR) 2021 (Oral, Top 2% of Submissions)
  Paper: [arXiv] [ICLR] [Google AI Blog] [Slides]     Code: [Protein LM Code] [Performer Code]
  External Media/Blogposts: [Forbes] [Google Research 2022] [Hopfield Networks (ICLR Blog Track)] [Schmidhuber AI Blog] [HuggingFace] [Kerasmitas] [IARAI] [Hopfield Networks] [InfoQ] [TowardsDataScience] [AIM] [Eyerys] [TeddyKoker] [ExBulletin] [MarkTechPost] [Synced (v2)] [Synced (v1)] [VentureBeat] [LinkResearcher (Chinese)] [Zhuanlan (Chinese)] [KeXue (Chinese)] [MDLI (Hebrew)] [MIT Press (Sec 15.6)] [Google Research 2020]
  Forums: [Hacker News (YCombinator)] [Paper Explained (Reddit)] [FAVOR Explained (Reddit)]
  Talk Videos: [LightOnAI] [IARAI Fireside Chat]
  External Videos: [Paper Explained] [High Performance NLP] [BiliBili (Chinese)]
• Hardness of Low Rank Approximation of Entrywise Transformed Matrix Products
  (a) Tamas Sarlos, Xingyou Song, David Woodruff, Qiuyi Zhang
  Neural Information Processing Systems (NeurIPS) 2023
  [arXiv] [NeurIPS]
• Unlocking Pixels for Reinforcement Learning via Implicit Attention
  Krzysztof Choromanski*, Deepali Jain*, Wenhao Yu*, Xingyou Song, Jack Parker-Holder, Tingnan Zhang, Valerii Likhosherstov, Aldo Pacchiano, Anirban Santara, Yunhao Tang, Jie Tan, Adrian Weller.
  [arXiv] [Site]
• Sub-Linear Memory: How to Make Performers SLiM
  Valerii Likhosherstov, Krzysztof Choromanski, Jared Davis, Xingyou Song, Adrian Weller
  Neural Information Processing Systems (NeurIPS) 2021
  [arXiv] [Code]

Miscellaneous

• Gradientless Descent: High-Dimensional Zeroth-Order Optimization
  (a) Daniel Golovin, John Karro, Greg Kochanski, Chansoo Lee, Xingyou Song, Qiuyi Zhang
  International Conference on Learning Representations (ICLR) 2020 (Spotlight, Top 6% of Submissions)
  [arXiv] [ICLR] [Slides]
• Debiasing First-order Heuristic for Approximate Bi-level Optimization
  Valerii Likhosherstov*, Xingyou Song*, Krzysztof Choromanski, Jared Davis, Adrian Weller
  Interntional Conference on Machine Learning (ICML) 2021
  [arXiv] [ICML] [Code]
• An Ode to an ODE
  Krzysztof Choromanski*, Jared Quincy Davis*, Valerii Likhosherstov*, Xingyou Song, Jean-Jacques Slotine, Jacob Varley, Honglak Lee, Adrian Weller, Vikas Sindhwani
  Neural Information Processing Systems (NeurIPS) 2020
  [arXiv] [NeurIPS]
• Stochastic Flows and Geometric Optimization on the Orthogonal Group
  Krzysztof Choromanski*, David Cheikhi*, Jared Davis*, Valerii Likhosherstov*, Achille Nazaret*, Achraf Bahamou*, Xingyou Song*, et al.
  International Conference on Machine Learning (ICML) 2020
  [arXiv] [ICML]