Language-Driven Learning for Interactive Robotics
Siddharth Karamcheti (Stanford University)
Colloquium
Thursday, February 20, 2025, 3:30 pm
Gates Center (CSE2), G20 | Amazon Auditorium
Abstract
Building and deploying broadly capable robots requires systems that can efficiently learn from and work with people. To achieve this, robots must balance capability — the fundamental tools necessary to enable real-world deployment — and sustainability — the ability to grow and adapt through human feedback. In this talk, I will motivate language-driven learning to tackle these axes, providing robots with better abstractions for perception, action, and human-robot interaction. Towards capability, I will present Voltron, our approach for using language to learn visual representations that can be efficiently adapted for diverse robotics tasks. Building on these ideas, I will discuss Prismatic, our experimental framework for developing visually-conditioned language models and vision-language-action policies at scale. Towards sustainability, I will introduce Vocal Sandbox, a new framework that integrates these models to develop collaborative robots that can work alongside human partners, using language to express uncertainty and learn new behaviors from real-time interactions. Finally, I will conclude with open challenges for enhancing both the capability and sustainability of modern robots, with directions for future work.
Bio
Siddharth Karamcheti is a final year PhD student at Stanford University advised by Dorsa Sadigh and Percy Liang, and a robotics intern at the Toyota Research Institute. His research focuses on robot learning, natural language processing, and human-robot interaction, with a goal of developing scalable approaches for human-robot collaboration. Prior to the PhD, he earned his bachelor's degree in Computer Science and Literary Arts at Brown University, where he worked with Eugene Charniak and Stefanie Tellex. He is a recipient of the Open Philanthropy AI Fellowship (2018), is an RSS Pioneer (2024), and his research has won paper awards at conferences such as RSS, CoRL, ICRA, and ACL.
This talk will be streamed live on our YouTube channel. Link will be available on that page one hour prior to the event.