CS Colloquium
Xin (Eric) Wang, UC Santa Cruz
Host: Alex Wong
Title: Foundations of Multimodal Embodied Agents for Human-Agent Collaboration
Abstract:
A long-term goal of AI research is to build intelligent agents that can effectively communicate with humans, perceive their multimodal environment, and execute a diverse range of real-world tasks, from everyday household chores to complex, mission-critical tasks such as battlefield reconnaissance. These embodied agents are envisioned to operate both autonomously and collaboratively via human interaction.
This talk focuses on addressing fundamental challenges in multimodal embodied agents for human-agent collaboration. First, we begin by showing our pioneering work in building generalizable embodied agents, ones that can adeptly navigate to novel objects in new environments without navigation training at all. Then, we underscore the significance of streamlined human-agent communication in fostering efficient collaboration, with our proposed new benchmark for aerial vision-and-dialog navigation. Furthermore, we present new methods modeling the two critical roles in human-agent collaboration, commander and follower, and finally conclude with a discussion of future research plans.
Bio:
Xin (Eric) Wang is an Assistant Professor of Computer Science and Engineering at UC Santa Cruz. His research interests include Natural Language Processing, Computer Vision, and Machine Learning, with a focus on Multimodal and Embodied AI. Before joining UCSC, he obtained his Ph.D. degree from UC Santa Barbara in 2020 and Bachelor’s degree from Zhejiang University in 2015. He worked at Google Research, Facebook AI Research, Microsoft Research, and Adobe Research.
Xin has served as Area Chair for conferences such as ACL, NAACL, EMNLP, ICLR, and NeurIPS, as well as Senior Program Committee for AAAI and IJCAI. He has organized numerous workshops and tutorials at conferences such as ACL, NAACL, CVPR, and ICCV. He has received several awards and recognitions for his work, including CVPR Best Student Paper Award, Google Research Faculty Award, and Amazon Alexa Prize Awards.