For her work on analyzing human-robot interactions, Marynel Vázquez has won a 2022 Faculty Early Career Development (CAREER) Award from the National Science Foundation (NSF).
Vázquez, assistant professor of computer science, will use the $600,000, five-year grant to develop methods for studying and advancing interactions between robots and humans in group settings. The NSF CAREER award is a prestigious honor for young faculty members and supports the early career activities of teachers and scholars who are most likely to become the academic leaders of the future.
The long-term goal is to enhance human-robot interaction so that robots can better take part in social encounters with multiple users. This has become increasingly important as robots are now being used to provide information (in public kiosks, for example, or museums), and working with multiple people at once, such as in elderly care centers or grocery stores. In such cases, deftly handling group dynamics is critical to the robots’ performance.
Much of the research in human-robot interaction has focused on one-on-one interaction, Vázquez said, but real-world applications often involve multi-party settings. “So the main goal of this project is to build foundational technologies and approaches to start advancing robot autonomy for this type of group interaction.”
Essential to the project is that these interactions happen outside of the lab.
“In the lab, people normally feel observed, and they are more cautious about what they do because they know everybody’s there to see what they do,” Vázquez said.
To do that, Vázquez’s research team will set up Shutter, a robot designed in her lab, on various parts of campus. Shutter will interact with passersb. For example, it will offer to take a photo as a memory of their time on campus. When passersby choose to interact with Shutter, and how they do so, are central to the project.
“Some of them will be in a rush, some of them will have lots of time to spend with the robot,” she said. “Some of them may come in groups, some of them may come individually.”
Vázquez is taking a three-pronged approach to the project. One is modeling the human-robot interactions with graphs, as is commonly done for social network analysis like Twitter, but rarely implemented in robotics. The second is using machine learning methods that are suited to handle this type of data, like graph neural networks. The third part has to do with how labels are obtained for the data - in this case, through what’s known as “self-supervision” in robotics. The idea is that instead of having researchers observe human-robot interactions and manually label relevant events in them, robots can do this labeling themselves. For example, this will help the Shutter robot predict when people will initiate encounters with it before they actually do. This ability will then be used to adapt its behavior to engage nearby pedestrians.
To create more robust systems, Vázquez also wants to develop ways to identify when interactions between the robot and humans break down. This could involve, for example, someone failing to follow the correct procedure in answering the robot’s questions or getting into a second conversation with another passerby. The robot could then prompt the participant about the cause of the breakdown and make note of it so that it can better react to these types of situations in the future.
Additionally, this project includes activities designed to advance diversity and inclusion in computer science, as well as increase STEM engagement through artificial intelligence (AI) technologies. For example, project activities include a partnership with the Yale Peabody Museum to help educate high-school and college students from primarily low-income communities, engage them in research, and engage the general public with AI and robotics.
In addition to Vázquez, many of her students have contributed to the design and operation of Shutter, including: Alex (Sasha) Lew, Sydney Thompson, Nathan Tsoi, Tim Adamson, Burton Lyng-Olsen, Sarah Gillet, Christine Yang, Eden Gorevoy, and Victor Del Carpio.