|
Jason Xinyu Liu I am a postdoc at MIT, working with Prof. Julie Shah and Prof. Andreea Bobu. I received my Ph.D. in Computer Science at Brown University in 2025, advised by Prof. Stefanie Tellex and worked with Prof. George Konidaris. I received my Bachelor of Science degree from Berkeley EECS in 2017. I am grateful for the support of the NSF Graduate Research Fellowship Program and the Jack Kent Cooke Foundation Graduate Scholarship. |
|
ResearchI am interested in developing algorithms and deployable systems that enable autonomous robots to reason under uncertainty and solve long-horizon tasks for and with people. My research aims to make robots more accessible to non-expert users through multimodal interaction and natural language dialogue, enabling safe and effective interaction. My PhD work focuses on robotic language grounding using foundation models and its integration with planning, learning, and formal methods. |
|
Language as a Sensor: Calibrated Spatial Belief Estimation in 3D Scenes from Natural Language
Aryan Naveen, Jason Xinyu Liu, Luca Carlone, Andreea Bobu Conference on Robot Learning (CoRL), 2026 paper |
|
GHOST in the Robots: Real-Time Exocentric Dual-Robot VR Teleoperation from Onboard Cameras
Yichen Wei, Faisal Zaghloul, Soujanya C Aryal, Aanya K. Agrawal, Chengfan Li, Jason Xinyu Liu, James Tompkin, Stefanie Tellex IEEE Robotics and Automation Letters (RA-L), 2026 paper |
|
LEGS-POMDP: Language and Gesture-Guided Object Search in Partially Observable Environments
Ivy He, Stefanie Tellex, Jason Xinyu Liu ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2026 paper / project page / news |
|
Robotic Language Grounding
Jason Xinyu Liu PhD Thesis, Brown University, 2025 paper / ProQuest / video |
|
Open-vocabulary Pick and Place via Patch-level Semantic Maps
Mingxi Jia, Haojie Huang, Zhewen Zhang, Chenghao Wang, Linfeng Zhao, Dian Wang, Jason Xinyu Liu, Robin Walters, Robert Platt, Stefanie Tellex IEEE Robotics and Automation Letters (RA-L), 2025 Best Paper Award Honorable Mention paper / project page |
|
Lang2LTL-2: Grounding Spatiotemporal Navigation Commands Using Large Language and Vision-Language Models
Jason Xinyu Liu, Ankit Shah, George Konidaris, Stefanie Tellex, David Paulius IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024 paper / project page |
|
A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddings
Vanya Cohen*, Jason Xinyu Liu*, Raymond Mooney*, Stefanie Tellex*, David Watkins* International Joint Conference on Artificial Intelligence (IJCAI) Survey Track, 2024 paper / project page / slides / poster |
|
LTL-Transfer: Skill Transfer for Temporally-Extended Task Specifications
Jason Xinyu Liu*, Ankit Shah*, Eric Rosen, Mingxi Jia, George Konidaris, Stefanie Tellex IEEE International Conference on Robotics and Automation (ICRA), 2024 paper / project page / code |
|
Lang2LTL: Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments
Jason Xinyu Liu*, Ziyi Yang*, Ifrah Idrees, Sam Liang, Benjamin Schornstein, Stefanie Tellex, Ankit Shah Conference on Robot Learning (CoRL), 2023 paper / project page / code |
|
Style-transfer based Speech and Audio-visual Scene Understanding for Robot Action Sequence Acquisition from Videos
Chiori Hori, Puyuan Peng, David Harwath, Jason Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh Jha, Diego Romeres, Jonathan Le Roux INTERSPEECH, 2023 paper |
|
Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments
Jason Xinyu Liu, Ziyi Yang, Ifrah Idrees, Sam Liang, Benjamin Schornstein, Stefanie Tellex, Ankit Shah AAAI Fall Symposium on Unifying Representations for Robot Application Development (UR-RAD), 2023 Best Paper Finalist paper / project page / code |
|
Generalizing to New Domains by Mapping Natural Language to Lifted LTL
Eric Hsiung, Hiloni Mehta, Junchi Chu, Jason Xinyu Liu, Roma Patel, Stefanie Tellex, George Konidaris IEEE International Conference on Robotics and Automation (ICRA), 2022 paper |
|
Leveraging Temporal Structure in Safety-Critical Task Specifications for POMDP Planning
Jason Xinyu Liu, Eric Rosen, Suchen Zheng, George Konidaris, Stefanie Tellex RSS Workshop on Robotics for People: Perspectives on Interaction, Learning and Safety, 2021 paper / poster |
|
Dialogue Object Search
Monica Roy, Kaiyu Zheng, Jason Xinyu Liu, Stefanie Tellex RSS Workshop on Robotics for People: Perspectives on Interaction, Learning and Safety, 2021 paper |
|
Specificity-Controlled Video Captioning
Jason Xinyu Liu, Ellie Pavlick, Daniel Ritchie, George Konidaris, Stefanie Tellex Technical Report, 2019 paper |
|
Detecting Phone Theft Using Machine Learning
Jason Xinyu Liu, David Wagner, Serge Egelman International Conference on Information Science and System (ICISS), 2018 paper |
|
Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds Using a New Analytic Model and Deep Learning
Jeffrey Mahler, Matthew Matl, Jason Xinyu Liu, Albert Li, David Gealy, Ken Goldberg IEEE International Conference on Robotics and Automation (ICRA), 2018 paper / project page |
|
Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Jason Xinyu Liu, Juan Aparicio Ojea, Ken Goldberg Robotics: Science and Systems (RSS), 2017 paper / project page |
Talks
CoRL 2026 Workshop on Spatial Intelligence and Structured Reasoning for Embodied Navigation, Oct 2026
|
Fun Outside WorkI love reading books, playing basketball, hiking, camping, and movies directed by Christopher Nolan. When I read, I listen to guitar or Jazz BGM. The books I recently read are The Road Less Traveled, The Autobiography of Benjamin Franklin, and How We Think by John Dewey. |
ContactLmk if you'd like to chat at jxyliu@mit.edu. Always up for having fun conversations and making new friends! |
|
Website template from Jon Barron. |