Hello and Welcome!

I received my Ph.D. in Computer Science at Brown University in 2025. I was fortunate to be advised by Prof. Stefanie Tellex and work with Prof. George Konidaris. I received my Bachelor of Science degree from Berkeley EECS in 2017. I am grateful for the supports of the NSF Graduate Research Fellowship Program, and the Jack Kent Cooke Foundation Graduate Scholarship.

I am interested in developing autonomous robots that assist people. To that end, I am building robot systems that leverage multimodal perception and interaction with human users to solve long-horizon mobile manipulation tasks in diverse human environments. I am currently investigating robotic language grounding using foundation models and the integration with planning, learning, and formal methods.

Publications

Lang2LTL-2: Grounding Spatiotemporal Navigation Commands Using Large Language and Vision-Language Models. [paper, website]
Jason Xinyu Liu, Ankit Shah, George Konidaris, Stefanie Tellex, David Paulius.
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddings. [paper, website, slides, poster]
Vanya Cohen^*, Jason Xinyu Liu^*, Raymond Mooney^*, Stefanie Tellex^*, David Watkins^*.
2024 International Joint Conference on Artificial Intelligence (IJCAI) Survey Track.

LTL-Transfer: Skill Transfer for Temporally-Extended Task Specifications. [paper, website, code]
Jason Xinyu Liu^*, Ankit Shah^*, Eric Rosen, Mingxi Jia, George Konidaris, Stefanie Tellex.
2024 IEEE International Conference on Robotics and Automation (ICRA).

Lang2LTL: Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments. [paper, website, code]
Jason Xinyu Liu^*, Ziyi Yang^*, Ifrah Idrees, Sam Liang, Benjamin Schornstein, Stefanie Tellex, Ankit Shah.
2023 Conference on Robot Learning (CoRL).

Style-transfer based Speech and Audio-visual Scene Understanding for Robot Action Sequence Acquisition from Videos. [paper]
Chiori Hori, Puyuan Peng, David Harwath, Jason Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh Jha, Diego Romeres, Jonathan Le Roux.
INTERSPEECH 2023.

Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments. [paper, website, code]
Jason Xinyu Liu, Ziyi Yang, Ifrah Idrees, Sam Liang, Benjamin Schornstein, Stefanie Tellex, Ankit Shah.
2023 AAAI Fall Symposium on Unifying Representations for Robot Application Development (UR-RAD). Best Paper Nominee.

Generalizing to New Domains by Mapping Natural Language to Lifted LTL. [paper]
Eric Hsiung, Hiloni Mehta, Junchi Chu, Jason Xinyu Liu, Roma Patel, Stefanie Tellex, George Konidaris.
2022 IEEE International Conference on Robotics and Automation (ICRA).

Leveraging Temporal Structure in Safety-Critical Task Specifications for POMDP Planning. [paper, poster]
Jason Xinyu Liu, Eric Rosen, Suchen Zheng, George Konidaris, Stefanie Tellex.
2021 Robotics: Science and Systems (RSS) Workshop Robotics for People: Perspectives on Interaction, Learning and Safety.

Dialogue Object Search. [paper]
Monica Roy, Kaiyu Zheng, Jason Xinyu Liu, Stefanie Tellex.
2021 Robotics: Science and Systems (RSS) Workshop Robotics for People: Perspectives on Interaction, Learning and Safety.

Specificity-Controlled Video Captioning. [paper]
Jason Xinyu Liu, Ellie Pavlick, Daniel Ritchie, George Konidaris, Stefanie Tellex.
Technical Report 2019.

Detecting Phone Theft Using Machine Learning. [paper]
Jason Xinyu Liu, David Wagner, Serge Egelman.
2018 International Conference on Information Science and System (ICISS).

Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds Using a New Analytic Model and Deep Learning. [paper, website]
Jeffrey Mahler, Matthew Matl, Jason Xinyu Liu, Albert Li, David Gealy, Ken Goldberg.
2018 IEEE International Conference on Robotics and Automation (ICRA).

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics. [paper, website]
Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Jason Xinyu Liu, Juan Aparicio Ojea, Ken Goldberg.
2017 Robotics: Science and Systems (RSS).

Talks

Robotic Language Grounding.
USC Robotic Seminar. January 30, 2025.

Robotic Language Grounding.
Brown University Database Group Meeting. December 6, 2024.

Robotic Language Grounding. [website, video]
UT Austin Forum for Artificial Intelligence. October 11, 2024.

Towards Robots that Use Natural Language: Robotic Language Grounding. [website, video]
CU Boulder CS Colloquium. October 11, 2024.

Towards Robots that Use Natural Language: Language Grounding for Robots.
MIT Interactive Robotics Group. September 25, 2024.

Towards Robots that Use Natural Language: Language Grounding for Robots.
UT Austin, Center of Autonomy. September 13, 2024.

Towards Robots that Use Natural Language: Language Grounding for Robots.
ASU, Logos Robotics Lab. August 12, 2024.

Towards Robots that Use Natural Language: Language Grounding for Robots.
Oxford, CS Seminar. July 12, 2024.

AI, Robots, and Human Language. [website, video]
Brown Research Matters. April 18, 2024.

Leverage Temporal Structure in Task Specification for Planning and Learning.
Northeast Robotics Colloquium. October 8, 2022.

Fun Outside Work

I love reading books, playing basketball, hiking, camping, and movies directed by Christopher Nolan. When I read, I listen to guitar or Jazz BGM. The books I recently read are The Road Less Traveled, The Autobiography of Benjamin Franklin, and How We Think by John Dewey.

Contact

Lmk if you'd like to chat. Always up for having fun conversations and making new friends!