Hello and Welcome!

I am a computer science Ph.D. candidate at Brown University. I am fortunate to be advised by Prof. Stefanie Tellex and work with Prof. George Konidaris. I received my bachelor of science degree from Berkeley EECS. I am grateful for the supports of the NSF Graduate Research Fellowship Program and Jack Kent Cooke Foundation Graduate Scholarship.

I am interested in developing autonomous robots that assist people. To that end, I am building robotic systems that leverage multimodal perception and interaction with human users to solve long-horizon mobile manipulation tasks in diverse environments. The approaches I am currently investigating are planning, learning, formal methods, and large pretrained models.

Publications

Lang2LTL-2: Grounding Spatiotemporal Navigation Commands Using Large Language and Vision-Language Models. [paper, website]
Jason Xinyu Liu, Ankit Shah, George Konidaris, Stefanie Tellex, David Paulius.
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddingss. [paper, slides, poster]
Vanya Cohen^*, Jason Xinyu Liu^*, Raymond Mooney^*, Stefanie Tellex^*, David Watkins^*.
2024 International Joint Conference on Artificial Intelligence (IJCAI) Survey Track.

LTL-Transfer: Skill Transfer for Temporally-Extended Task Specifications. [paper, website, code]
Jason Xinyu Liu^*, Ankit Shah^*, Eric Rosen, Mingxi Jia, George Konidaris, Stefanie Tellex.
2024 IEEE International Conference on Robotics and Automation (ICRA).

Lang2LTL: Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments. [paper, website, code]
Jason Xinyu Liu^*, Ziyi Yang^*, Ifrah Idrees, Sam Liang, Benjamin Schornstein, Stefanie Tellex, Ankit Shah.
2023 Conference on Robot Learning (CoRL).

Style-transfer based Speech and Audio-visual Scene Understanding for Robot Action Sequence Acquisition from Videos. [paper]
Chiori Hori, Puyuan Peng, David Harwath, Jason Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh Jha, Diego Romeres, Jonathan Le Roux.
INTERSPEECH 2023.

Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments. [paper, website, code]
Jason Xinyu Liu, Ziyi Yang, Ifrah Idrees, Sam Liang, Benjamin Schornstein, Stefanie Tellex, Ankit Shah.
2023 AAAI Fall Symposium on Unifying Representations for Robot Application Development (UR-RAD). Best Paper Nominee.

Generalizing to New Domains by Mapping Natural Language to Lifted LTL. [paper]
Eric Hsiung, Hiloni Mehta, Junchi Chu, Jason Xinyu Liu, Roma Patel, Stefanie Tellex, George Konidaris.
2022 IEEE International Conference on Robotics and Automation (ICRA).

Leveraging Temporal Structure in Safety-Critical Task Specifications for POMDP Planning. [paper, poster]
Jason Xinyu Liu, Eric Rosen, Suchen Zheng, George Konidaris, Stefanie Tellex.
2021 Robotics: Science and Systems (RSS) Workshop Robotics for People: Perspectives on Interaction, Learning and Safety.

Dialogue Object Search. [paper]
Monica Roy, Kaiyu Zheng, Jason Xinyu Liu, Stefanie Tellex.
2021 Robotics: Science and Systems (RSS) Workshop Robotics for People: Perspectives on Interaction, Learning and Safety.

Specificity-Controlled Video Captioning. [paper]
Jason Xinyu Liu, Ellie Pavlick, Daniel Ritchie, George Konidaris, Stefanie Tellex.
Technical Report 2019.

Detecting Phone Theft Using Machine Learning. [paper]
Jason Xinyu Liu, David Wagner, Serge Egelman.
2018 International Conference on Information Science and System (ICISS).

Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds Using a New Analytic Model and Deep Learning. [paper, website]
Jeffrey Mahler, Matthew Matl, Jason Xinyu Liu, Albert Li, David Gealy, Ken Goldberg.
2018 IEEE International Conference on Robotics and Automation (ICRA).

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics. [paper, website]
Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Jason Xinyu Liu, Juan Aparicio Ojea, Ken Goldberg.
2017 Robotics: Science and Systems (RSS).

Talks

AI, Robots, and Human Language. [website, video]
Research Matters 2024 at Brown University Graduate School.

Fun Outside Work

I love reading books, playing basketball, hiking, camping and movies directed by Christopher Nolan. When I read, I listen to guitar or Jazz BGM. The books I recently read are The Road Less Traveled, The Autobiography of Benjamin Franklin, and How We Think by John Dewey.

Contact

Lmk if you'd like to chat. Always up to have fun conversations and make new friends!