Shiyu Jin

Senior Researcher @ Baidu Research, Ph.D. UC Berkeley

Email: jsy [at] berkeley [dot] edu

I’m a robotic researcher at Baidu Research in Sunnyvale, California. My research interests focus on robotic manipulation and robot learning.

I obtained my Ph.D. from UC Berkeley in Dec 2021, supervised by Prof. Masayoshi Tomizuka. Before joining Baidu, I had the opportunities to work at Google X/Intrinsic, MERL, FANUC.

EDUCATION

Ph.D. in Mechanical Engineering (2021.12)

University of California, Berkeley

M.S. in Mechanical Engineering (2018.5)

University of California, Berkeley

B.Sc. in Vehicle Engineering (2016.6)

Beijing Institute of Technology, Beijing, China

Research

Image description

Reasoning Grasping via Multimodal Large Language Model

Shiyu Jin, Jinxuan Xu, Yutian Lei, Liangjun Zhang

Under review

[Paper]

Image description

Reasoning Tuning Grasp: Adapting Multimodal Large Language Models for Robotic Grasping

Jinxuan Xu, Shiyu Jin, Yutian Lei, Yuqian Zhang, Liangjun Zhang

Workshop in CoRL 2023

Under review

[Paper] [Website]

Image description

Transformer-Based 3D Object Manipulation using Virtual In-Hand Views

Weiyao Wang, Yutian Lei, Shiyu Jin, Gregory D. Hager, Liangjun Zhang

Workshop in CoRL 2023

Under review

[Website]

Image description

Robotic Excavation of Rigid Objects using Offline Reinforcement Learning

Shiyu Jin, Zhixian Ye, Liangjun Zhang

Under review

[Paper]

Image description

RLingua: Improving Reinforcement Learning Sample Efficiency in Robotic Manipulations with Large Language Models

Liangliang Chen, Yutian Lei, Shiyu Jin, Ying Zhang, Liangjun Zhang

IEEE Robotics and Automation Letters (RA-L) 2024 (Accepted)

[Website]

Image description

Autonomous Excavator System for Construction Earth Moving

Shiyu Jin, Liuwang Kang, Yuan-Chih Peng, Zhenpeng He, Ruitao Song, Lingfeng Qian, Liangjun Zhang

Workshop in ICRA 2023, workshop in RSS 2023

[Paper] [Video]

Image description

Autonomous Wheel Loader Simulation

Shiyu Jin

Image description

Goats: Goal sampling adaptation for scooping with curriculum reinforcement learning

Yaru Niu, Shiyu Jin, Zeqing Zhang, Jiacheng Zhu, Ding Zhao, Liangjun Zhang

IROS 2023

[Paper] [Website]

Image description

Robotic Cable Routing with Spatial Representation

Shiyu Jin, Wenzhao Lian, Changhao Wang, Masayoshi Tomizuka, Stefan Schaal

IEEE Robotics and Automation Letters (RA-L) 2022

[Paper] [Code]

Image description

Offline-Online Learning of Deformation Model for Cable Manipulation with Graph Neural Networks

Changhao Wang, Yuyou Zhang, Xiang Zhang, Zheng Wu, Xinghao Zhu, Shiyu Jin, Te Tang, Masayoshi Tomizuka

IEEE Robotics and Automation Letters (RA-L) 2022

[Paper] [Website] [Code]

Image description

Learning Insertion Primitives with Hybrid Action Space for Robotic Assembly Tasks

Xiang Zhang, Shiyu Jin, Changhao Wang, Xinghao Zhu, Masayoshi Tomizuka

ICRA 2022

[Paper] [Website]

Image description

BPOMP: A Bilevel Path Optimization Formulation for Motion Planning

Changhao Wang, Hsien-Chung Lin, Shiyu Jin, Xinghao Zhu, Liting Sun, Masayoshi Tomizuka

American Control Conference (ACC) 2022

[Paper] [Website]

Image description

Trajectory Optimization for Manipulation of Deformable Objects: Assembly of Belt Drive Units

Shiyu Jin, Diego Romeres, Arvind Ragunathan, Devesh K. Jha， Masayoshi Tomizuka

ICRA 2021

[Paper]

Image description

Contact pose identification for peg-in-hole assembly under uncertainties

Shiyu Jin, Xinghao Zhu, Changhao Wang, Masayoshi Tomizuka

American Control Conference (ACC) 2021

[Paper]

Image description

Robust deformation model approximation for robotic cable manipulation

Shiyu Jin*, Changhao Wang*, Masayoshi Tomizuka

IROS 2019

[Paper] [Website]