Ju Dong董驹

PhD Candidate, Robot Learning & Manipulation

TAMS, Department of Informatics, University of Hamburg

Portrait of Ju Dong

About

I am a PhD candidate at TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, University of Hamburg, supervised by Prof. Jianwei Zhang. I also work with the AI department at Agile Robots SE in Munich.

My research focuses on robot learning and robotic manipulation. I study diffusion and flow-matching policies, real-time policy distillation, mobile manipulation and whole-body control, and vision-language-action models for bimanual manipulation. My recent work explores force-conditioned action corrections and action-conditioned latent foresight for humanoid loco-manipulation, alongside benchmarks for safe and responsible robot behavior.

I combine learned policies with feedback and control to help robots respond to changing scenes and physical contact. I also study how robot planners recognize hazards, choose mitigation actions, and complete tasks safely.

Students interested in a bachelor's or master's thesis or a research project in these areas are welcome to contact me.

News

Aug 2026

Helping run the EBiM Benchmark, a globally coordinated benchmark for real-world embodied bimanual manipulation, with cross-continent real-robot validation in Hamburg, Munich, and Shanghai.

Mar 2026

Our paper “From Flow to One Step: Real-Time Multi-Modal Trajectory Policies via Implicit Maximum Likelihood Estimation-based Distribution Distillation” is accepted by IROS 2026.

Mar 2026

Completed my M.Sc. in Mechatronics and Robotics at the Technical University of Munich.

Jan 2026

Our paper “M4Diffuser: Multi-View Diffusion Policy with Manipulability-Aware Control for Robust Mobile Manipulation” is accepted by ICRA 2026.

Dec 2025

Released “ResponsibleRobotBench”, a benchmark for evaluating responsible robot manipulation with multi-modal large language models.

Mar 2025

Joined the Artificial Intelligence department at Agile Robots SE in Munich.

Selected Publications

Humanoid basket-placement comparison and action-conditioned latent foresight: command sequences are evaluated for visual progress and temporal consistency before a short prefix is executed.

Coordinating Humanoid Loco-Manipulation through Action-Conditioned Latent Foresight

Ju Dong†, Xinyu Xiao, Yiming Zhou†, Tianshuo Yang, Xuerong Liu, Lei Zhang, Jian Chen, Haocheng Zhao, Yu Fu, Yujie Wei, Diwen Zheng, Alois Christian Knoll, Angela P. Schoellig, Ping Luo*, and Jianwei Zhang

Under review † Project leads · * Corresponding author

A latent world–action model evaluates proposed motion–hand command sequences before execution. An action-conditioned endpoint predictor estimates visual progress toward demonstration-derived subgoals, while a joint temporal prior scores temporal consistency. The robot executes a short prefix of the best-scoring sequence and replans from the next observation.

ForceDelta-VLA framework: a frozen force-aware teacher defines force and delay correction targets that are distilled into a fast student correction policy running at 100 Hz on top of slow reference actions, with a real-world comparison where ForceVLA fails and ForceDelta-VLA succeeds at lower contact force.

ForceDelta-VLA: Distilling Force-Conditioned Action Corrections for Contact-Rich Manipulation

Ju Dong, Yu Fu, Jian Chen, Yimeng Liu, Haocheng Zhao, Lei Zhang, Kaixin Bai, Liding Zhang, Diwen Zheng, Alois Christian Knoll, Angela P. Schoellig, and Jianwei Zhang

Under review

A correction-distillation framework that separates a slow reference action from a fast force correction: paired predictions from a frozen teacher's force-conditioned and force-agnostic modes define an explicit correction target, so a lightweight student can adjust actions at 100 Hz from force history and robot state without regenerating action chunks. 82.2% mean success across nine single-arm and bimanual contact-rich tasks versus 54.4% for ForceVLA, with roughly 26% lower mean peak contact force.

Teacher–student distillation framework and four real-world manipulation tasks: door opening, dynamic pick and place, pick and place into a microwave, and cleaning.

From Flow to One Step: Real-Time Multi-Modal Trajectory Policies via Implicit Maximum Likelihood Estimation-based Distribution Distillation

Ju Dong*, Liding Zhang, Lei Zhang*, Yu Fu, Kaixin Bai, Zoltán-Csaba Márton, Zhenshan Bing, Zhaopeng Chen, Alois Christian Knoll, and Jianwei Zhang

IROS 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems · *Equal contribution

Distills a Conditional Flow Matching expert into a single-step student policy via Implicit Maximum Likelihood Estimation, using a bi-directional Chamfer objective to preserve multi-modality instead of collapsing to averaged trajectories. Achieves 70.0% success at 125 Hz on real hardware — a 43x speedup over the multi-step teacher.

M4Diffuser architecture with multi-view cameras feeding a MultiView-DiT policy and ReM-QP whole-body controller, above six mobile manipulation subtasks: reaching, grasping, search, navigate, place, and open.

M4Diffuser: Multi-View Diffusion Policy with Manipulability-Aware Control for Robust Mobile Manipulation

Ju Dong, Lei Zhang, Liding Zhang, Yao Ling, Yu Fu, Kaixin Bai, Zoltán-Csaba Márton, Zhenshan Bing, Zhaopeng Chen, Alois Christian Knoll, and Jianwei Zhang

ICRA 2026 IEEE International Conference on Robotics and Automation

A multi-view diffusion transformer generates end-effector goals in the world frame, executed by ReM-QP — a whole-body QP controller that drops slack variables for efficiency and adds an inverse-condition-number manipulability preference for stability near singularities. 7–56% higher success rates and 3–31% fewer collisions than baselines.

ResponsibleRobotBench overview: hazardous manipulation scenes, skill, pose and code action representations, and evaluation from hazard prediction through mitigation and execution to task completion and terminal safety.

ResponsibleRobotBench: Benchmarking Risk-Aware Task-Level Decision Making for Robotic Manipulation with Large Multimodal Models

Lei Zhang*, Ju Dong*, Kaixin Bai, Minheng Ni, Yu Fu, Zoltán-Csaba Márton, Zhaopeng Chen, and Jianwei Zhang

Under review * Equal contribution

A benchmark of 23 manipulation task configurations connects hazard recognition, mitigation decisions, action prerequisites, execution feedback, and terminal outcomes. Evaluations of five large multimodal models show that recognizing a hazard does not ensure executable mitigation, and terminal safety can diverge from safe task completion.

Education

2026–present

Ph.D. in Informatics — University of Hamburg, Germany

TAMS (Technical Aspects of Multimodal Systems), advised by Prof. Jianwei Zhang.

2023–2026

M.Sc. in Mechatronics and Robotics — Technical University of Munich, Germany

GPA 1.3/1.0. Master thesis: “Multi-Modal Bimanual Robotic Manipulation via Vision-Language-Action Models.”

Experience

Mar 2025–present

Working Student — Artificial Intelligence Department, Agile Robots SE, Munich, Germany

Developed, trained, and deployed diffusion policies on a dual-arm robotic platform; built a benchmark for evaluating LLMs/VLMs in responsible robotic manipulation; implemented high-speed ball catching with a dexterous robotic hand.

Service

2026

EBiM Benchmark — Toward a Globally Coordinated Benchmark for Real-World Embodied Bimanual Manipulation

Competition support team, running cross-site real-robot validation on Mobile FR3 Duo platforms in Hamburg, Munich, and Shanghai.

Contact

Office 04-735
TAMS, Department of Informatics
University of Hamburg
Bundesstraße 56b, Hamburg, Germany

ju.dong@uni-hamburg.de

Scan with WeChat to add me

WeChat QR code to add Ju Dong as a contact