Research Directions

Our research develops general robot intelligence by integrating learning with priors from physics, geometry, and control. Topics below are organized by theme—each with an overview, demo videos, and selected publications.

Reactive Grasp-and-Place Policy Learning

  • Grasping and placing rigid objects is fundamental, yet robust and general performance—across arbitrary objects, poses, and dynamically changing conditions—remains unsolved, especially for dexterous hands.
  • Real-time feedback control using vision, touch, and proprioception is essential; manipulation should not be reduced to tracking a preplanned trajectory.
  • Imitation learning alone is unlikely to generalize sufficiently. A promising direction is to combine physics-based feedback control, such as MPC, with reinforcement learning.

Selected publications

  1. Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control (Preprint, arXiv)
  2. Hierarchical Reactive Grasping via Task-Space Velocity Fields and Joint-Space Quadratic Programming (ICRA 2026)
  3. High-Bandwidth Tactile-Reactive Control for Grasp Adjustment (ICRA 2026)

Movement Primitives Learning

  • Movement primitives encode reusable “muscle memory” for real-time motion generation, particularly when online optimization is too slow or the task is difficult to model.
  • Examples include complex collision avoidance, athletic motions near kinodynamic limits, and manipulation of non-rigid materials such as water.
  • A promising direction is to combine imitation learning, reinforcement learning, and model-based control.

Selected publications

  1. Differentiable Motion Manifold Primitives for Reactive Motion Generation under Kinodynamic Constraints (ICRA 2026)
  2. Behavior-Controllable Stable Dynamics Models on Riemannian Configuration Manifolds (T-RO 2025)
  3. Equivariant Motion Manifold Primitives (CoRL 2023)

Perception for Robot Manipulation

  • Perception aims to extract task-relevant representations from sensory observations. A fundamental question is: What constitutes a good perceptual representation for robot manipulation?
  • Explicit representations, such as object poses, shapes, and geometric relationships, enable intuitive and structured policy design. However, they may limit expressiveness and require difficult or fragile state-estimation pipelines.
  • Implicit representations, such as learned neural features, can be more flexible and broadly applicable, but they often make policy design, interpretation, and generalization more challenging.
  • Finding the right balance between explicit structure and learned representations for a given class of manipulation tasks remains a key challenge in robot perception.

Selected publications

  1. Point2Pose: Occlusion-Recovering 6D Pose Tracking and 3D Reconstruction for Multiple Unknown Objects Via 2D Point Trackers (ECCV 2026)
  2. ScrewSplat: An End-to-End Method for Articulated Object Recognition (CoRL 2025)
  3. T2SQNet: A Recognition Model for Manipulating Partially Observed Transparent Tableware Objects (CoRL 2024)

Real World Online Adaptation

  • Policies developed in simulation inevitably face modeling errors and unforeseen failure modes in the real world.
  • Real-world trials provide essential feedback on when and why a policy fails, enabling continual online improvement.
  • This requires highly sample-efficient adaptation algorithms that can learn safely from limited real-world experience.

Vision-Language-Object Model + Low-Level Policy

  • Directly learning robot actions with VLAs makes it difficult to achieve robustness to variations in robot configurations and external perturbations, while also scaling across embodiments, action spaces, and hardware platforms.
  • An object-centric model can instead predict object-level goals and interactions, enabling scalable learning from egocentric data collected with devices such as smart glasses.
  • Given an object-level plan, a low-level policy is required to generate reactive, embodiment-specific motions and contact-rich control.

Geometric Machine Learning

  • Data often lie in non-Euclidean spaces; for example, directional data form a circle in 2D and a sphere in 3D, requiring geometry-aware treatment.
  • High-dimensional data may also concentrate near nonlinear low-dimensional manifolds, much like Earth’s two-dimensional surface embedded in three-dimensional space.
  • Algorithms that exploit these geometric structures are essential for accurate, efficient, and robust learning.

Selected publications

  1. Graph Geometry-Preserving Autoencoders (ICML 2024)
  2. A Statistical Manifold Framework for Point Cloud Data (ICML 2022)
  3. Regularized Autoencoders for Isometric Representation Learning (ICLR 2022)