Publicaions

We frame open-vocabulary long-horizon manipulation as physical orchestration and present VoLoAgent, a VLM agent that plans, monitors, and recovers by steering a VLA/WAM as an interruptible tool alongside perception models and grasp/place primitives. We also introduce RoboVoLo, a high-fidelity benchmark of 126 tasks across common sense, memory, complex references, and world knowledge, to support the evaluation of open-vocabulary long-horizon robot manipulation.

We propose dense alignment for heterogeneous KV-cache communication, enabling agents with different model architectures to transfer both contextual knowledge and reasoning signals.

We provide theoretical insights into the connection between diffusion models and subspace clustering, shedding light on the transition from memorization to generalization and how diffusion models break the curse of dimensionality.

SpaceTools empowers VLMs with vision and robotic tools for spatial reasoning via Double Interactive Reinforcement Learning (DIRL), enabled by our Toolshed infrastructure. Achieves state-of-the-art performance on spatial reasoning benchmarks and enables precise real-world robot manipulation.

We study token-reduction-based acceleration for unified VLM training, uncover a fundamental asymmetry in visual token usage across depth, and reveal that naive task-specific token dropping breaks cross-task synergy in joint optimization.

We investigate why diffusion models excel at representation learning despite being designed for generation. We show that unimodal representation dynamics—where feature quality peaks at an intermediate noise level—emerge when the model captures the data distribution, and reliably indicate generalization versus memorization.

We introduce a novel framework for auditing and improving the robustness of unlearned diffusion models by proposing an interpretable subspace attack, that reveals latent vulnerabilities and inspires a corresponding projection-based defense to surgically remove them.

We propose probability flow distance (PFD), a theoretically grounded and computationally efficient metric to measure distributional generalization. By using PFD under a teacher-student evaluation protocol, we empirically uncover several key novel generalization behaviors in diffusion models.

We study gradient flow to explain modality gap in CLIP, and propose new methods to mitigate modality gap while improving model performance.

We enable localized, transferable, linear, and composable image editing on diffsion models by exploring their low-rank and locally linear semantic spaces.

Inspired by physical motion, we unfold a video clip via Taylor expansion and design an alternative algorithm for self-supervised video representation learning. Our proposed method can steer the model to dynamic parts in the video.

We propose to investigate detecting and characterizing the 3D planar articulation of objects from ordinary videos.

Teaching

Graduate Student Instructor, EECS 559 Optimization, 2024 & 2025

Undergraduate Instructional Assistant, EECS 442 Computer Vision, 2022

Teaching Assistant, VE 401 Probabilitic Methods, 2020

Teaching Assistant, VV 286 Honorable Mathematics, 2020

Explorations & earlier projects

As a team of five, we designed a horror game, Asylum 7, with Unity.

We generate 3D synthetic video dataset containing a moving object and a scene. The pose and position of the object is optimized via a differential render.

We predict 3D partial human poses as SMPL meshes, predict 2D plane masks as well as 3D articulation information, and use a differential render to optimize the position and pose of the person considering 3d space interactions.

We designed and implemented beta versions of enumerating origamis in H(2) and utilized SageMath to implement the convexity test presented by Lelievre and Weiss.

We devise a rule to evaluate the safety and difficulty level of roller coasters, propose a novel roller coster model, and give a through analysis based on Euler's method and natural axes.

This webpage is based on Jon Barron.