Toward Diffusible High-Dimensional Latent Spaces: A Frequency Perspective
Why higher-dimensional autoencoder latents improve reconstruction but eventually hurt generation, studied through a frequency lens.
23 papers · 2019 – 2026
Research
Generative models for images and video, and before that object discovery, detection, tracking and re-identification. * equal contribution, † project lead. The full list lives on Google Scholar.
Highlights
Video
Movie Gen: A Cast of Media Foundation Models
Foundation models that generate HD video with synchronized audio, plus instruction-based video editing and personalized video generation.
Video
Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning
State-of-the-art diffusion-based text-to-video generation by first generating an image, then a video conditioned on it.
Toward Diffusible High-Dimensional Latent Spaces: A Frequency Perspective
Why higher-dimensional autoencoder latents improve reconstruction but eventually hurt generation, studied through a frequency lens.
Diffusion Autoencoders are Scalable Image Tokenizers
DiTo: a simple diffusion tokenizer that learns compact visual representations for image generation with a single learning objective.
MotiF: Making Text Count in Image Animation with Motion Focal Loss
A motion focal loss that makes text-guided image animation follow the motion described in the prompt.
InstanceDiffusion: Instance-level Control for Image Generation
Precise instance-level control — points, boxes, scribbles, masks — for text-to-image generation.
MOST: Multiple Object Localization with Self-supervised Transformers for Object Discovery
Localizes multiple objects in real-world images without any training, using self-supervised transformers.
Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognition
Semantic-aware point selection and relational trajectory tokens that model motion and appearance for few-shot action recognition.
Trajectory-aligned Space-time Tokens for Few-shot Action Recognition
Point trajectories and self-supervised representations for few-shot action recognition.
UVIS: Unsupervised Video Instance Segmentation
Video instance segmentation without any video annotations or densely labeled pre-training.
The Pursuit of Knowledge: Discovering and Localizing Novel Categories using Dual Memory
Automatically discovers and learns models for new categories from a large unlabeled dataset.
SparseDet: Improving Sparsely Annotated Object Detection with Pseudo-positive Mining
Treats missing annotations as region-based semi-supervised learning to improve detectors.
Towards Accurate Visual and Natural Language-based Vehicle Retrieval Systems
A real-time system for image-based vehicle re-identification and natural-language vehicle retrieval (AI City Challenge).
A Dual-Path Model With Adaptive Attention for Vehicle Re-Identification
A dual-path adaptive attention model for vehicle re-identification.
Body Part Alignment and Temporal Attention Pooling for Video-Based Person Re-Identification
Feature alignment across body parts for state-of-the-art video person re-identification.
An Empirical Analysis of Boosting Deep Networks
A single large network is usually more accurate than a boosted ensemble with the same total parameters.
All papers
2026
2025
Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognition
Semantic-aware point selection and relational trajectory tokens that model motion and appearance for few-shot action recognition.
MotiF: Making Text Count in Image Animation with Motion Focal Loss
A motion focal loss that makes text-guided image animation follow the motion described in the prompt.
Diffusion Autoencoders are Scalable Image Tokenizers
DiTo: a simple diffusion tokenizer that learns compact visual representations for image generation with a single learning objective.
2024
Video
Movie Gen: A Cast of Media Foundation Models
Foundation models that generate HD video with synchronized audio, plus instruction-based video editing and personalized video generation.
Video
Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning
State-of-the-art diffusion-based text-to-video generation by first generating an image, then a video conditioned on it.
Trajectory-aligned Space-time Tokens for Few-shot Action Recognition
Point trajectories and self-supervised representations for few-shot action recognition.
InstanceDiffusion: Instance-level Control for Image Generation
Precise instance-level control — points, boxes, scribbles, masks — for text-to-image generation.
UVIS: Unsupervised Video Instance Segmentation
Video instance segmentation without any video annotations or densely labeled pre-training.
SelfEval: Leveraging the Discriminative Nature of Generative Models for Evaluation
Repurposes generative models as discriminative models to evaluate generative performance.
2023
MOST: Multiple Object Localization with Self-supervised Transformers for Object Discovery
Localizes multiple objects in real-world images without any training, using self-supervised transformers.
SparseDet: Improving Sparsely Annotated Object Detection with Pseudo-positive Mining
Treats missing annotations as region-based semi-supervised learning to improve detectors.
2022
Recognizing Actions using Object States
Recognizes object-centric actions from just two object states, start and end.
An Empirical Analysis of Boosting Deep Networks
A single large network is usually more accurate than a boosted ensemble with the same total parameters.
2021
The Pursuit of Knowledge: Discovering and Localizing Novel Categories using Dual Memory
Automatically discovers and learns models for new categories from a large unlabeled dataset.
Towards Discovery and Attribution of Open-world GAN Generated Images
Discovers and attributes images from unseen GANs in the open world.
Self-Denoising Neural Networks for Few-shot Learning
An architecture based on denoising auto-encoders that improves few-shot learning.
Towards Accurate Visual and Natural Language-based Vehicle Retrieval Systems
A real-time system for image-based vehicle re-identification and natural-language vehicle retrieval (AI City Challenge).
2020
Towards Real-Time Systems for Vehicle Re-Identification, Multi-Camera Tracking, and Anomaly Detection
Real-time vehicle re-ID, multi-camera tracking and anomaly detection across a network of traffic cameras (AI City Challenge).
Detecting Human-Object Interactions via Functional Generalization
Humans interact with functionally similar objects in similar ways.
Spatial Priming for Detecting Human-Object Interactions
Exploits the spatial layout of a human and an object to detect HOIs in images.
2019
A Dual-Path Model With Adaptive Attention for Vehicle Re-Identification
A dual-path adaptive attention model for vehicle re-identification.
Body Part Alignment and Temporal Attention Pooling for Video-Based Person Re-Identification
Feature alignment across body parts for state-of-the-art video person re-identification.