[go: up one dir, main page]

I am VP Research at Stability AI, leading the generative vision research on image, video and 3D foundation models. In the past, I was a researcher at Google Research and Nvidia Research. I completed my PhD at Perceiving Systems department, Max-Planck Institute (MPI) for Intelligent Systems in Tübingen, Germany. My PhD advisor was Prof. Peter V. Gehler. I did my bachelors and masters in Computer Science at IIIT-Hyderabad, India.

At Stability AI, we strive to advance the state-of-the-art in vision foundation models with diverse applications in understanding, reconstruction and generation tasks in both 2D and 3D. A central research question is how to train generative models that can act as generic yet powerful priors for the visual world around us.

If you are interested in joining my Generative Vision team at Stability or research collaborations or internships, please drop me an email with your CV.

News

July, 2024 : Papers on ‘Stable Video 3D’, ‘3D-Congealing’, ‘Zero-shot material transfer’, ‘stylized human motion diffusion’, ‘ZipLoRA for merging style LoRAs’, ‘Text generation of 3D garments’, ‘Joint 2D-3D pretraining from multi-view images’ accepted to ECCV’24.

July, 2024 : Area chairing for CVPR’25

May, 2024 : Area chairing for BMVC’24 and NeurIPS’24.

March, 2024 : Papers on ‘3D awareness in foundation models’, ‘Multi-view diffusion’, ‘One-shot affordance learning’, ‘Neural BRDF decomposition’, ‘DiffusionLight’, ‘Shape Regression’, ‘NeRF Inpainting’, ‘Diffusion material control’, ‘Joint image generation and segmentation’, ‘Geometric semantic correspondences’, ‘HyperDreamBooth’ accepted to CVPR’24.

January, 2024 : A paper on ‘Controllable human motion generation’ accepted to ICLR’24.

January, 2024 : Area chairing for ICML’24 and ECCV’24.

November, 2023 : Joined Stability AI as a research lead.

October, 2023 : Invited talk on ‘3D of Everything from internet image collections’ at ICCV’23 tutorial on Learning with Noisy and Unlabeled Data.

October, 2023 : A paper on ‘Articulated hand-object pose estimation’ (oral) accepted to 3DV’24.

see all news

varunjampani@gmail.com
Stability AI
Boston, MA, USA.

Publications

Refer to my Google Scholar page for a more up-to-date list of publications.

Probing the 3D Awareness of Visual Foundation Models

M. El. Banani, A. Raj, K. Maninis, A. Kar, Y. Li, M. Rubinstein, D. Sun, L. Guibas, J. Johnson, V. Jampani

Computer Vision and Pattern Recognition, CVPR’24

pdf / code (github)

SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild

A. Engelhardt, A. Raj, M. Boss, Y. Zhang, A. Kar, Y. Li, D. Sun, R. M. Brualla, J. T. Barron, H. Lensch, V. Jampani

Computer Vision and Pattern Recognition, CVPR’24

pdf / project page / code (coming soon)

One-Shot Open Affordance Learning with Foundation Models

G. Li, D. Sun, L. Sevilla-Lara, V. Jampani

Computer Vision and Pattern Recognition, CVPR’24

pdf

Alchemist: Parametric Control of Material Properties with Diffusion Models

P. Sharma, V. Jampani, Y. Li, X. Jia, D. Lagun, F. Durand, W. T. Freeman, M. Matthews

Computer Vision and Pattern Recognition, CVPR’24 (oral)

pdf / project page / code (coming soon)

DiffusionLight: Light Probes for Free by Painting a Chrome Ball

P. Phongthawee, W. Chinchuthakun, N. Sinsunthithet, A. Raj, V. Jampani, P. Khungurn, S. Suwajanakorn

Computer Vision and Pattern Recognition, CVPR’24 (oral)

pdf / project page / code (github) / colab notebook

MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generation

H. Hu, Z. Zhou, V. Jampani, S. Tulsiani

Computer Vision and Pattern Recognition, CVPR’24

pdf / project page / code (github)

UniGS: Unified Representation for Image Generation and Segmentation

L. Qi, L. Yang, W. Guo, Y. Xu, B. Du, V. Jampani, M. H. Yang

Computer Vision and Pattern Recognition, CVPR’24

pdf / code (github)

ZeroShape: Regression-based Zero-shot Shape Reconstruction

Z. Huang, S. Stojanov, A. Thai, V. Jampani, J. M. Rehg

Computer Vision and Pattern Recognition, CVPR’24

pdf / project page / code (github) / demo (HF)

Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence

J. Zhang, C. Herrmann, J. Hur, E. Chen, V. Jampani, D. Sun, M. H. Yang

Computer Vision and Pattern Recognition, CVPR’24

pdf / project page / code (github)

Nerfiller: Completing Scenes via Generative 3D Inpainting

E Weber, A Holynski, V Jampani, S Saxena, N Snavely, A Kar, A. Kanazawa

Computer Vision and Pattern Recognition, CVPR’24

pdf / project page / code (github)

HyperDreambooth: Hypernetworks for Fast Personalization of Text-to-Image Models

N. Ruiz, Y. Li, V. Jampani, W.Wei, T. Hou, Y. Pritch, N. Wadhwa, M. Rubinstein, K. Aberman

Computer Vision and Pattern Recognition, CVPR’24

pdf / project page

OmniControl: Control Any Joint at Any Time for Human Motion Generation

Y. Xie, V. Jampani, L. Zhong, D. Sun, H. Jiang

International Conference on Learning Representations, ICLR’24

pdf / project page / code (github)

ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation

Z. Zhu, J. Wang, Y. Qin, D. Sun, V. Jampani, X. Wang

International Conference on 3D Vision, 3DV’24 (oral)

pdf / project page

NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotations

V. Jampani*, K. K. Maninis*, A. Engelhardt, A. Karpur, K. Truong, K. Sargent, S. Popov, A. Araujo, R. M. Brualla, K. Patel, D. Vlasic, V. Ferrari, A. Makadia, C. Liu, Y. Li, H. Zhou (*equal contribution)

Neural Information Processing Systems, NeurIPS’23

pdf / project page / dataset (github)

ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections

C-H. Yao, A. Raj, W-C. Hung, Y. Li, M. Rubinstein, M-H. Yang, V. Jampani

Neural Information Processing Systems, NeurIPS’23

pdf / project page / code (github)

LayoutGPT: Compositional Visual Planning and Generation with Large Language Models

W. Feng*, W. Zhu*, T-J. Fu, V. Jampani, A. Akula, X. He, S. Basu, X. E. Wang, W. Y. Wang (*equal contribution)

Neural Information Processing Systems, NeurIPS’23

pdf / project page / code (github)

A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence

J. Zhang, C. Herrmann, J. Hur, L. P. Cabrera, V. Jampani, D. Sun, M-H. Yang

Neural Information Processing Systems, NeurIPS’23

pdf / project page / code (github)

DreamBooth3D: Subject-Driven Text-to-3D Generation

A. Raj, S. Kaza, B. Poole, M. Niemeyer, N. Ruiz, B. Mildenhall, S. Zada, K. Aberman, M. Rubinstein, J. Barron, Y, Li, V. Jampani

International Conference on Computer Vision, ICCV’23

pdf / project page / video

ASIC: Aligning Sparse in-the-wild Image Collections

K. Gupta, V. Jampani, C. Esteves, A. Shrivastava, A. Makadia, N. Snavely, A. Kar

International Conference on Computer Vision, ICCV’23

pdf / project page / video

LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFs

Z. Cheng, C. Esteves, V. Jampani, A. Kar, S. Maji, A. Makadia

International Conference on Computer Vision, ICCV’23

pdf / project page

MetaCLUE: Towards Comprehensive Visual Metaphors Research

A. Akula, B. Driscoll, P. Narayana, S. Changpinyo, Z. Jia, S. Damle, G. Pruthi, S. Basu, L. Guibas, W. T. Freeman, Y. Li, V. Jampani

Computer Vision and Pattern Recognition, CVPR’23

pdf / project page / video

Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemble

C-H. Yao, W-C. Hung, Y. Li, M. Rubinstein, M-H. Yang, V. Jampani

Computer Vision and Pattern Recognition, CVPR’23

pdf / project page / video / code (github)

LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Grounding

G. Li, V. Jampani, D. Sun, L. Sevilla-Lara

Computer Vision and Pattern Recognition, CVPR’23

pdf / project page / video / code (github)

DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, K. Aberman (oral, best student paper honorable mention)

Computer Vision and Pattern Recognition, CVPR’23

pdf / project page / code (dataset - github)

ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistency

Z. Huang, V. Jampani, N. A. Thai, Y. Li, S. Stojanov, J. M. Rehg

Computer Vision and Pattern Recognition, CVPR’23

pdf / project page / video / code (github)

NoisyTwins: Class-Consistent and Diverse Image Generation through StyleGANs

H. Rangwani, L. Bansal, K. Sharma, T. Karmali, V. Jampani, R. V. Babu

Computer Vision and Pattern Recognition, CVPR’23

pdf / project page / code (github)

Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis

W. Feng, X. He, T. J. Fu, V. Jampani, A. Akula, P. Narayana, S. Basu, X. E. Wang, W. Y. Wang

International Conference on Learning Representations, ICLR’23

pdf / project page / code (github)

LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery

C-H. Yao, W-C. Hung, Y. Li, M. Rubinstein, M-H. Yang, V. Jampani

Neural Information Processing Systems, NeurIPS’22

pdf / project page / video / code (github)

SAMURAI: Shape and Material from Unconstrained Real-world Arbitrary Image Collections

M. Boss, A. Engelhardt, A. Kar, Y. Li, D. Sun, J. Barron, H. Lensch, V. Jampani

Neural Information Processing Systems, NeurIPS’22

pdf / project page / video / code (github)

Polynomial Neural Fields for Subband Decomposition and Manipulation

G. Yang, S. Benaim, V. Jampani, K. Genova, J. Barron, T. Funkhouser, B. Hariharan, S. Belongie

Neural Information Processing Systems, NeurIPS’22

pdf / code (github)


see all publications

Theses

Learning Inference Models for Computer Vision

Learning based techniques for better inference in several computer vision models ranging from inverse graphics to freely parameterized neural networks.

V. Jampani

PhD Thesis, MPI for Intelligent Systems and University of Tübingen, December, 2016

pdf / slides / library

A Study of X-Ray Image Perception for Pneumoconiosis Detection

Eye tracking experimental studies and models for X-Ray Image Perception and Diagnosis.

V. Jampani

Master Thesis, IIIT-Hyderabad, January, 2013

pdf