[go: up one dir, main page]
MPI-INF Logo
Homepage

Contact

Firstname Lastname

Dr. Vladislav Golyanik

Research Group Leader and Principal Investigator
4D and Quantum Vision
Max-Planck-Institut für Informatik
D6: Visual Computing and Artificial Intelligence
 office: Campus E1 4, Room 219
Saarland Informatics Campus
66123 Saarbrücken
Germany
 email: golyanik at mpi hyphen inf dot mpg dot de
 phone: +49 681 9325-4505
 fax: +49 681 9325-7505

NEWS

Open Positions

  • PhD positions, post-doc positions and internships are available. Check how to apply.
  • If you are interested in a bachelor/master thesis, an internship, research immersion lab (RIL) or a HiWi position on one of the topics listed below, do not hesitate to reach me out.

Research Profile

    I am currently leading "4D and Quantum Vision" group at Max Planck Institute for Informatics, D6 Department. The focus of our team lies on 3D reconstruction and analysis of general deformable scenes, 3D reconstruction of the human body and matching problems on point sets and graphs. We are interested in neural approaches (both supervised and unsupervised), physics-based methods as well as new hardware and sensors (e.g., quantum computers and event cameras).

    I have initiated several workshops at international computer vision conferences and multiple state-of-the-art reports at EUROGRAPHICS (different years). I have also published under the VIA Center affiliation.

    Many research questions at the intersection of computer graphics, computer vision and machine learning involve challenging search problems (e.g., graph matching) or the optimisation of non-convex objectives. For such problems, we develop new algorithmic formulations that can be solved on modern adiabatic quantum annealers or universal quantum computers and investigate which advantages these approaches offer compared to existing classical methods.

    My research interests include:
    • 3D Reconstruction and Neural Rendering of Rigid and Non-Rigid Scenes
    • 3D Generative Models
    • Quantum Algorithms for Computer Vision and Graphics

    Community service and recent event organisation:

Slides/Recordings of Recent Talks

Publications

2024

    NeuralClothSim: Neural Deformation Fields Meet the Thin Shell Theory.
    N. Kairanda, M. Habermann, C. Theobalt and V. Golyanik.
    Neural Information Processing Systems (NeurIPS), 2024.
    [paper] [project page] [source code]

    Projected Stochastic Gradient Descent with Quantum Annealed Binary Gradients.
    M. Krahn, M. Sasdelli, F. Yang, V. Golyanik, J. Kannala, T.-J. Chin and T. Birdal.
    Accepted at British Machine Vision Conference (BMVC), 2024.
    [paper] [project page]

    ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions.
    A. Ghosh, R. Dabral, V. Golyanik, C. Theobalt and P. Slusallek.
    European Conference on Computer Vision (ECCV), 2024.
    [paper] [project page] [dataset]

    Relightable Neural Actor with Intrinsic Decomposition and Pose Control.
    D. Luvizon, V. Golyanik, A. Kortylewski, M. Habermann and C. Theobalt.
    European Conference on Computer Vision (ECCV), 2024.
    [paper] [project page] [source code] [data]

    Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models.
    W. Menapace, A. Siarohin, S. Lathuilière, P. Achlioptas, V. Golyanik, S. Tulyakov and E. Ricci.
    ACM Transactions on Graphics (ToG), 2024 (TBP at SIGGRAPH 2024).
    [paper] [project page] [github] [bibtex]

    EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams.
    C. Millerdurai, H. Akada, J. Wang, D. Luvizon, C. Theobalt and V. Golyanik.
    Computer Vision and Pattern Recognition (CVPR), 2024.
    [paper] [project page]

    3D Human Pose Perception from Egocentric Stereo Videos.
    H. Akada, J. Wang, V. Golyanik and C. Theobalt.
    Computer Vision and Pattern Recognition (CVPR), 2024;
    CVPR Highlight.
    [paper] [project page] [UnrealEgo Benckmark]

    VINECS: Video-based Neural Character Skinning.
    Z. Liao, V. Golyanik, M. Habermann and C. Theobalt.
    Computer Vision and Pattern Recognition (CVPR), 2024.
    [paper]

    Holoported Characters: Real-time Free-viewpoint Rendering of Humans from Sparse RGB Cameras.
    A. Shetty, M. Habermann, G. Sun, D. Luvizon, V. Golyanik and C. Theobalt.
    Computer Vision and Pattern Recognition (CVPR), 2024.
    [paper] [project page]

    State of the Art on Diffusion Models for Visual Computing.
    R. Po*, W. Yifan*, V. Golyanik*, K. Aberman, J. Barron, A. H. Bermano, E. R. Chan, T. Dekel, A. Holynski, A. Kanazawa, C. K. Liu, L. Liu, B. Mildenhall, M. Niessner, B. Ommer, C. Theobalt, P. Wonka and G. Wetzstein.
    *Equal contribution
    Eurographics 2024 (Full STARs).
    [paper] [project page] [bibtex]

    Recent Trends in 3D Reconstruction of General Non-Rigid Scenes.
    R. Yunus, J. E. Lenssen, M. Niemeyer, Y. Liao, C, Rupprecht, C. Theobalt, G. Pons-Moll, J.-B. Huang, V. Golyanik and E. Ilg.
    Eurographics 2024 (Full STARs).
    [paper] [talk slides] [project page] [bibtex]

    Fast Non-Rigid Radiance Fields from Monocularized Data.
    M. Kappel, V. Golyanik, S. Castillo, C. Theobalt and M. Magnor.
    IEEE Transactions on Visualization and Computer Graphics (TVCG), 2024.
    [paper] [project page] [source code] [bibtex] [video]

    3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera.
    C. Millerdurai, D. Luvizon, V. Rudnev, A. Jonas, J. Wang, C. Theobalt and V. Golyanik.
    International Conference on 3D Vision (3DV), 2024; Spotlight
    [paper] [project page] [poster]

    Quantum-Hybrid Stereo Matching With Nonlinear Regularization and Spatial Pyramids.
    C. Braunstein, E. Ilg and V. Golyanik.
    International Conference on 3D Vision (3DV), 2024.
    [paper] [project page]

    SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes.
    E. Tretschk, V. Golyanik, M. Zollhöfer, A. Bozic, C. Lassner and C. Theobalt.
    International Conference on 3D Vision (3DV), 2024.
    [paper] [project page]

    ROAM: Robust and Object-aware Motion Generation using Neural Pose Descriptors.
    W. Zhang, R. Dabral, T. Leimkühler, V. Golyanik*, M. Habermann* and C. Theobalt.
    * equal advising and contribution.
    International Conference on 3D Vision (3DV), 2024.
    [paper] [project page]

    MACS: Mass-Conditioned 3D Hand and Object Motion Synthesis.
    S. Shimada, F. Mueller, J. Bednařík, B. Doosti, B. Bickel, D. Tang, V. Golyanik, J. Taylor, C. Theobalt and T. Beeler.
    International Conference on 3D Vision (3DV), 2024.

    [paper] [project page]

2023

    Decaf: Monocular Deformation Capture for Face and Hand Interactions.
    S. Shimada, V. Golyanik, P. Pérez, and C. Theobalt.
    ACM Transactions on Graphics (TOG), SIGGRAPH Asia, 2023.
    [paper] [project page] [supplementary video]

    AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars.
    M. Mendiratta, X. Pan, M. Elgharib, K. Teotia, Mallikarjun B R, A. Tewari, V. Golyanik, A. Kortylewski and C. Theobalt.
    ACM Transactions on Graphics (TOG), SIGGRAPH ASIA, 2023.
    [arxiv] [project page]

    Discovering Fatigued Movements for Virtual Character Animation.
    N. Cheema, R, Xu, N. H. Kim, P. Hämäläinen, V. Golyanik, M. Habermann, C. Theobalt, P. Slusallek.
    SIGGRAPH ASIA, 2023 (conference paper).
    [arxiv] [project page]


    3D-QAE: Fully Quantum Auto-Encoding of 3D Point Clouds.
    L. Rathi, E. Tretschk, C. Theobalt, R. Dabral and V. Golyanik.
    British Machine Vision Conference (BMVC), 2023.
    [paper] [project page]

    EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors.
    X. Yi, Y. Zhou, M. Habermann, V. Golyanik, S. Pan, C. Theobalt and F. Xu.
    SIGGRAPH, 2023.
    [arxiv] [project page] [source code]

    Quantum Multi-Model Fitting.
    M. Farina, L. Magri, W. Menapace, E. Ricci, V. Golyanik and F. Arrigoni.
    Computer Vision and Pattern Recognition (CVPR), 2023;
    CVPR Highlight (selected 10%).
    [paper] [project page] [source code]

    MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis.
    R. Dabral, M. H. Mughal, V. Golyanik and C. Theobalt.
    Computer Vision and Pattern Recognition (CVPR), 2023;
    CVPR Highlight (selected 10%).
    [paper] [project page] [bibtex]

    EventNeRF: Neural Radiance Fields from a Single Colour Event Camera.
    V. Rudnev, M. Elgharib, C. Theobalt, V. Golyanik.
    Computer Vision and Pattern Recognition (CVPR), 2023.
    [paper] [project page] [bibtex]

    CCuantuMM: Cycle-Consistent Quantum-Hybrid Matching of Multiple Shapes.
    H. Bhatia, E. Tretschk, Z. Lähner, M. Seelbach Benkner, M. Moeller, C. Theobalt and V. Golyanik.
    Computer Vision and Pattern Recognition (CVPR), 2023.
    [paper] [project page] [source code]

    Self-supervised Pre-training with Masked Shape Prediction for 3D Scene Understanding.
    L. Jiang, Z. Yang, S. Shi, V. Golyanik, D. Dai and B. Schiele.
    Computer Vision and Pattern Recognition (CVPR), 2023.
    [paper]

    Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Model.
    E. C. M. Johnson, M. Habermann, S. Shimada, V. Golyanik and C. Theobalt.
    Computer Vision and Pattern Recognition (CVPR) Workshops, 2023.
    [paper] [project page] [source code] [data] [bibtex]

    QuAnt: Quantum Annealing with Learnt Couplings.
    M. Seelbach Benkner, M. Krahn, E. Tretschk,
    Z. Lähner, M. Moeller and V. Golyanik.
    International Conference on Learning Representations (ICLR), 2023;
    Oral (top 25% of the accepted papers).
    [project page] [paper] [bibtex]


    State of the Art in Dense Monocular Non-Rigid 3D Reconstruction.
    E. Tretschk*, N. Kairanda*, M. B R, R. Dabral, A. Kortylewski, B. Egger, M. Habermann, P. Fua, C. Theobalt and V. Golyanik.
    * equal contribution.
    Eurographics 2023 (Full STARs).
    [draft] [project page] [bibtex]

    Scene-Aware 3D Multi-Human Motion Capture from a Single Camera.
    D. Luvizon, M. Habermann, V. Golyanik, A. Kortylewski and C. Theobalt.
    Eurographics 2023.
    [paper] [project page]

    IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions.
    A. Ghosh, R. Dabral, V. Golyanik, C. Theobalt and P. Slusallek.
    Eurographics 2023.
    [project page] [paper] [bibtex]

2022

    HiFECap: Monocular High-Fidelity and Expressive Capture of Human Performances.
    Y. Jiang, M. Habermann, V. Golyanik and C. Theobalt.
    British Machine Vision Conference (BMVC), 2022.
    [project page] [paper] [bibtex]

    Generation of Truly Random Numbers on a Quantum Annealer.
    H. Bhatia, E. Tretschk, C. Theobalt and V. Golyanik.
    IEEE Access 2022.
    [project page] [paper] [bibtex]

    Q-FW: A Hybrid Classical-Quantum Frank-Wolfe for Quadratic Binary Optimization.
    A. Yurtsever, T. Birdal and V. Golyanik.
    European Conference on Computer Vision (ECCV), 2022.
    [project page] [paper] [bibtex] [poster]

    UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture.
    H. Akada, J. Wang, S. Shimada, M. Takahashi, C. Theobalt and V. Golyanik.
    European Conference on Computer Vision (ECCV), 2022.
    [project page] [paper] [bibtex]

    Quantum Motion Segmentation.
    F. Arrigoni, W. Menapace, M. Seelbach Benkner, E. Ricci and V. Golyanik.
    European Conference on Computer Vision (ECCV), 2022.
    [project page] [paper] [bibtex]

    HULC: 3D HUman Motion Capture with Pose Manifold Sampling and Dense Contact Guidance.
    S. Shimada, V. Golyanik, Z. Li, P. Pérez, W. Xu and C. Theobalt.
    European Conference on Computer Vision (ECCV), 2022.
    [paper] [project page]

    Neural Radiance Fields for Outdoor Scene Relighting.
    V. Rudnev, M. Elgharib, W. Smith, L. Liu, V. Golyanik and C. Theobalt.
    European Conference on Computer Vision (ECCV), 2022.
    [paper] [project page] [bibtex]

    MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes.
    Z. Li, S. Shimada, B. Schiele, C. Theobalt and V. Golyanik.
    International Conference on 3D Vision (3DV), 2022; Oral.
    Best Student Paper Award.
    [project page] [paper] [bibtex]

    φ-SfT: Shape-from-Template with a Physics-Based Deformation Model.
    N. Kairanda, E. Tretschk, M. Elgharib, C. Theobalt and V. Golyanik.
    Computer Vision and Pattern Recognition (CVPR), 2022.
    [paper] [project page] [source code] [bibtex]


    Playable Environments: Video Manipulation in Space and Time.
    W. Menapace, S. Lathuilière*, A. Siarohin, C. Theobalt*, S. Tulyakov*, V. Golyanik*, and E. Ricci*.
    * equal senior contribution.
    Computer Vision and Pattern Recognition (CVPR), 2022.
    [paper] [project page] [github] [bibtex]

    Advances in Neural Rendering.
    A. Tewari*, J. Thies*, B. Mildenhall*, P. Srinivasan*, E. Tretschk, Y. Wang, C. Lassner, V. Sitzmann, R. Martin-Brualla, S. Lombardi, C. Theobalt, M. Niessner, J. T. Barron, G. Wetzstein, M. Zollhöfer and V. Golyanik.
    * equal contribution.
    State of the Art Report at Eurographics 2022.
    [paper] [project page] [bibtex]

2021

    Convex Joint Graph Matching and Clustering via Semidefinite Relaxations.
    M. Krahn, F. Bernard and V. Golyanik.
    International Conference on 3D Vision (3DV), 2021.
    [paper] [project page] [bibtex]

    HumanGAN: A Generative Model of Human Images.
    K. Sarkar, L. Liu, V. Golyanik, and C. Theobalt.
    International Conference on 3D Vision (3DV), 2021; Oral
    [paper] [project page] [bibtex]

    HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks.
    J. Malik, S. Shimada, A. Elhayek, S. A. Ali, C. Theobalt, V. Golyanik and D. Stricker.
    Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021.
    [IEEE Xplore] [arXiv.org] [project page] [bibtex]


    Gravity-Aware 3D Human-Object Reconstruction.
    R. Dabral, S. Shimada, A. Jain, C. Theobalt and V. Golyanik.
    International Conference on Computer Vision (ICCV), 2021.
    [paper] [project page] [bibtex]



    Q-Match: Iterative Shape Matching via Quantum Annealing.
    M. Seelbach Benkner, Z. Lähner, V. Golyanik, C. Wunderlich, C. Theobalt and M. Moeller.
    International Conference on Computer Vision (ICCV), 2021.
    [paper] [project page] [bibtex]


    Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video.
    E. Tretschk, A. Tewari, V. Golyanik, M. Zollhöfer, C. Lassner and C. Theobalt.
    International Conference on Computer Vision (ICCV), 2021.
    [paper] [project page] [source code] [bibtex]

    Neural Monocular 3D Human Motion Capture with Physical Awareness.
    ("Neural PhysCap")
    S. Shimada, V. Golyanik, W. Xu, P. Pérez and C. Theobalt.
    SIGGRAPH, 2021.
    [paper] [arXiv] [bibtex] [project page] [source code]


    High-Fidelity Neural Human Motion Transfer from Monocular Video.
    M. Kappel, V. Golyanik, M. Elgharib, J.-O. Henningson, H.-P. Seidel, S. Castillo, C. Theobalt and M. Magnor.
    Computer Vision and Pattern Recognition (CVPR), 2021; Oral.
    [paper] [project page] [bibtex] [source code]

    Pose-Guided Human Animation from a Single Image in the Wild.
    J. S. Yoon, L. Liu, V. Golyanik, K. Sarkar, H. S. Park, and C. Theobalt.
    Computer Vision and Pattern Recognition (CVPR), 2021.
    [paper] [project page] [video] [bibtex]


    Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations.
    S. A. Ali, K. Kahraman, C. Theobalt, D. Stricker and V. Golyanik.
    IEEE Access, 2021.
    [paper] [arXiv] [project page] [bibtex]

2020

    PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time.
    S. Shimada, V. Golyanik, W. Xu and C. Theobalt.
    SIGGRAPH Asia, 2020.
    [paper (arXiv.org)] [bibtex] [project page]

    Egocentric Videoconferencing.
    M. Mendiratta*, M. Elgharib*, J. Thies, M. Nießner, H.-P. Seidel, A. Tewari,
    V. Golyanik and C. Theobalt.
    * equal contribution.
    SIGGRAPH Asia, 2020.
    [draft] [supplement] [bibtex] [project page]

    Fast Simultaneous Gravitational Alignment of Multiple Point Sets.
    V. Golyanik, S. Shimada and C. Theobalt.
    3DV, 2020; Oral.
    [draft] [bibtex] [project page]

    Adiabatic Quantum Graph Matching with Permutation Matrix Constraints.
    M. Seelbach Benkner, V. Golyanik, C. Theobalt and M. Moeller.
    3DV, 2020.
    [draft] [supplement] [bibtex] [project page]



    HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization.
    N. Qian, J. Wang, F. Müller, F. Bernard, V. Golyanik and C. Theobalt.
    European Conference on Computer Vision (ECCV), 2020.
    [paper] [supplement] [video] [bibtex] [project page]




    A Quantum Computational Approach to Correspondence Problems on Point Sets.
    V. Golyanik and C. Theobalt.
    In Computer Vision and Pattern Recognition (CVPR), 2020.
    [paper] [slides] [poster] [bibtex] [arXiv] [project page]

    EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera.
    L. Xu, W. Xu, V. Golyanik, M. Habermann, L. Fang and C. Theobalt.
    In Computer Vision and Pattern Recognition (CVPR), 2020; Oral
    [paper] [supplement] [bibtex] [arXiv] [project page]


    HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map.
    J. Malik, I. Abdelaziz, A. Elhayek, S. Shimada, S. A. Ali, V. Golyanik, C. Theobalt and D. Stricker.
    In Computer Vision and Pattern Recognition (CVPR), 2020.
    [paper] [supplement] [bibtex] [arXiv] [project page]



2019 and before (selected publications)

    Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data.
    O. Kovalenko, V. Golyanik, J. Malik, A. Elhayek and D. Stricker.
    Sensors (Volume 19, Issue 20), 2019.
    [paper] [project page]


    DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxies.
    S. Shimada, V. Golyanik, E. Tretschk, D. Stricker and C. Theobalt.
    In International Conference on 3D Vision (3DV), 2019; Oral
    [paper] [poster] [presentation] [project page] [arXiv] [bibtex]

    IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction.
    S. Shimada, V. Golyanik, C. Theobalt and D. Stricker.
    Computer Vision and Pattern Recognition Workshops
    (Photogrammetric Computer Vision Workshop), 2019; Oral
    [paper] [bibtex] [arXiv] [project page]

    Consolidating Segmentwise Non-Rigid Structure from Motion.
    V. Golyanik, A. Jonas and D. Stricker.
    Machine Vision Applications (MVA), 2019; Oral
    [paper] [project page] [bibtex]

    NRGA: Gravitational Approach for Non-Rigid Point Set Registration.
    S. A. Ali. V. Golyanik and D. Stricker.
    International Conference on 3D Vision (3DV), 2018; Oral
    [paper] [Supplementary Video (Download, YouTube)] [poster] [bibtex]


    HDM-Net: Monocular Non-Rigid 3D Reconstruction with Learned Deformation Model.
    V. Golyanik, S. Shimada, K. Varanasi and D. Stricker.
    EuroVR, 2018; Oral (Long Paper)
    [paper] [HDM-Net data set] [bibtex]

    Multiframe Scene Flow with Piecewise Rigid Motion.
    V. Golyanik, K. Kim, R. Maier, M. Nießner, D. Stricker and J. Kautz.
    International Conference on 3D Vision (3DV), 2017; Spotlight Oral
    [paper] [arXiv] [supplementary material] [poster] [bibtex]

    Scalable Dense Monocular Surface Reconstruction.
    M.D.Ansari, V. Golyanik and D. Stricker.
    International Conference on 3D Vision (3DV), 2017.
    [paper] [arXiv] [bibtex]

    A Framework for an Accurate Point Cloud Based Registration of Full 3D Human Body Scans.
    V. Golyanik, G. Reis, B. Taetz and D. Stricker.
    Machine Vision Applications (MVA), 2017.
    [paper] [bibtex]

    Dense Batch Non-Rigid Structure from Motion in a Second.
    V. Golyanik and D. Stricker.
    Winter Conference on Applications of Computer Vision (WACV), 2017.
    [paper] [supplementary video] [poster] [bibtex]

    Accurate 3D Reconstruction of Dynamic Scenes from Monocular Image Sequences with Severe Occlusions.
    V. Golyanik, T. Fetzer and D. Stricker.
    Winter Conference on Applications of Computer Vision (WACV), 2017.
    [paper] [supplementary material] [poster] [arXiv] [bibtex]

    Gravitational Approach for Point Set Registration.
    V. Golyanik, S. A. Ali and D. Stricker.
    Computer Vision and Pattern Recognition (CVPR), 2016.
    [paper] [supplementary material] [bibtex]

    Extended Coherent Point Drift Algorithm with Correspondence Priors and Optimal Subsampling.
    V. Golyanik, B. Taetz, G. Reis and D. Stricker.
    Winter Conference on Applications of Computer Vision (WACV), 2016.
    [paper] [poster] [bibtex] [WACV Talk]

    Occlusion-Aware Video Registration for Highly Non-Rigid Objects.
    B. Taetz, G. Bleser, V. Golyanik and D. Stricker.
    Winter Conference on Applications of Computer Vision (WACV), 2016.
    Best Paper Award.
    [paper] [supplementary material] [bibtex] [WACV Talk]