Head Image

Zhaoyang Lv  

Research Scientist, Facebook Reality Labs Research

Previous Education:
Ph.D. in Robotics, School of Interactive Computing, Georgia Institute of Technology
M.Sc., Artificial Intelligence in Computing, Imperial College London
B.Sc., Electrical Engineering in Aeronauntics, Northwestern Polytechnical University

I am a research scientist in Facebook Reality Labs Research, Machine Perception team in Redmond. I finished my Ph.D. at Georgia Tech, jointly advised by Prof. James Rehg, and Prof. Frank Dellaert. During my Ph.D., I am also fortunate to intern at Nvidia Research in the group of Jan Kautz and at Max Planck Institute with Prof. Andreas Geiger. Before I started my Ph.D., I finished my Master thesis under the supervision of Prof. Andrew Davison at Imperial College London.

I am a believer that VR/AR will become ubiquitous and fundamentally change the way we interact with world. Quote Steve Jobs' comments on GUI when he visited Xerox PARC in 1979:
You could argue about the number of years it would take, and you could argue about who would be the winners and the losers, but I don't think you could argue that every computer in the world wouldn't eventually work this way.
I am super excited to work on the multidisciplinary research in this field, with a focus in computer vision, graphics and machine learning.

Our team research focus is to enable the future in LiveMaps . My research interest is to explore how we can photorealistically digitalize the complex dynamic world and render anything at anytime and anywhere, by rethinking the system end-to-end, from sensing and image formation system to the virtual rendered novel view video.

Neural 3D Video Synthesis

Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Zhaoyang Lv
arXiv 2103.02597
Project Page

STaR: Self-supervised Tracking and Reconstruction of Rigid Objects in Motion with Neural Rendering

Wentao Yuan, Zhaoyang Lv, Tanner Schmidt, Steven Lovegrove
Computer Vision and Pattern Recognition (CVPR) 2021, arXiv 2101.01602
Project Page

SENSE: A Shared Encoder Network for Scene-flow Estimation

Huaizu Jiang, Deqing Sun, Varun Jampani, Zhaoyang Lv, Erik Learned-Miller, Jan Kautz
International Conference in Computer Vision (ICCV) 2019 , Supplementary Materials
Oral Presentation

Taking a Deeper Look at the Inverse Compositional Algorithm

Zhaoyang Lv, Frank Dellaert, James M. Rehg, Andreas Geiger
Computer Vision and Pattern Recognition (CVPR) 2019, Supplementary Materials, arXiv 1812.06861
Oral Presentation, Best Paper Finalist (<1%)
Video Slides (5 mins) , Live Recorded Video Presentation (5 mins)
Code , Poster

A Continuous Optimization Approach for Efficient and Accurate Scene Flow

Zhaoyang Lv, Chris Beall, Pablo F. Alcantarilla, Fuxin Li, Zsolt Kira, Frank Dellaert
European Conference on Computer Vision (ECCV) 2016 , arXiv 1607.07983
Project Page

KinfuSeg System Image

KinfuSeg: A Dynamic SLAM Approach Based on KinectFusion

Zhaoyang Lv
Master Thesis , Imperial College London
Video Slides
Thesis Advisor: Prof. Andrew Davison
Distinguished Thesis in Department of Computing (3 among 71), Top 5%

Multi-class Classification without Multi-class Labels

Yen-Chang Hsu, Zhaoyang Lv, Joel Schlosser, Phillip Odom, Zsolt Kira
International Conference on Learning Representations (ICLR) 2019, openreview

Learning to Cluster in Order to Transfer across Domains and Tasks

Yen-Chang Hsu, Zhaoyang Lv, Zsolt Kira
International Conference on Learning Representations (ICLR) 2018, arXiv:1711.10125
Code , A blog post on Machine Learning @ Gerogia Tech

Deep Image Category Discovery using a Transferred Similarity Function

Yen-Chang Hsu, Zhaoyang Lv, Zsolt Kira

miniSAM: A Flexible Factor Graph Non-linear Least Squares Optimization Framework

Jing Dong (main contributor), Zhaoyang Lv
Code , arXiv
Project Website

Motion Planning and Intention Prediction for Autonomous Driving in Highway Scenarios via Graphical Model-Based Factorization

Zhaoyang Lv, Aliakbar Aghamohammadi, Amirhossein Tamjidi
US Patent App. 15/601,047

Holistic Planning with Multiple Intentions for Self-driving Cars

Zhaoyang Lv, Aliakbar Aghamohammadi
US Patent App. 15/604,437


Large-Scale Collaborative Semantic Mapping using 3D Structure from Motion Data

I build a Dense Reconstruction of Georgia Tech with Dr. Chris Beall from Stereo Images Only for this project.
In this video, you can have a fly-through view of the reconstructed campus .

Nvidia Research, Santa Clara, Jan. 2019 - May 2019

Director: Dr. Jan Kautz, Mentors: Dr. Kihwan Kim, Dr. Deqing Sun, Dr. Alejandro Troccoli

Autonomous Vision Group, Max Planck Institute for Intelligent System, Tuebingen, June 2018 - Nov. 2018

Advisor: Prof. Andreas Geiger

Nvidia Research, Santa Clara, May 2017 - Aug. 2017

Director: Dr. Jan Kautz, Mentors: Dr. Kihwan Kim, Dr. Deqing Sun, Dr. Alejandro Troccoli

Qualcomm Research, Greater San Diego, May 2016 - Aug. 2016

Manager: Dr. Ali Agha

Zhejiang University, Hangzhou, Dec. 2013 - July 2014

Mentor: Prof. Guofeng Zhang

Instructor for CS 4476 Introduction to Computer Vision, Georgia Tech, Summer 2019

Teaching assistant for CS 7643 Deep Learning, Georgia Tech, Fall 2017

Instructor: Prof. Dhruv Batra

Teaching assistant for CS 4476 / 6476 Computer Vision, Georgia Tech, Fall 2016

Instructor: Prof. James Hays

Vice President in Public Relation for RoboGrads, Georgia Tech, Fall 2016 - Spring 2017

Organizer for GT Computer Vision Reading Group, Georgia Tech, Spring 2015 - Fall 2018

I started to organize the CPL reading group as a computer vision research discussion group across Computational Perception Lab (CPL) since 2015, and now there have been an active particaption from students in computer vision research in different labs across the campus. If you are interested to join or receive future notifications, please join our google group (It's open access, you can enroll yourself with your gmail account).

Multiple reviewer services for T-PAMI, IJCV, T-MM, CVPR, ICCV, ICRA, IROS

Outstanding Reviewer, CVPR 2019

Back to top

© Zhaoyang Lv. · Contact ·