Head Image

Zhaoyang Lv  


Research Scientist, Reality Labs Research, Meta.


Previous Education:
Ph.D. in Robotics, School of Interactive Computing, Georgia Institute of Technology
M.Sc., Artificial Intelligence in Computing, Imperial College London
B.Sc., Electrical Engineering in Aeronauntics, Northwestern Polytechnical University


I am a research scientist in Reality Labs Research, Meta. I finished my Ph.D. at Georgia Tech, jointly advised by Prof. James Rehg, and Prof. Frank Dellaert. During my Ph.D., I am also fortunate to intern at Nvidia Research in the group of Jan Kautz and at Max Planck Institute with Prof. Andreas Geiger. Before I started my Ph.D., I finished my Master thesis under the supervision of Prof. Andrew Davison at Imperial College London.

I am a believer that VR/AR will become ubiquitous and fundamentally change the way we interact with world. Quote Steve Jobs' comments on GUI when he visited Xerox PARC in 1979 :
Within 10 minutes, it was obvious to me that all computers would work like this someday. You could argue about how many number of years it would take, and you could argue about who would be the winners and the losers might be, but you couldn't argue about the inevitability.
It will be a long way forward. I am super excited to work on the multidisciplinary research in this fields, explore the unknowns, contribute to make it happen at some day.

EgoLifter: Open-world 3D Segmentation for Egocentric Perception

Qiao Gu, Zhaoyang Lv, Duncan Frost, Simon Green, Julian Straub, Chris Sweeney,
Preprint, arXiv 2403.18118
Project Page
Video

LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing

Bryan Wang, Yuliang Li, Zhaoyang Lv, Haijun Xia, Yan Xu, Raj Sodhi
ACM IUI 2024, arXiv 2402.10294
Project Page
Video

Aria Everyday Activities Dataset

Zhaoyang Lv, Nicholas Charron, Pierre Moulon, Alexander Gamino, Cheng Peng, Chris Sweeney, Edward Miller, Huixuan Tang, Jeff Meissner, Jing Dong, Kiran Somasundaram, Luis Pesqueira, Mark Schwesinger, Omkar Parkhi, Qiao Gu, Renzo De Nardi, Shangyi Cheng, Steve Saarinen, Vijay Baiyya, Yuyang Zou, Richard Newcombe, Jakob Julian Engel, Xiaqing Pan, Carl Ren
Project Page
Technical Report
Project Aria Tools

AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields

Andreas Kurz, Thomas Neff, Zhaoyang Lv, Michael Zollhöfer, Markus Steinberger
European Conference on Computer Vision (ECCV) 2022, arXiv 2207.10312
Project Page
Code

1st Project Aria Tutorial & Aria Pilot Dataset & Aria Data Tools

Project lead for Project Aria Tutorial Program Team ; Aria Pilot Dataset; Aria Data Tool Team
Computer Vision and Pattern Recognition (CVPR) 2022 Tutorial Program
CVPR 2022 Tutorial Program Page
Aria Pilot Dataset
Aria Data Tools
Project Aria Paper

Neural 3D Video Synthesis from Multi-view Video

Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, Zhaoyang Lv
Computer Vision and Pattern Recognition (CVPR) 2022, arXiv 2103.02597
Oral Presentation
Project Page
Dataset

STaR: Self-supervised Tracking and Reconstruction of Rigid Objects in Motion with Neural Rendering

Wentao Yuan, Zhaoyang Lv, Tanner Schmidt, Steven Lovegrove
Computer Vision and Pattern Recognition (CVPR) 2021, arXiv 2101.01602
Project Page

SENSE: A Shared Encoder Network for Scene-flow Estimation

Huaizu Jiang, Deqing Sun, Varun Jampani, Zhaoyang Lv, Erik Learned-Miller, Jan Kautz
International Conference in Computer Vision (ICCV) 2019 , Supplementary Materials
Oral Presentation
Code

Taking a Deeper Look at the Inverse Compositional Algorithm

Zhaoyang Lv, Frank Dellaert, James M. Rehg, Andreas Geiger
Computer Vision and Pattern Recognition (CVPR) 2019, Supplementary Materials, arXiv 1812.06861
Oral Presentation, Best Paper Finalist (<1%)
Video Slides (5 mins) , Live Recorded Video Presentation (5 mins)
Code , Poster

Multi-class Classification without Multi-class Labels

Yen-Chang Hsu, Zhaoyang Lv, Joel Schlosser, Phillip Odom, Zsolt Kira
International Conference on Learning Representations (ICLR) 2019, openreview
Code

Learning to Cluster in Order to Transfer across Domains and Tasks

Yen-Chang Hsu, Zhaoyang Lv, Zsolt Kira
International Conference on Learning Representations (ICLR) 2018, arXiv:1711.10125
Code , A blog post on Machine Learning @ Gerogia Tech

Deep Image Category Discovery using a Transferred Similarity Function

Yen-Chang Hsu, Zhaoyang Lv, Zsolt Kira
arXiv:1612.01253


A Continuous Optimization Approach for Efficient and Accurate Scene Flow

Zhaoyang Lv, Chris Beall, Pablo F. Alcantarilla, Fuxin Li, Zsolt Kira, Frank Dellaert
European Conference on Computer Vision (ECCV) 2016 , arXiv 1607.07983
Project Page

KinfuSeg System Image

KinfuSeg: A Dynamic SLAM Approach Based on KinectFusion

Zhaoyang Lv
Master Thesis , Imperial College London
Video Slides
Thesis Advisor: Prof. Andrew Davison
Distinguished Thesis in Department of Computing (3 among 71), Top 5%



miniSAM: A Flexible Factor Graph Non-linear Least Squares Optimization Framework

Jing Dong (main contributor), Zhaoyang Lv
Code , arXiv
Project Website

Motion Planning and Intention Prediction for Autonomous Driving in Highway Scenarios via Graphical Model-Based Factorization

Zhaoyang Lv, Aliakbar Aghamohammadi, Amirhossein Tamjidi
US Patent App. 15/601,047

Holistic Planning with Multiple Intentions for Self-driving Cars

Zhaoyang Lv, Aliakbar Aghamohammadi
US Patent App. 15/604,437

Georgia_tech_bird_view

Large-Scale Collaborative Semantic Mapping using 3D Structure from Motion Data

I build a Dense Reconstruction of Georgia Tech with Dr. Chris Beall from Stereo Images Only for this project.
In this video, you can have a fly-through view of the reconstructed campus .




Reality Labs Research, Redmond, Sept. 2019 - Present

Research Scientist

Nvidia Research, Santa Clara, Jan. 2019 - May 2019

Research Intern
Director: Dr. Jan Kautz, Mentors: Dr. Kihwan Kim, Dr. Deqing Sun, Dr. Alejandro Troccoli

Autonomous Vision Group, Max Planck Institute for Intelligent System, Tuebingen, June 2018 - Nov. 2018

Visiting student
Advisor: Prof. Andreas Geiger

Nvidia Research, Santa Clara, May 2017 - Aug. 2017

Research Intern
Director: Dr. Jan Kautz, Mentors: Dr. Kihwan Kim, Dr. Deqing Sun, Dr. Alejandro Troccoli

Qualcomm Research, Greater San Diego, May 2016 - Aug. 2016

Research Intern
Manager: Dr. Ali Agha

Zhejiang University, Hangzhou, Dec. 2013 - July 2014

Visiting student
Mentor: Prof. Guofeng Zhang



Instructor for CS 4476 Introduction to Computer Vision, Georgia Tech, Summer 2019


Teaching assistant for CS 7643 Deep Learning, Georgia Tech, Fall 2017

Instructor: Prof. Dhruv Batra

Teaching assistant for CS 4476 / 6476 Computer Vision, Georgia Tech, Fall 2016

Instructor: Prof. James Hays

Vice President in Public Relation for RoboGrads, Georgia Tech, Fall 2016 - Spring 2017


Organizer for GT Computer Vision Reading Group, Georgia Tech, Spring 2015 - Fall 2018

I started to organize the CPL reading group as a computer vision research discussion group across Computational Perception Lab (CPL) since 2015, and now there have been an active particaption from students in computer vision research in different labs across the campus. If you are interested to join or receive future notifications, please join our google group (It's open access, you can enroll yourself with your gmail account).

Multiple reviewer services for T-PAMI, IJCV, T-MM, CVPR, ICCV, ICRA, IROS

Outstanding Reviewer, CVPR 2019



Back to top

© Zhaoyang Lv. · Contact ·