Publications
Offboard 3D Object Detection from Point Cloud Sequences, CVPR 2021 While current 3D object recognition research mostly focuses on the real-time, onboard scenario, there are many offboard use cases of perception that are largely under-explored, such as using machines to automatically generate high-quality 3D labels. In this paper, we propose a novel offboard 3D object detection pipeline using point cloud sequence data. This work was used to auto label the Waymo Motion Dataset. Feel free to check it out! paper / blog post / bibtex |
|
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding, Spotlight, ECCV 2020 Local contrastive learning for 3D representation learning. The unsupervisely learned representation can generalize across tasks and helps improve severl high-level semantic understanding problems rangining from semgentation to detection on six different datasets. paper / code / bibtex |
|
ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votes, CVPR 2020 Extensions of VoteNet to leverage RGB images. By lifting 2D image votes to 3D, RGB images can provide strong geometric cues for 3D object localization and pose estimation, while their textures and colors provide semantic cues. A special multi-tower training scheme also makes the 2D-3D feature fusion more effective. paper / bibtex / code |
|
Deep Hough Voting for 3D Object Detection in Point Clouds, Oral Presentation, ICCV 2019 Best Paper Award Nomination (one of the seven among 1,075 accepted papers) [link] We show a revive of generalize Hough voting in the era of deep learning for the task of 3D object detection in point clouds. Our voting-based detection network (VoteNet) is both fast and top performing. paper / bibtex / code / talk |
|
KPConv: Flexible and Deformable Convolution for Point Clouds, ICCV 2019 Proposed a point centric way for deep learning on 3D point clouds with kernel point convolution (KPConv) where we define a convolution kernel as a set of spatially localized and deformable points. paper / bibtex / code |
|
Generating 3D Adversarial Point Clouds, CVPR 2019 Proposed several novel algorithms to craft adversarial point clouds against 3D deep learning models with adversarial points perturbation and adversarial points generation. paper / bibtex / code |
|
FlowNet3D: Learning Scene Flow in 3D Point Clouds, CVPR 2019 Proposed a novel deep neural network that learns scene flow from point clouds in an end-to-end fashion. paper / bibtex / code |
|
Exploring Hidden Dimensions in Parallelizing Convolutional Neural Networks, ICML 2018 We studied how to parallelize training of deep convolutional networks beyond simple data or model parallelism. Proposed a layer-wise parallelism that allows each layer in a network to use an individual parallelization strategy. paper / bibtex |
|
Frustum PointNets for 3D Object Detection from RGB-D Data, CVPR 2018 Proposed a novel framework for 3D object detection with image region proposals (lifted to 3D frustums) and PointNets. Our method is simple, efficient and effective, ranking at first place for KITTI 3D object detection benchmark on all categories (11/27/2017). paper / bibtex / code / website |
|
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space, NIPS 2017 Proposed a hierarchical neural network on point sets that captures local context. Compared with PointNet, PointNet++ achieves better performance and generalizability in complex scenes and is able to deal with non-uniform sampling density. paper / bibtex / code / website / poster |
|
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, Oral Presentation, CVPR 2017 Proposed novel neural networks to directly consume an unordered point cloud as input, without converting to other 3D representations such as voxel grids first. Rich theoretical and empirical analyses are provided. paper / bibtex / code / website / presentation video |
|
Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis, Spotlight Presentation, CVPR 2017 A data-driven approach to complete partial 3D shapes through a combination of volumetric deep neural networks and 3D shape synthesis. paper / bibtex / website (code & data available) |
|
Volumetric and Multi-View CNNs for Object Classification on 3D Data, Spotlight Presentation, CVPR 2016 Novel architectures for 3D CNNs that take volumetric or multi-view representations as input. paper / bibtex / code / website / supp / presentation video |
|
FPNN: Field Probing Neural Networks for 3D Data, NIPS 2016 A very efficient 3D deep learning method for volumetric data processing that takes advantage of data sparsity in 3D fields. paper / bibtex / code / website |
|
Joint Embeddings of Shapes and Images via CNN Image Purification, SIGGRAPH Asia 2015 Cross-modality learning of 3D shapes and 2D images by neural networks. A joint embedding space that is sensitive to 3D geometry difference but agnostic to other nuisances is constructed. paper / bibtex / code / website / live demo |
|
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views, Oral Presentation, ICCV 2015 Pioneering work that shows large-scale synthetic data rendered from virtual world may greatly benefit deep learning to work in real world. Deliver a state-of-the-art viewpoint estimator. paper / bibtex / code / website / presentation video |