Motivation
Representation gap between aerial and ground sources: We use the bounding box with the same color to focus on the same region and highlight the differences between aerial (left) and ground (right) LiDAR scans.
- Scopes: The aerial scans cover a large region, while ground scans cover only a local area.
- Coverages: The aerial scans cover the top of the buildings, while ground scans cover more details on the ground.
- Densities: The distribution and density of the points are different because of various scan patterns, effective ranges, and fidelity of LiDARs.
- Noise Patterns: The aerial scans have larger noises, as we can see from a bird-eye view and top-down view of a corner of the building.
Network Architecture
Architecture of our network CROSSLOC3D: Our network consists of three parts: multi-grained feature selection, iterative refinement, and NetVLAD. First, we run the points through multi-grained voxelizations and several streams of sparse convolution operations with different kernel sizes. We use feature selection to choose the best field of view and keep the corresponding features F̂i for each voxelization Vi. Second, we perform an iterative refinement process to obtain a set of local features F̄. Finally, the NetVLAD module will produce the global descriptor Fg by aggregating local features F̄.
Results
Figure 7: More qualitative comparisons between CROSSLOC3D and other SOTA methods [32, 17, 18]: For each row, top: Input ground query (blue) and top 5 retrievals (ranked from left to right) from the database of aerial data (true neighbor – green, false neighbor – red). For each row, bottom: Distributions of ground query and top 25 retrievals.
Interactive Refinement Visualization
Local Point Cloud Geometry (3D)
Input Query (Ground)
QueryStep 1 Retrieval
Loading...Step 2 Retrieval
Loading...Step 3 (Final) Retrieval
Loading...Refinement Summary
Iterative refinement steps:
Active Visual Step
RGB Global Pointcloud Map (Centered 3D)
Leaflet Satellite Map (2D UTM Projection)
2D MapContinuous Trajectory Playback (umd_gps_3)
Trajectory Controls
Trajectory on Global 3D Map
Trajectory Query / Retrieved Point Clouds
Trajectory Query (Ground)
QueryRetrieved (Aerial)
--BibTeX
@InProceedings{Guan_2023_ICCV,
author = {Guan, Tianrui and Muthuselvam, Aswath and Hoover, Montana and Wang, Xijun and Liang, Jing and Sathyamoorthy, Adarsh Jagan and Conover, Damon and Manocha, Dinesh},
title = {CrossLoc3D: Aerial-Ground Cross-Source 3D Place Recognition},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
}