A Transformer-Based Method for UAV-View Geo-Localization

Abstract

The geo-localization is the task of matching a query image depicting the ground-view of an unknown location with a group of satellite-view images with GPS tags. UAV-view can mitigate the large visual differences between images from different viewpoints. CNN-based approaches have achieved great success in cross-view geo-localization, but they rely on polar transform and have limited receptive field of convolution. Therefore, we investigate the geo-localization in terms of both feature representation and viewpoint transformation, design an end-to-end multitask jointly trained network model, introduce an efficient Transformer-based lightweight structure, use the strengths of transformer related to global information modeling and explicit position information encoding, and propose an end-to-end geo-localization architecture integrating a geo-localization module and a cross-view synthesis module, called Transformer-Based for UAV-View Geo-Localization (TUL). The geo-localization module combines feature segmentation and region alignment to achieve cross-view image matching. The cross-view synthesis module synthesizes maps close to real satellite images by conditional generative adversarial networks (cGAN). Image matching and synthesis are also considered, using the network to match images over two input domains so that the network is biased to learn potential feature representations that are useful for image synthesis. Furthermore, an image augmentation strategy is proposed for data enhancement to address the problem of sample imbalance due to the difference in the number of satellite images and images from other sources in University-1652 dataset. Experiments show that our method has significant performance improvement and reaches the state-of-the-art application level.