{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,7]],"date-time":"2024-09-07T06:39:56Z","timestamp":1725691196785},"reference-count":53,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,2,27]],"date-time":"2022-02-27T00:00:00Z","timestamp":1645920000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61901122"],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007219","name":"Natural Science Foundation of Shanghai","doi-asserted-by":"publisher","award":["20ZR1406300"],"id":[{"id":"10.13039\/100007219","id-type":"DOI","asserted-by":"publisher"}]},{"name":"China High-resolution Earth Observation System (CHEOS)-Aerial Observation System Project","award":["30-H30C01-9004-19\/21"]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"As an all-weather and all-day remote sensing image data source, SAR (Synthetic Aperture Radar) images have been widely applied, and their registration accuracy has a direct impact on the downstream task effectiveness. The existing registration algorithms mainly focus on small sub-images, and there is a lack of available accurate matching methods for large-size images. This paper proposes a high-precision, rapid, large-size SAR image dense-matching method. The method mainly includes four steps: down-sampling image pre-registration, sub-image acquisition, dense matching, and the transformation solution. First, the ORB (Oriented FAST and Rotated BRIEF) operator and the GMS (Grid-based Motion Statistics) method are combined to perform rough matching in the semantically rich down-sampled image. In addition, according to the feature point pairs, a group of clustering centers and corresponding images are obtained. Subsequently, a deep learning method based on Transformers is used to register images under weak texture conditions. Finally, the global transformation relationship can be obtained through RANSAC (Random Sample Consensus). Compared with the SOTA algorithm, our method\u2019s correct matching point numbers are increased by more than 2.47 times, and the root mean squared error (RMSE) is reduced by more than 4.16%. The experimental results demonstrate that our proposed method is efficient and accurate, which provides a new idea for SAR image registration.<\/jats:p>","DOI":"10.3390\/rs14051175","type":"journal-article","created":{"date-parts":[[2022,2,28]],"date-time":"2022-02-28T01:48:33Z","timestamp":1646012913000},"page":"1175","source":"Crossref","is-referenced-by-count":18,"title":["A Transformer-Based Coarse-to-Fine Wide-Swath SAR Image Registration Method under Weak Texture Conditions"],"prefix":"10.3390","volume":"14","author":[{"given":"Yibo","family":"Fan","sequence":"first","affiliation":[{"name":"Key Laboratory for Information Science of Electromagnetic Waves (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai 200433, China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-2378-9126","authenticated-orcid":false,"given":"Feng","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory for Information Science of Electromagnetic Waves (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai 200433, China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-1912-7143","authenticated-orcid":false,"given":"Haipeng","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory for Information Science of Electromagnetic Waves (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai 200433, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.inffus.2020.01.003","article-title":"Pixel level fusion techniques for SAR and optical images: A review","volume":"59","author":"Kulkarni","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1080\/2150704X.2019.1594430","article-title":"A novel change detection method combined with registration for SAR images","volume":"10","author":"Song","year":"2019","journal-title":"Remote Sens. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Tapete, D., and Cigna, F. (2019). Detection of Archaeological Looting from Space: Methods, Achievements and Challenges. Remote Sens., 11.","DOI":"10.3390\/rs11202389"},{"key":"ref_4","unstructured":"Suri, S., Schwind, P., Reinartz, P., and Uhl, J. (2009, January 9\u201313). Combining mutual information and scale invariant feature transform for fast and robust multisensor SAR image registration. Proceedings of the American Society of Photogrammetry and Remote Sensing (ASPRS) Annual Conference, Baltimore, MD, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2235","DOI":"10.1109\/36.957286","article-title":"A Fast Direct Fourier-Based Algorithm for Sub-pixel Registration of Images","volume":"39","author":"Stone","year":"2001","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Wang, F., and You, H. (2018). An Automatic and Novel SAR Image Registration Algorithm: A Case Study of the Chinese GF-3 Satellite. Sensors, 18.","DOI":"10.3390\/s18020672"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4132","DOI":"10.1109\/TGRS.2019.2961245","article-title":"Subpixel SAR image registration through parabolic interpolation of the 2-D cross correlation","volume":"58","author":"Pallotta","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_9","unstructured":"Yan, K., and Sukthankar, R. (July, January 27). PCA-SIFT: A more distinctive representation for local image descriptors. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, Washington, DC, USA."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1109\/TGRS.2014.2323552","article-title":"SAR-SIFT: A SIFT-like algorithm for SAR images","volume":"53","author":"Dellinger","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_11","first-page":"5207612","article-title":"KAZE-SAR: SAR Image Registration Using KAZE Detector and Modified SURF Descriptor for Tackling Speckle Noise","volume":"60","author":"Pourfard","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Alcantarilla, P.F., Bartoli, A., and Davison, A.J. (2012, January 7\u201313). KAZE features. Proceedings of the European Conference on Computer Vision, Florence, Italy.","DOI":"10.1007\/978-3-642-33783-3_16"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Jiao, N., Wang, F., and You, H. (2021). A Robust Two-Stage Registration Algorithm for Large Optical and SAR Images. IEEE Trans. Geosci. Remote Sens., in press.","DOI":"10.1109\/TGRS.2021.3133863"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4806","DOI":"10.1109\/TGRS.2016.2551720","article-title":"Target Classification using the Deep Convolutional Networks for SAR Images","volume":"54","author":"Chen","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"140303","DOI":"10.1007\/s11432-019-2772-5","article-title":"FUSAR-Ship: Building a high-resolution SAR-AIS matchup dataset of Gaofen-3 for ship detection and recognition","volume":"63","author":"Hou","year":"2020","journal-title":"Sci. China Inf. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.isprsjprs.2017.12.012","article-title":"A deep learning framework for remote sensing image registration","volume":"145","author":"Wang","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"7365","DOI":"10.1109\/TGRS.2019.2913095","article-title":"Saliency-Guided Deep Neural Networks for SAR Image Change Detection","volume":"57","author":"Geng","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"He, H., Chen, M., Chen, T., and Li, D. (2018). Matching of remote sensing images with complex background variations via Siamese convolutional neural network. Remote Sens., 10.","DOI":"10.3390\/rs10020355"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zheng, Y., Sui, X., Jiang, Y., Che, T., Zhang, S., Yang, J., and Li, H. (2021). SymReg-GAN: Symmetric Image Registration with Generative Adversarial Networks. IEEE Trans. Pattern Anal. Mach. Intell., in press.","DOI":"10.1109\/TPAMI.2021.3083543"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, Z., Zhang, H., and Huang, Y. (2021). A Rotation-Invariant Optical and SAR Image Registration Algorithm Based on Deep and Gaussian Features. Remote Sens., 13.","DOI":"10.3390\/rs13132628"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Mao, S., Yang, J., Gou, S., Jiao, L., Xiong, T., and Xiong, L. (2021). Multi-Scale Fused SAR Image Registration Based on Deep Forest. Remote Sens., 13.","DOI":"10.3390\/rs13112227"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Luo, X., Lai, G., Wang, X., Jin, Y., He, X., Xu, W., and Hou, W. (2021). UAV Remote Sensing Image Automatic Registration Based on Deep Residual Features. Remote Sens., 13.","DOI":"10.3390\/rs13183605"},{"key":"ref_24","first-page":"189","article-title":"Deep learning algorithm for feature matching of cross modality remote sensing images","volume":"50","author":"Lan","year":"2021","journal-title":"Acta Geodaetica et Cartographica Sinica."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G.R. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Bian, J., Lin, W.Y., Matsushita, Y., Yeung, S.K., Nguyen, T.D., and Cheng, M.M. (2017, January 25\u201330). Gms: Grid-based motion statistics for fast, ultra-robust feature correspondence. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.302"},{"key":"ref_27","unstructured":"Arthur, D., and Vassilvitskii, S. (2007, January 7\u20139). K-means++: The Advantages of Careful Seeding. Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, New Orleans, LA, USA."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus\u2014A paradigm for model-fitting with applications to image-analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_29","unstructured":"Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems 25, Curran Associates, Inc."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"7177","DOI":"10.1109\/TGRS.2017.2743222","article-title":"Complex-valued convolutional neural network and its application in polarimetric SAR image classification","volume":"55","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_31","first-page":"1","article-title":"Scattering Enhanced attention pyramid network for aircraft detection in SAR images","volume":"99","author":"Guo","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1016\/j.patcog.2016.11.015","article-title":"SAR Image segmentation based on convolutional-wavelet neural network and markov random field","volume":"64","author":"Duan","year":"2016","journal-title":"Pattern Recognit."},{"key":"ref_33","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wang, Z., Zhao, J., Zhang, R., Li, Z., Lin, Q., and Wang, X. (2022). UATNet: U-Shape Attention-Based Transformer Net for Meteorological Satellite Cloud Recognition. Remote Sens., 14.","DOI":"10.3390\/rs14010104"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zhao, C., Wang, J., Su, N., Yan, Y., and Xing, X. (2022). Low Contrast Infrared Target Detection Method Based on Residual Thermal Backbone Network and Weighting Loss Function. Remote Sens., 14.","DOI":"10.3390\/rs14010177"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Xu, X., Feng, Z., Cao, C., Li, M., Wu, J., Wu, Z., Shang, Y., and Ye, S. (2021). An Improved Swin Transformer-Based Model for Remote Sensing Object Detection and Instance Segmentation. Remote Sens., 13.","DOI":"10.3390\/rs13234779"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Sun, J., Shen, Z., Wang, Y., Bao, H., and Zhou, X. (2021, January 11\u201317). LoFTR: Detector-free local feature matching with transformers. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Montreal, QC, Canada.","DOI":"10.1109\/CVPR46437.2021.00881"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Bay, H., Tuytelaars, T., and Van Gool, L. (2006, January 7\u201313). SURF: Speeded Up Robust Features. Proceedings of the 9th European Conference on Computer Vision, Graz, Austria.","DOI":"10.1007\/11744023_32"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Rosten, E., and Drummond, T. (2006, January 7\u201313). Machine learning for high-speed corner detection. Proceedings of the 9th European Conference on Computer Vision, Graz, Austria.","DOI":"10.1007\/11744023_34"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Calonder, M., Lepetit, V., Strecha, C., and Fua, P. (2010, January 5\u201311). BRIEF: Binary Robust Independent Elementary Features. Proceedings of the 11th European Conference on Computer Vision, Heraklion, Greece.","DOI":"10.1007\/978-3-642-15561-1_56"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Sun, K., Xiao, B., Liu, D., and Wang, J. (2019, January 16\u201320). Deep high-resolution representation learning for human pose estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00584"},{"key":"ref_42","unstructured":"Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., and Weller, A. (May, January 26). Rethinking attention with performers. Proceedings of the International Conference on Learning Representations, Virtual."},{"key":"ref_43","unstructured":"Simonyan, K., and Zisserman, A. (2014, January 14\u201316). Very deep convolutional networks for large-scale image recognition. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_45","unstructured":"Tan, M., and Le, Q. (2019, January 9\u201315). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (July, January 21). Feature Pyramid Networks for Object Detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Li, Z., and Snavely, N. (2018, January 18\u201323). MegaDepth: Learning Single-View Depth Prediction from Internet Photos. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00218"},{"key":"ref_48","unstructured":"Mishchuk, A., Mishkin, D., Radenovic, F., and Matas, J. (2017). Working hard to know your neighbor\u2019s margins: Local descriptor learning loss. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Balntas, V., Riba, E., Ponsa, D., and Mikolajczyk, K. (2016, January 19\u201322). Learning Local Feature Descriptors with Triplets and Shallow Convolutional Neural Networks. Proceedings of the British Machine Vision Association (BMVC) 2016, York, UK.","DOI":"10.5244\/C.30.119"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Tian, Y., Yu, X., Fan, B., Wu, F., Heijnen, H., and Balntas, V. (2019, January 16\u201320). SOSNet: Second Order Similarity Regularization for Local Descriptor Learning. Proceedings of the Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01127"},{"key":"ref_51","unstructured":"Shi, J., and Tomasi, C. (1994, January 21\u201323). Good features to track. Proceedings of the 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Pu, W. (2022). SAE-Net: A Deep Neural Network for SAR Autofocus. IEEE Trans. Geosci. Remote Sens., in press.","DOI":"10.1109\/TGRS.2021.3139914"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"772","DOI":"10.1109\/LGRS.2009.2025059","article-title":"Unsupervised change detection in satellite images using principal component analysis and k-means clustering","volume":"6","author":"Celik","year":"2009","journal-title":"IEEE Geosci. Remote Sens. Lett."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/5\/1175\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,26]],"date-time":"2024-07-26T20:22:19Z","timestamp":1722025339000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/5\/1175"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,27]]},"references-count":53,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["rs14051175"],"URL":"https:\/\/doi.org\/10.3390\/rs14051175","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,27]]}}}