Abstract
Current Artificial Intelligence (AI) machine learning approaches perform well with similar sensors for data collection, training, and testing. The ability to learn and analyze data from multiple sources would enhance capabilities for Artificial Intelligence (AI) systems. This paper presents a deep learning-based multi-source self-correcting approach to fuse data with different modalities. The data-level fusion approach maximizes the capability to detect unanticipated events/targets augmented with machine learning methods. The proposed Domain Adaptation for Efficient Learning Fusion (DAELF) deep neural network adapts to changes of the input distribution allowing for self-correcting of multiple source classification and fusion. When supported by a distributed computing hierarchy, the proposed DAELF scales up in neural network size and out in geographical span. The design of DAELF includes various types of data fusion, including decision-level and feature-level data fusion. The results of DAELF highlight that feature-level fusion outperforms other approaches in terms of classification accuracy for the digit data and the Aerial Image Data analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zheng, Y., Blasch, E., Liu, Z.: Multispectral Image Fusion and Colorization, SPIE Press, Bellingham (2018)
Zhang, R., Bin, J., Liu, Z., et al.: WGGAN: a wavelet-guided generative adversarial network for thermal image translation. In: Naved, M. (ed.), Generative Adversarial Networks for Image-to-Image Translation. Elsevier (2020)
Shen, D., et al.: A joint manifold leaning-based framework for heterogeneous upstream data fusion. J. Algorithms Comput. Technol. (JACT) 12(4), 311–332 (2018)
Vakil, A., Liu, J., Zulch, P., et al.: A survey of multimodal sensor fusion for passive RF and EO information integration. In: IEEE Aerospace and Electronics System Magazine (2020)
Bunyak, F., Palaniappan, K., Nath, S.K., Seetharaman, G.: Flux tensor constrained geodesic active contours with sensor fusion for persistent object tracking. J. Multimedia 2(4), 20 (2007)
Jia, B., Pham, K.D., et al.: Cooperative space object tracking using space-based optical sensors via consensus-based filters. IEEE Tr. Aerosp. Electron. Syst. 52(3), 1908–1936 (2016)
Shen, D., Sheaff, C., Guo, M., et al.: Enhanced GANs for satellite behavior discovery. In: Proc SPIE, p. 11422 (2020)
Nicolaou, M.A., Gunes, H., Pantic, M.: Audio-visual classification and fusion of spontaneous affective data in likelihood space. In: ICPR (2010)
Muller, H., Kalpathy–Cramer, J.: The image CLEF medical retrieval task at ICPR 2010 — information fusion to combine visual and textual information. In: ICPR (2010)
Li, H., Wu, X.J., Kittler, J.: Infrared and visible image fusion using a deep learning framework. In: ICPR (2018)
Blasch, E., Seetharaman, G., Palaniappan, K., Ling, H., Chen, G.: Wide-area motion imagery (WAMI) exploitation tools for enhanced situation awareness. In: IEEE Applied Imagery Pattern Recognition Workshop (2012)
Palaniappan, K., et al.: Moving object detection for vehicle tracking in wide area motion imagery using 4D filtering. In: International Conference on Pattern Recognition (ICPR) (2016)
Kozma, R.: A cognitively motivated algorithm for rapid response in emergency situations. In: IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA) (2017)
Kozma, R.: Intentional systems: review of neurodynamics, modeling, and robotics implementation. Phys. Life Rev. 5(1), 1–21 (2008)
Majumder, U., Blasch, E., Garren, D.: Deep Learning for Radar and Communications Automatic Target Recognition. Artech House, Norwood (2020)
Taigman, Y., Polyak, A., Wolf, L.: Unsupervised cross-domain image generation. arXiv preprint arXiv:1611.02200 (2016)
Goodfellow, I.J.: NIPS 2016 tutorial: Generative adversarial networks. arXiv:1701.00160v4 (2016)
Goodfellow, I.J., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (NIPS), p. 27 (2014)
Tzeng, E., Devin, C., Hoffman, J., Finn, C., Abbeel, P.: Adapting deep visuomotor representations with weak pairwise constraints. arXiv, https://arxiv.org/abs/1511.07111 (2015)
Sankaranarayanan, S., Balaji, Y., Castillo, C.D.: Generate to adapt: aligning domains using generative adversarial networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8503–8512 (2018)
Savakis, A., Nagananda, N., Kerekes, J.P., et al.: Change detection in satellite imagery with region proposal networks. Defense Syst. Inform. Anal. Center (DSIAC) J. 6(4), 23–28 Fall (2019)
Metz, L., Poole, B., Pfau, D., Sohl-Dickstein, J.: Unrolled generative adversarial networks. arXiv preprint arXiv:1611.02163 (2016)
Werbos, P.J.: Backpropagation through time: what it does and how to do it. Proc. IEEE 78(10), 1550–1560 (1990)
Ilin, R., Kozma, R., Werbos, P.J.: Beyond feedforward models trained by backpropagation: a practical training tool for a more efficient universal approximator. IEEE Trans. Neural Netw. 19(6), 929–937 (2008)
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv preprint arXiv:1701.07875 (2017)
Gulrajani, F., Ahmed, M., Arjovsky, V., Dumoulin Courville, A.C.: Improved training of Wasserstein GANs. In: Advances in Neural Information Processing Systems, pp. 5767–5777 (2017). https://arxiv.org/pdf/1704.00028.pdf.
Acknowledgements
This material is based on research sponsored by Air Force under contract FA864920P0350. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the United States Air Force.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Lu, J. et al. (2021). Deep Learning Based Domain Adaptation with Data Fusion for Aerial Image Data Analysis. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12668. Springer, Cham. https://doi.org/10.1007/978-3-030-68793-9_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-68793-9_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68792-2
Online ISBN: 978-3-030-68793-9
eBook Packages: Computer ScienceComputer Science (R0)
