Multimodal model for the Spanish sentiment analysis in a tourism domain | Social Network Analysis and Mining Skip to main content
Springer Nature Link
Log in

Multimodal model for the Spanish sentiment analysis in a tourism domain

  • Original Article
  • Published:
Social Network Analysis and Mining Aims and scope Submit manuscript

Abstract

The problem of sentiment analysis of tourism data focuses on the analysis of the multimodal characteristics of the data generated digitally by tourists on each platform or social network. Generally, their opinions have multimodal characteristics, since they combine text, images or numbers (ratings), which represents an important challenge in sentiment analysis that requires new models or multimodal data classification techniques. This work proposes a multimodal sentiment analysis model for data in Spanish in the tourism domain composed of four main phases (extraction, classification, fusion, visualization), and a transversal phase to evaluate the quality of the multimodal sentiment analysis process. Thus, the multimodal sentiment analysis model integrates a data quality model to improve multimodal sentiment analysis tasks, but in addition, the linguistic resource "SenticNet 5" is adapted to Spanish. The model was validated by applying various classification metrics, and the classification results were compared to a manually labeled dataset (TASS) using two machine learning classification algorithms. The first was Random Forest, where the manually labeled dataset has a 50% F1 score compared to the adapted SenticNet automatically generated dataset, which has a 71% F1 score measure and a 70% accuracy. The classification generated by SenticNet is 21% higher than that of the TASS data set. The second algorithm applied was Support Vector Machine (SVM), which classified the SenticNet-generated dataset with an F1 score of 72% versus the manually created dataset with 57.7% (14.3% more effective). In the fusion tests of the multimodal sentiment inputs, the accuracy results for text were 65%, for images 33%, and the fusion of both was 71%. In general, it was identified that the opinions made by users composed of text in Spanish and images improve polarity identification if an independent classification is carried out, and then apply a polarity fusion process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. TASS: Workshop on Semantic Analysis at SEPLN.

References

  • Akhtar N, Zubair N, Kumar A, Ahmad T (2017) Aspect based sentiment oriented summarization of hotel reviews. Procedia Comput Sci 115:563–571

    Article  Google Scholar 

  • Álvarez-Carmona MÁ, Aranda R, Guerrero-Rodríguez R, Rodríguez-González AY, López-Monroy AP (2022) A combination of sentiment analysis systems for the study of online travel reviews: many heads are better than one. Computación y Sistemas, 26(2)

  • Athuraliya B, Farook C (2018) “revyew” hotel maintenance issue classifier and analyzer using machine learning and natural language processing. In: 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), pp 274–280. IEEE

  • Baecchi C, Uricchio T, Bertini M, Del Bimbo A (2016) A multimodal feature learning approach for sentiment analysis of social network multimedia. Multimed Tools Appl 75(5):2507–2525

    Article  Google Scholar 

  • Bordoloi M, Biswas SK (2023) Sentiment analysis: a survey on design framework, applications and future scopes. Artific Intell Rev pp 1–56

  • Cai L, Zhu Y (2015) The challenges of data quality and data quality assessment in the big data era. Data Sci J p 14

  • Cambria E, Hussain A (2012) Sentic computing. Marketing 59(2):557–577

    Google Scholar 

  • Cambria E, Livingstone A, Hussain A (2012) The hourglass of emotions. In: Cognitive Behavioral systems, pp 144–157. Springer

  • Cambria E, Poria S, Hazarika D, Kwok K (2018) Senticnet 5: Discovering conceptual primitives for sentiment analysis by means of context embeddings. In: Proceedings of the AAAI conference on artificial intelligence

  • Chaturvedi I, Satapathy R, Cavallari S, Cambria E (2019) Fuzzy commonsense reasoning for multimodal sentiment analysis. Patt Recognit Lett 125:264–270

    Article  ADS  Google Scholar 

  • Chiu C, Chiu N-H, Sung R-J, Hsieh P-Y (2015) Opinion mining of hotel customer-generated contents in chinese weblogs. Curr Issues Tour 18(5):477–495

    Article  Google Scholar 

  • Cordero J, Aguilar J, Aguilar K, Chávez D, Puerto E (2020) Recognition of the driving style in vehicle drivers. Sensors, 20(9)

  • Díaz-Galiano M, Martínez-Cámara E, García-Cumbreras M, García-Vega M, Villena-Román J (2018) The democratization of deep learning in tass 2017. Procesamiento de Lenguaje Nat 60

  • Farisi AA, Sibaroni Y, Al Faraby S (2019) Sentiment analysis on hotel reviews using multinomial naïve bayes classifier. In: Journal of Physics: Conference Series

  • Flores-Ruiz D, Elizondo-Salto A, MdlO Barroso-González (2021) Using social media in tourist sentiment analysis: a case study of andalusia during the covid-19 pandemic. Sustainability 13(7):3836

    Article  CAS  Google Scholar 

  • Gandhi A, Adhvaryu K, Poria S, Cambria E, Hussain A (2022) Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions. Inf Fusion

  • Giancristofaro GT, Panangadan A (2016) Predicting sentiment toward transportation in social media using visual and textual features. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), pp 2113–2118. IEEE

  • Huang F, Zhang X, Zhao Z, Xu J, Li Z (2019) Image-text sentiment analysis via deep multimodal attentive fusion. Knowl Based Syst 167:26–37

    Article  Google Scholar 

  • Kaehler A, Bradski G (2016) Learning OpenCV 3: computer vision in C++ with the OpenCV library. " O’Reilly Media, Inc."

  • Kumar A, Garg G (2019) Sentiment analysis of multimodal twitter data. Multimed Tools Appl 78(17):24103–24119

    Article  Google Scholar 

  • Kumar A, Srinivasan K, Cheng W-H, Zomaya AY (2020) Hybrid context enriched deep learning model for fine-grained sentiment analysis in textual and visual semiotic modality social data. Inf Proc Manag 57(1):102141

    Article  Google Scholar 

  • Li Q, Gkoumas D, Lioma C, Melucci M (2021) Quantum-inspired multimodal fusion for video sentiment analysis. Inf Fusion 65:58–71

    Article  Google Scholar 

  • Lucas L, Tomás D, Garcia-Rodriguez J (2022) Sentiment analysis and image classification in social networks with zero-shot deep learning: applications in tourism. In: 16th International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO 2021), pp 419–428. Springer

  • Martínez-Cámara E, Martín-Valdivia MT, Urena-Lopez LA, Mitkov R (2015) Polarity classification for spanish tweets using the cost corpus. J Inf Sci 41(3):263–272

    Article  Google Scholar 

  • Minsky M (2007) The emotion machine: Commonsense thinking, artificial intelligence, and the future of the human mind. Simon and Schuster

    Google Scholar 

  • Molina-González MD, Martínez-Cámara E, Martín-Valdivia M-T, Perea-Ortega JM (2013) Semantic orientation for polarity classification in spanish reviews. Expert Syst Appl 40(18):7250–7257

    Article  Google Scholar 

  • Moreo A, Romero M, Castro J, Zurita JM (2012) Lexicon-based comments-oriented news sentiment analyzer system. Expert Syst Appl 39(10):9166–9180

    Article  Google Scholar 

  • Poria S, Chaturvedi I, Cambria E, Hussain A (2016) Convolutional mkl based multimodal emotion recognition and sentiment analysis. In: 2016 IEEE 16th international conference on data mining (ICDM), pp 439–448. IEEE

  • Salazar C, Aguilar J, Monsalve-Pulido J, Montoya E (2021) Affective recommender systems in the educational field a systematic literature review. Comput Sci Rev 40:100377

    Article  Google Scholar 

  • Susanto Y, Livingstone AG, Ng BC, Cambria E (2020) The hourglass model revisited. IEEE Intell Syst 35(5):96–102

    Article  Google Scholar 

  • Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: Closing the gap to human-level performance in face verification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1701–1708

  • Takahashi F, Kawabata Y (2018) The association between colors and emotions for emotional words and facial expressions. Color Res Appl 43(2):247–257

    Article  Google Scholar 

  • Taleb I, Dssouli R, Serhani MA (2015) Big data pre-processing: A quality framework. In: 2015 IEEE international congress on big data, pp 191–198. IEEE

  • Tao Y, Zhang F, Shi C, Chen Y (2019) Social media data-based sentiment analysis of tourists’ air quality perceptions. Sustainability 11(18):5070

    Article  Google Scholar 

  • Tsytsarau M, Palpanas T (2012) Survey on mining subjective data on the web. Data Min Knowl Discov 24(3):478–514

    Article  Google Scholar 

  • Viñán-Ludeña MS, de Campos LM (2021) Analyzing tourist data on twitter: a case study in the province of granada at spain. J Hospital Tour Insights

  • Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, volume 1, pp I–I. IEEE

  • Xu Q, Chang V, Jayne C (2022) A systematic review of social media-based sentiment analysis: Emerging trends and challenges. Decis Anal J 3:100073

    Article  Google Scholar 

  • Zhang N, Paluri M, Taigman Y, Fergus R, Bourdev L (2015) Beyond frontal faces: Improving person recognition using multiple cues. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4804–4813

  • Zhao Z, Zhu H, Xue Z, Liu Z, Tian J, Chua MCH, Liu M (2019) An image-text consistency driven multimodal sentiment analysis approach for social media. Inf Proc Manag 56(6):102097

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. GIMI, Universidad Pedagógica y Tecnológica de Colombia, Tunja, Colombia

    Julian Monsalve-Pulido

  2. Pontificia Universidad Javeriana, Bogotá, Colombia

    Carlos Alberto Parra

  3. CEMISID, Universidad de Los Andes, Mérida, Venezuela

    Jose Aguilar

  4. CIDITIC, Universidad EAFIT, Medellín, Colombia

    Jose Aguilar

  5. IMDEA Networks Institute, Madrid, Spain

    Jose Aguilar

Authors
  1. Julian Monsalve-Pulido
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Alberto Parra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jose Aguilar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julian Monsalve-Pulido.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Monsalve-Pulido, J., Parra, C.A. & Aguilar, J. Multimodal model for the Spanish sentiment analysis in a tourism domain. Soc. Netw. Anal. Min. 14, 46 (2024). https://doi.org/10.1007/s13278-024-01202-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13278-024-01202-3

Keywords

Search

Navigation