Multimodal NLP for image captioning : Fusing text and image modalities for accurate and informative descriptions
*Manisha TiwariCorresponding authormanisha.tiwari@nmims.eduDepartment of Computer EngineeringMukesh Patel School of Technology, Management & EngineeringNMIMS Deemed to be UniversityMumbai, Maharashtra, IndiaView full profile → , Pragati Kharepragati.khare@nmims.eduDepartment of Computer EngineeringMukesh Patel School of Technology, Management & EngineeringNMIMS Deemed to be UniversityMumbai, Maharashtra, IndiaView full profile → , Ishani Sahaishani.saha@nmims.eduDepartment of Computer EngineeringMukesh Patel School of Technology, Management & EngineeringNMIMS Deemed to be UniversityMumbai, Maharashtra, IndiaView full profile → , Mahesh Malimahesh.mali@nmims.eduDepartment of Computer EngineeringMukesh Patel School of Technology, Management & EngineeringNMIMS Deemed to be UniversityMumbai, Maharashtra, IndiaView full profile →
* Corresponding author · click or hover a name for details
- Published Online:
- 08 Jun 2024
- Article type:
- Research Article
- Language:
- EN
- Article no.:
- JIOS-1626
- Pages:
- 1041–1049
Abstract
Keywords
Subject Classifications
References
[1] Sri Neha, V., et al. “A Comparative Analysis on Image Caption Generator Using Deep Learning Architecture—ResNet and VGG16.” Computational Vision and Bio-Inspired Computing: Proceedings of ICCVBIC 2021. Singapore: Springer Singapore, 209-218 (2022).
[2] du Plessis, M., Brink, W. (2022). “Improving the Performance of Image Captioning Models Trained on Small Datasets. “ In: Jembere, E., Gerber, A.J., Viriri, S., Pillay, A. (eds) Artificial Intelligence Research. SACAIR. Communications in Computer and Information Science, vol 1551 (2021).
[3] Cornia, M., Baraldi, L., Tavakoli, H.R. et al. “A unified cycle-consistent neural model for text and image retrieval.” Multimed Tools Appl 79, 25697–25721 (2020).
[4] Anitha Kumari, K., et al. “Automated image captioning for Flickr8K Dataset.” Proceedings of International Conference on Artificial Intelligence, Smart Grid and Smart City Applications: AISGSC 2019. Springer International Publishing (2020).
[5] Singh, Vijander, et al. “Prediction of COVID-19 corona virus pandemic based on time series data using Support Vector Machine.” Journal of Discrete Mathematical Sciences and Cryptography 23.8 : 1583-1597 (2020).
[6] Nan, Lin., Shuo, Guo., Lipeng, Xie. “TB-Transformer: Integrating Mouse Trace with Object Bounding-box for Image Caption.” undefined (2022). doi: 10.1109/HPCC-DSS-SmartCity-DependSys57074.2022.00133
[7] Bhatnagar, Vaibhav, et al. “Descriptive analysis of COVID-19 patients in the context of India.” Journal of Interdisciplinary Mathematics 24.3 : 489-504 (2021).
[8] Zhu, Wanrong et al. “Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved With Text.” ArXiv abs/2304.06939 (2023): n. pag.
[9] Pourkeshavarz, M., Nabavi, S., Moghaddam, M.E. et al. Stacked cross-modal feature consolidation attention networks for image captioning. Multimed Tools Appl 83, 12209–12233 (2024).
[10] F. Chen, J. Tang, S. Li, X. Li and T. Wang, “Looking Back and Forward: Enhancing Image Captioning with Global Semantic Guidance,” 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China, pp. 1-8, (2021).
[11] Elhagry, Ahmed and Karima Kadaoui. “A Thorough Review on Recent Deep Learning Methodologies for Image Captioning.” ArXiv abs/2107.13114 (2021): n. pag.
[12] Wang, Jiuniu et al. “Group-based Distinctive Image Captioning with Memory Attention.” Proceedings of the 29th ACM International Conference on Multimedia (2021): n. pag.
[13] Z. Deng, B. Zhou, P. He, J. Huang, O. Alfarraj et al., «A position-aware transformer for image captioning,» Computers, Materials & Continua, vol. 70, no.1, pp. 2065–2081 (2022).
[14] Hu,Anwen et al. “Question-controlled Text-aware Image Captionin.” Proceedings of the 29th ACM International Conference on Multimedia (2021).
