TARU PUBLICATIONS
Journal of Interdisciplinary Mathematics cover

Journal of Interdisciplinary Mathematics

(Since 1998)
Hybrid ·Peer-reviewed·ISSN (Online): 2169-012X·ISSN (Print): 0972-0502
CiteScore 2025: 1.9 (Q2)

CiteScore 2025: 1.9 (Q2)

Powered by:Powered by

Monthly Journal: Publishes the methodological and theoretical role of mathematics and mathematical applications underpinning scientific research.

Issues up to 2022 co-published with and available at:Taylor & Francis Online
submissions@tarupublications.com
Taru Online First Current IssueSpecial IssuesBrowse all Issues
Journal of Interdisciplinary Mathematics
Open Access Research Article

Mathematical modelling and statistical analysis in designing deep learning-based shot boundary detection and aesthetic assessment of videos

* ,

* Corresponding author · click or hover a name for details

pp. 369–382Vol. 27Issue 2March 2024DOI: 10.47974/JIM-1857XML
Published Online:
19 Mar 2024
Article type:
Research Article
Language:
EN
Article no.:
JIM-1857
Pages:
369–382

Abstract

The main goal of this study is to create a strong deep learning-based system for finding shot boundaries and judging the quality of movies by combining mathematical models and statistical analysis. Aesthetic evaluation helps people understand visual material better, and shot border recognition is an important part of video analysis. We come up with a new approach that uses convolutional neural networks (CNNs) to improve the accuracy of shot border recognition by extracting features and using mathematical modeling. The model’s performance is improved through statistical analysis, which makes sure that changes between shots are correctly identified. Proposed method also includes judging how something looks by combining deep learning methods with mathematical models that understand how things look. This two-in-one model gives a full picture of video material, which helps with both content selection and improving the user experience. We test our method on a variety of video datasets and show that it works better than other methods. The way our deep learning approach uses mathematical models and statistical analysis together not only raises the bar for finding shot boundaries, but it also helps the new field of judging the aesthetic quality of video material. This study creates a useful and flexible tool for analyzing videos, which opens the door to better uses in fun, content creation, and multimedia. The proposed method is implemented MATLAB and the performances evaluated based on performance metrics such as such as accuracy, precision, sensitivity, specificity and recall. And the accuracy of the proposed system is obtained as 93.12%. And to assure that the proposed system functions effectively in aesthetic assessment of video a contrast is made with the conventional technique such as Hand-Crafted Feature (HCF) technique, Neural Network (NN) and Support Vector Machine (SVM).

Keywords

Shot boundary detectionAesthetic classificationAlexNet CNNQuality videoVideo processing

Subject Classifications

03D78

References

[1] Bendraou, Youssef, E. Fedwa, A. Driss, and S. Ahmed, “Shot boundary detection via adaptive low rank and svd-updating, Computer Vision and Image Understanding, 161, 20-28 (2017).
[2] L Jingwei, and R. Xie, Shot boundary detection using convolutional neural networks, In 2016 Visual Communications and Image Processing (VCIP) 1-4 (2016).
[3] Sawitchaya, Tippaya, Suchada Sitjongsataporn, Tele Tan, Masood Mehmood Khan, and Kosin Chamnongthai, “Multi-modal visual features-based video shot boundary detection, IEEE Access 5, 12563-12575 (2017).
[4] Bogusław, Cyganek and M.Woźniak, Tensor-based shot boundary detection in video streams, New Generation Computing, 35(4), 311-340 (2017).
[5] Qi, Kuang, X. Jin, Q. Zhao, and B.Zhou, Deep Multimodality Learning for UAV Video Aesthetic Quality Assessment, Transactions on Multimedia, (2019).
[6] Fang-Lue, Zhang, X. Wu, R-L. Li, J. Wang, Z-H. Zheng and S-M. Hu, Detecting and removing visual distractors for video aesthetic enhancement, IEEE Transactions on Multimedia, 20(8), 1987-1999 (2018).
[7] Xingmin, Shi, Z. Shan, and N. Zhao, Learning for an aesthetic model for estimating the traffic state in the traffic video, Neurocomputing, 181, 29-37 (2016).
[8] Ajani, S. N., Khobragade, P., Dhone, M., Ganguly, B., Shelke, N., & Parati, N., Advancements in Computing: Emerging Trends in Computational Science with Next-Generation Computing. International Journal of Intelligent Systems and Applications in Engineering, 12(7s), 546–559 (2023).
[9] A.Naci, Erdem and U.Halici, Applying Computational Aesthetics to a Video Game Application Using Machine Learning, IEEE Computer Graphics and Applications, 36(4), 23-33, (2016). 
[10] M. Bende, M. Khandelwal, D. Borgaonkar and P. Khobragade, “VISMA: A Machine Learning Approach to Image Manipulation,” 2023 6th International Conference on Information Systems and Computer Networks (ISCON), Mathura, India, pp. 1-5 (2023), doi: 10.1109/ISCON57294.2023.10112168.
[11] Yueying, Kao, K. Huang and S. Maybank, Hierarchical aesthetic quality assessment using deep convolutional neural networks, Signal Processing: Image Communication, 47, 500-510 (2016).
[12] K. Agnihotri, P. Chilbule, S. Prashant, P. Jain and P. Khobragade, “Generating Image Description Using Machine Learning Algorithms,” 2023 11th International Conference on Emerging Trends in Engineering & Technology - Signal and Information Processing (ICETET - SIP), Nagpur, India, pp. 1-6 (2023), doi: 10.1109/ICETET-SIP58143.2023.10151472.
[13] Wei-Tse, Sun, T-H. Chao, Y-H. Kuo and W.H. Hsu, Photo filter recommendation by category-aware aesthetic learning, IEEE Transactions on Multimedia, vol:19, no. 8, pp: 1870-1880 (2017).
[14] Alejandro, Hernández-García, F. Fernández-Martínez, and F.Díaz-de-María, “Comparing visual descriptors and automatic rating strategies for video aesthetics prediction, Signal Processing: Image Communication, 47, pp: 280-288. 
[15] Shete, Dhanashri, and Prashant Khobragade. “An empirical analysis of different data visualization techniques from statistical perspective.” AIP Conference Proceedings. Vol. 2839. No. 1. AIP Publishing (2023).
[16] Sadiq H., Abdulhussain, Basheera M. Mahmmod, M. I. Saripan, S.A.R. Al-Haddad, and Wissam A. Jassim, Shot boundary detection based on orthogonal polynomial, Multimedia Tools and Applications, 78(14), 20361-20382 (2019). 
[17] Jaydeb, Mondal, M.K. Kundu, S. Das and M. Chowdhury, Video shot boundary detection using multiscale geometric analysis of nsct and least squares support vector machine, Multimedia Tools and Applications, 77(7), 8139-8161 (2018). 
[18] Sawitchaya, Tippaya, S. Sitjongsataporn, T.Tan, M.M. Khan and K. Chamnongthai, Multi-modal visual features-based video shot boundary detection, IEEE Access, 5, 12563-12575 (2017). 
[19] Luming, Zhang, R. Hong, L. Nie and C. Hong, A biologically inspired automatic system for media quality assessment, IEEE Transactions on Automation Science and Engineering, 13(2), 894-902 (2015).
[20] Xin, Jin, L.Wu, X. Zhou, G. Zhao, X. Zhang, X. Li and S. Ge, Predicting aesthetic radar map using a hierarchical multi-task network, Pattern Recognition and Computer Vision, 41-50, (2018).
[21] R. Hannane, A. Elboushaki, K. Afdel, P. Naghabhushan and M. Javed, An efficient method for video shot boundary detection and keyframe extraction using SIFT-point distribution histogram, International Journal of Multimedia Information Retrieval, 5(2), 89-104 (2016). 
[22] Klaus, Schoeffmann, M.D. Fabro, T. Szkaliczki, L. Böszörmenyi and J. Keckstein, Keyframe extraction in endoscopic video, Multimedia Tools and Applications, 74(24), 11187-11206 (2015).
[23] Zhang, Yong, Y. Jiazheng, L. Hongzhe and L. Qing, GrabCut image segmentation algorithm based on structure tensor, The Journal of China Universities of Posts and Telecommunications, 24(2), 38-47 (2017). 
[24] Jingwei, Xu, L. Song and R.Xie, Shot boundary detection using convolutional neural networks, In Visual Communications and Image Processing (VCIP), 1-4 (2016).

Views: 256Downloads: 43Citations: 0