Enhanced security using video summarization for surveillance system using deep LSTM model with K-means clustering technique

Sharma, Shikha; Goyal, Dinesh

doi:10.47974/JDMSC-1778

Open Access Research Article

Enhanced security using video summarization for surveillance system using deep LSTM model with K-means clustering technique

^*Shikha SharmaCorresponding authorShikha.sharma@poornima.edu.inDepartment of CSE Poornima University, Jaipur Rajasthan IndiaDepartment of Computer Science and Engineering Poornima UniversityJaipur, Rajasthan, 302022, India0009-0000-8736-3936View full profile → , Dinesh Goyaldinesh8dg@gmail.com; dinesh.goyal@poornima.orgDepartment of CSE Poornima University, Jaipur Rajastshan IndiaDepartment of Creative Technologies and Product Design School of Innovative Design and Management National Taipei University of BusinessJaipur, New Taipei City, 303905, Taiwan (R.O.C.)View full profile →

* Corresponding author · click or hover a name for details

pp. 913–925Vol. 26Issue 3April 2023DOI: 10.47974/JDMSC-1778 Crossmark XML

Published Online:: 01 Apr 2023
Article type:: Research Article
Language:: EN
Article no.:: JDMSC-1778
Pages:: 913–925

Abstract

For the security of the Society, Video surveillance systems have become an important tool that can help in preventing crime in public or private establishments, leading to more safe and secure environment. With the rise of technology that can handle high volune video data and the need for continuous monitoring for security to citizens, the analysis of the video footages, during security breach attempts has become more challenging. Video summarization can play a crucial role in secure surveillance systems by optimizing the analysis time of the security breach. This research attempts to summarise the important aspects of the original video without losing its context and also optimizes the storage requirement. To achieve the same a unique KM-LSTM based technique has been developed that utilizes deep learning methods to understand links among highlight as well as non-highlight video segments through the use of pairwise distance calculations. A new Deep LSTM model has been used to extract the features from all segments of frames. These extracted features are clustered using k-means clustering. Then, such features are trained using a pairwise deep learning model to get highlight scores as well as leverage the comparative similarity among pairs of the highlight as well as non-highlight segments. At last, video summarization takes place to generate the video summary from the highlight segments only. From the results of these experiments discovering the moments of the user’s major or special interest (namely, highlights) in a video, to generating summarization of videos. The above technique can help in increasing the analysis of Video content and thus achieving a more vigilant and secure society.

Keywords

Security Video summarization Deep learning LSTM TVSum50 K-means clustering Pairwise distance calculation Surveillance systems

Subject Classifications

68M2068T07

References

[1] Wesch, M. Digital ethnography: YouTube statistics (2008). On http://mediatedcultures. net/ksudigg.
[2] Otani, M., Nakashima, Y., Rahtu, E., Heikkilä, J., & Yokoya, N. Video summarization using deep semantic features. In Asian Conference on Computer Vision, pp. 361-377 (2016, November). Springer, Cham.
[3] Balas, D. M. & Agravat, S. J., Video Summarization using CNN and Clustering Algorithm. Journal of Applied Science and Computations, pp. 1249-1254 (2018).
[4] Haq, H. B. U., Asif, M., & Ahmad, M. B., Video summarization techniques: a review. International Journal of Scientific & Technology Research, 9, 146-153 (2020).
[5] Agyeman, R., Muhammad, R., & Choi, G. S., Soccer video summarization using deep learning. In 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp. 270-273 (2019, March). IEEE.
[6] Li, Y., Zhang, H., Xue, X., Jiang, Y., & Shen, Q., Deep learning for remote sensing image classification: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(6), e1264 (2018).
[7] Aboelenien, M., & Salem, M. A. M., Reinforcement Learning Versus Generative Adversarial Networks for Video Summarization. In 2021 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), pp. 110-116 (2021, May). IEEE.
[8] Hussain, T., Muhammad, K., Ullah, A., Cao, Z., Baik, S. W., & de Albuquerque,V. H. C., Cloud-assisted multiview video summarization using CNN and bidirectional LSTM. IEEE Transactions on Industrial Informatics, 16(1), 77-86 (2019).
[9] Muhammad, K., Hussain, T., Tanveer, M., Sannino, G., & de Albuquerque, V. H. C., Cost-effective video summarization using deep CNN with hierarchical weighted fusion for IoT surveillance networks. IEEE Internet of Things Journal, 7(5), 4455-4463 (2019).
[10] Ji, Z., Xiong, K., Pang, Y., & Li, X., Video summarization with attention-based encoder–decoder networks. IEEE Transactions on Circuits and Systems for Video Technology, 30(6), 1709-1717 (2019).
[11] Khan, M. Z., Jabeen, S., ul Hassan, S., Hassan, M. A., & Khan, M. U. G., Video summarization using cnn and bidirectional lstm by utilizing scene boundary detection. In 2019 International Conference on Applied and Engineering Mathematics (ICAEM), pp. 197-202 (2019, August). IEEE.
[12] Ma, M., Mei, S., Ji, J., Wan, S., Wang, Z., & Feng, D., Exploring the influence of feature representation for dictionary selection based
video summarization. In 2017 IEEE International Conference on Image Processing (ICIP), pp. 2911-2915 (2017, September). IEEE.
[13] Song, X., Chen, K., Lei, J., Sun, L., Wang, Z., Xie, L., & Song, M. Category driven deep recurrent neural network for video summarization. In 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1-6 (2016, July). IEEE.
[14] Bansal, N., Sharma, A., & Singh, R. K. Recurrent neural network for abstractive summarization of documents. Journal of Discrete Mathematical Sciences and Cryptography, 23(1), 65-72 (2020).
[15] Sharma, G., & Goyal, D., Video tempering detection assessment in full reference mode using difference matrices. Journal of Discrete Mathematical Sciences and Cryptography, 22(4), 645-659 (2019).
[16] Sharma, S., & Saini, M. L. Analyzing the Need for Video Summarization for Online Classes Conducted During Covid-19 Lockdown. In Data, Engineering and Applications: Select Proceedings of IDEA 2021, pp. 333-342 (2022). Singapore: Springer Nature Singapore.
[17] Girdhar, P., Johri, P., & Virmani, D. Incept_LSTM: Accession for human activity concession in automatic surveillance. Journal of Discrete Mathematical Sciences and Cryptography, 25(8), 2259-2273 (2022).
[18] Deng, L., & Yu, D., Deep learning: methods and applications. Foundations and trends® in signal processing, 7(3-4), 197-387 (2014).
[19] Zhang, K., Chao, W. L., Sha, F., & Grauman, K., Video summarization with long short-term memory. In European Conference on Computer Vision, pp. 766-782, (2016, October). Springer, Cham.
[20] Zhang, L., Sun, L., Wang, W., & Tian, Y., KaaS: A standard framework proposal on video skimming. IEEE Internet Computing, 20(4), 54-59 (2016).

Views: 211Downloads: 9Citations: 2