FOREX Press I. J. of Electrical & Electronics Research
Support Open Access
  • Home/
  • IJEER-11ngwcn04

Research Article |

Adaptive Video Coding Framework with Spatial-Temporal Fusion for Optimized Streaming in Next-Generation Networks

Author(s): Pranob Kumar Charles, Habibulla Khan* and K S Rao,

Published In : International Journal of Electrical and Electronics Research (IJEER)        Volume 11, Special Issue on NGWCN

Publisher : FOREX Publication

Published : 30 December 2023

e-ISSN : 2347-470X

Page(s) : 20-24

DOI: https://doi.org/10.37391/ijeer.11ngwcn04

Abstract

Predicting future frames and improving inter-frame prediction are ongoing challenges in the field of video streaming. By creating a novel framework called STreamNet (Spatial-Temporal Video Coding), fusing bidirectional long short-term memory with temporal convolutional networks, this work aims to address the issue at hand. The development of STreamNet, which combines spatial hierarchies with local and global temporal dependencies in a seamless manner, along with sophisticated preprocessing, attention mechanisms, residual learning, and effective compression techniques, is the main contribution. Significantly, STreamNet claims to provide improved video coding quality and efficiency, making it suitable for next-generation networks. STreamNet has the potential to provide reliable and optimal streaming in high-demand network environments, as shown by preliminary tests that show a performance advantage over existing methods.

Keywords: Video Coding, Temporal Convolutional Networks, Next-Generation Networks, Spatial-Temporal Fusion, Optimized Streaming, STreamNet, BiLSTM.




Pranob Kumar Charles, Research Scholar, Dept of ECE, JNTUH, India; Email: pranob2005@gmail.com

Habibulla Khan*, Professor - Dept of ECE, K L University India; Email: habibulla@kluniversity.in

K S Rao, Principal - Jyothishmathi Institute of Technology and Science (JITS), India; Email: drksraodir@gmail.com

    [1] Zhang, Yun, Sam Kwong, Xu Wang, Hui Yuan, Zhaoqing Pan, and Long Xu. (2015), "Machine learning-based coding unit depth decisions for flexible complexity allocation in high efficiency video coding." IEEE Transactions on Image Processing 24, no. 7 (2015): 2225-2238.
    [2] Puri, Saurabh. (2017), "Learning, selection and coding of new block transforms in and for the optimization loop of video coders." PhD diss., Nantes, 2017.
    [3] Topiwala, Pankaj, Madhu Krishnan, and Wei Dai. (2018), "Deep learning techniques in video coding and quality analysis." In Applications of Digital Image Processing XLI, vol. 10752, pp. 353-367. SPIE, 2018.
    [4] Wiegand, Thomas, Gary J. Sullivan, et al., (2003), "Overview of the H. 264/AVC video coding standard." IEEE Transactions on circuits and systems for video technology 13, no. 7 560-576.
    [5] Wang, Tingting, Mingjin Chen, and Hongyang Chao. (2017), "A novel deep learning-based method of improving coding efficiency from the decoder-end for HEVC." In 2017 data compression conference (DCC), pp. 410-419. IEEE, 2017.
    [6] Zhang, Yun, Sam Kwong, and Shiqi Wang. (2020), "Machine learning based video coding optimizations: A survey." Information Sciences 506 (2020): 395-423.
    [7] Liu, Dong, Yue Li, Jianping Lin, Houqiang Li, and Feng Wu. (2020), "Deep learning-based video coding: A review and a case study." ACM Computing Surveys (CSUR) 53, no. 1 (2020): 1-35.
    [8] Dharwadkar, Shri N., and Nabegha Masood. (2007), "Next Generation Network." In 2007 IEEE International Symposium on Consumer Electronics, pp. 1-4. IEEE, 2007.
    [9] Modarressi, Abdi R., and Seshadri Mohan. (2000), "Control and management in next-generation networks: challenges and opportunities." IEEE Communications Magazine 38, no. 10 (2000): 94-102.
    [10] Ding, Dandan, Lingyi Kong, Guangyao Chen, Zoe Liu, and Yong Fang. (2019), "A switchable deep learning approach for in-loop filtering in video coding." IEEE Transactions on Circuits and Systems for Video Technology 30, no. 7 (2019): 1871-1887.
    [11] Liu, Chao, Heming Sun, Jiro Katto, Xiaoyang Zeng, and Yibo Fan. (2020), "A learning-based low complexity in-loop filter for video coding." In 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1-6. IEEE, 2020.
    [12] Murn, Luka, Saverio Blasi, Alan F. Smeaton, and Marta Mrak. (2021), "Improved CNN-based learning of interpolation filters for low-complexity inter prediction in video coding." IEEE Open Journal of Signal Processing 2 (2021): 453-465.
    [13] Wang, Taiyu, Fan Li, Xiaoya Qiao, and Pamela C. Cosman. (2020), "Low-Complexity Error Resilient HEVC Video Coding: A Deep Learning Approach." IEEE Transactions on Image Processing 30 (2020): 1245-1260.
    [14] Çetinkaya, Ekrem. (2021), "Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming." In Proceedings of the 12th ACM Multimedia Systems Conference, pp. 418-422. 2021.
    [15] Ho, Yung-Han, Yun Liang, Chia-Hao Kao, and Wen-Hsiao Peng. (2022), "Action-Constrained Reinforcement Learning for Frame-Level Bit Allocation in HEVC/H. 265 through Frank-Wolfe Policy Optimization." arXiv preprint arXiv:2203.05127 (2022).
    [16] Bouaafia, Soulef, Randa Khemiri, Seifeddine Messaoud, Olfa Ben Ahmed, and Fatma Ezahra Sayadi. (2022), "Deep learning-based video quality enhancement for the new versatile video coding." Neural Computing and Applications 34, no. 17 (2022): 14135-14149.
    [17] Chen, Mei-Juan, Cheng-An Lee, Yu-Hsiang Tsai, Chieh-Ming Yang, Chia-Hung Yeh, Lih-Jen Kau, and Chuan-Yu Chang. (2022), "Efficient partition decision based on visual perception and machine learning for H. 266/versatile video coding." IEEE Access 10 (2022): 42141-42150.
    [18] Zhu, Linwei, Yun Zhang, Na Li, Gangyi Jiang, and Sam Kwong. (2023), "Deep Learning-Based Intra Mode Derivation for Versatile Video Coding." ACM Transactions on Multimedia Computing, Communications and Applications 19, no. 2s (2023): 1-20.
    [19] Liu, Bowen, Yu Chen, Shiyu Liu, and Hun-Seok Kim. (2021), "Deep learning in latent space for video prediction and compression." In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 701-710. 2021.
    [20] Zhang, Yun, et al. (2023), "A survey on perceptually optimized video coding." ACM Computing Surveys 55.12 (2023): 1-37.
    [21] Korhonen, Jari, and Junyong You. (2012), "Peak signal-to-noise ratio revisited: Is simple beautiful?." In 2012 Fourth International Workshop on Quality of Multimedia Experience, pp. 37-38. IEEE, 2012.
    [22] Nasr, M. Abdel-Salam, Mohammed F. et al. (2017), "multi-scale structural similarity index for motion detection." Journal of King Saud University-Computer and Information Sciences 29, no. 3, 399-409.
    [23] Barman, Nabajeet, Maria G. Martini, and Yuriy Reznik. (2022), "Revisiting Bjontegaard delta bitrate (BD-BR) computation for codec compression efficiency comparison." In Proceedings of the 1st Mile-High Video Conference, pp. 113-114.

Pranob Kumar Charles, Habibulla Khan and K S Rao (2023), Adaptive Video Coding Framework with Spatial-Temporal Fusion for Optimized Streaming in Next-Generation Networks. IJEER 11(ngwcn), 20-24. DOI: 10.37391/ijeer.11ngwcn04.