f MCBIR: Deep Learning based Framework for Efficient Content Based Image Retrieval System of Medical Images
FOREX Press I. J. of Electrical & Electronics Research
Support Open Access
  • Home/
  • IJEER-120430

Research Article |

MCBIR: Deep Learning based Framework for Efficient Content Based Image Retrieval System of Medical Images

Author(s): Dr. T Bhaskar*, Dr. Y. Ramadevi, Dr. Pasam Naga Kavitha, and Padala Sravan

Published In : International Journal of Electrical and Electronics Research (IJEER)        Volume 12, Issue 4

Publisher : FOREX Publication

Published : 15 December 2024

e-ISSN : 2347-470X

Page(s) : 1364-1373

DOI: https://doi.org/10.37391/IJEER.120430

Abstract

Content-Based Image Retrieval (CBIR) in computer vision applications, enables retrieval of images reflecting user intent. Traditionally CBIR is based on image processing techniques. With the emergence of Artificial Intelligence (AI), it is now possible to realize CBIR using learning-based approaches. Particularly deep learning techniques such as Convolutional Neural Network (CNN) are efficient for image analysis. In this paper, we proposed a framework known Medical Content Based Image Retrieval System (MCBIRS), which exploits pre-trained CNN variants for retrieving medical images based on image input. The framework has an offline phase for extracting visual features from training data and an online phase for processing given user queries. The descriptors obtained by CNN variants in the offline phase are persisted in a database. These are later used in the online phase to compute the distance between persisted descriptors and input image descriptor. A set of closely matching images are returned against the query image based on similarity. We proposed an algorithm known as Learning-based Medical Image Retrieval (LbMIR) to realize MCBIRS. We also implemented a re-ranking of results retrieved by the framework using other techniques. The performance of LbMIR is evaluated and compared with the state-of-the-art methods such as Bag of Visual Words (BoVW) and Histogram of Oriented Gradients (HOG). Empirical results using medical image dataset revealed that CNN variants outperformed BoVW and HOG methods. On test data, the highest performance is achieved by the proposed system with 90% mean top-k precision, demonstrating its practical implications. On the training data highest performance is achieved by proposed system (CNN variants) re-ranked with HOG with 92.30% mean top-k precision.

Keywords: Deep Learning, Convolutional Neural Network, Content-Based Image Retrieval, Medical CBIR.




Dr. T Bhaskar*, Assistant professor, Department of Computer Science and Engineering, CMR College of Engineering & Technology, Kandlakoya, Medchal Road, Hyderabad, India; Email: bhalu7cs@gmail.com

Dr. Y. Ramadevi, Professor, Department of AIML, Chaitanya Bhàrathi Institute of Technology Osman Sagar Rd, Kokapet, Gandipet, Hyderabad, Telangana, India; Email: yramadevi_cseaiml@cbit.ac.in

Dr. Pasam Naga Kavitha, Assistant Professor, Department of Computer Science, St.Ann's College For Women, Santosh Nagar, Mehdipatnam, Hyderabad, Telangana, India; Email: kavithacs.stanns@gmail.com

Padala Sravan, Assistant Professor, Department of School of CS & AI, SR University, India; Email: padalasravanwgl@gmail.com

    [1] Owais Muhammad, Arsalan Muhammad, Choi Jiho and Park Kang Ryoung. (2019). Effective Diagnosis and Treatment through Content-Based Medical Image Retrieval (CBMIR) by Using Artificial Intelligence. Journal of Clinical Medicine, 8(4), pp.1–31. doi:10.3390/jcm8040462.
    [2] Srinivas M., Naidu R. Ramu, Sastry C.S. and Mohan C. Krishna. (2015). Content-based medical image retrieval using dictionary learning. Neurocomputing, 168, pp.880–895. doi:10.1016/j.neucom.2015.05.036.
    [3] Ma Ling, Liu Xiabi, Gao Yan, Zhao Yanfeng, Zhao Xinming and Zhou Chunwu. (2017). A new method of content-based medical image retrieval and its applications to CT imaging sign retrieval. Journal of Biomedical Informatics, 66, pp.148–158. doi:10.1016/j.jbi.2017.01.002.
    [4] Ahmed and Ali (2020). Implementing Relevance Feedback for Content-Based Medical Image Retrieval. IEEE Access, 8, pp.79969–79976. doi:10.1109/ACCESS.2020.2990557.
    [5] Şaban Öztürk (2020). Stacked auto-encoder-based tagging with deep features for content-based medical image retrieval. Expert Systems with Applications, pp.1–19. doi:10.1016/j.eswa.2020.113693.
    [6] Das Pranjit and Neelima Arambam (2017). An overview of approaches for content-based medical image retrieval. International Journal of Multimedia Information Retrieval, pp.1–10. doi:10.1007/s13735-017-0135-x.
    [7] Cook Tessa S, Zhang Jianguo, Sun Qinpei, Yang Yuanyuan, Sun Jianyong, Yang Zhiming and Zhang Jianguo (2017). Using deep learning for content-based medical image retrieval. Imaging Informatics for Healthcare, Research, and Applications, pp.1–11. doi:10.1117/12.2251115.
    [8] Shamna P, Govindan V.K and Abdul Nazeer K.A. (2018). Content-based medical image retrieval by spatial matching of visual words. Journal of King Saud University - Computer and Information Sciences. pp.1-14. doi:10.1016/j.jksuci.2018.10.002.
    [9] Renita D. Benyl and Christopher C. Seldev. (2020). Novel real time content based medical image retrieval scheme with GWO-SVM. Multimedia Tools and Applications, pp.1–17. doi:10.1007/s11042-019-07777-w.
    [10] Seetharaman K. and Sathiamoorthy S. (2016). A unified learning framework for content-based medical image retrieval using a statistical model. Journal of King Saud University - Computer and Information Sciences, 28(1), pp.110–124. doi:10.1016/j.jksuci.2014.10.006.
    [11] Vo Thi Hong Tuyet, Nguyen Thanh Binh, Nguyen Kim Quoc and Ashish Khare. (2021). Content-based Medical image retrieval is based on salient regions combined with deep learning. Mobile Networks and Applications, pp.1–11. doi:10.1007/s11036-021-01762-0.
    [12] Mohd Zin Nor Asma, Yusof Rozianiwati, Lashari Saima Anwar, Mustapha Aida, Senan Norhalina and Ibrahim Rosziati. (2018). Content-Based Image Retrieval in Medical Domain: A Review. Journal of Physics: Conference Series, pp.1–13. doi:10.1088/1742-6596/1019/1/012044.
    [13] Karthik K. and Kamath S. Sowmya (2020). A deep neural network model for content-based medical image retrieval with multi-view classification. The Visual Computer, pp.1–14. doi:10.1007/s00371-020-01941-2.
    [14] Şaban Ozturk. (2021). Class-driven content-based medical image retrieval using hash codes of deep features. Biomedical Signal Processing and Control, pp.1–9. doi:10.1016/j.bspc.2021.102601.
    [15] Shamna P, Govindan V.K and Abdul Nazeer K.A. (2019). Content-Based Medical Image Retrieval using Topic and Location Model. Journal of Biomedical Informatics, pp.1–42. doi:10.1016/j.jbi.2019.103112.
    [16] Ahmed Ali and Malebary Sharaf J. (2020). Query Expansion Based on Top-Ranked Images for Content-Based Medical Image Retrieval. IEEE Access, 8, pp.194541–194550. doi:10.1109/ACCESS.2020.3033504.
    [17] Widmer Antoine, Schaer Roger, Markonis Dimitrios and Müller Henning. (2014). Gesture Interaction for Content-based Medical Image Retrieval, ACM, pp.503–506. doi:10.1145/2578726.2578804.
    [18] Mathew Jimson, Patra Priyadarshan, Pradhan Dhiraj K. and Kuttyamma A. J. (2012). CBMIR: Content-Based Medical Image Retrieval System Using Texture and Intensity for Dental Images, pp.125–134. doi:10.1007/978-3-642-32112-2_16.
    [19] B. Jyothi, Y. Madhaveelatha, and P. Mohan. (2015). A compelling multiple visual features for Content Based Medical Image Retrieval, pp.1–5. doi:10.1109/ISCO.2015.7282301.
    [20] Haripriya P. and Porkodi R. (2020). Parallel deep convolutional neural network for content based medical image retrieval. Journal of Ambient Intelligence and Humanized Computing, pp1–15. doi:10.1007/s12652-020-02077-w.
    [21] Muramatsu and Chisako (2018). Overview on subjective similarity of images for content-based medical image retrieval. Radiological Physics and Technology, 11(2), pp.109–124. doi:10.1007/s12194-018-0461-6.
    [22] Rajalakshmi T. and Minu R. I. (2014). Improving relevance feedback for content based medical image retrieval, IEEE, pp.1–5. doi:10.1109/icices.2014.7033863.
    [23] Bedo Marcos Vinicius Naves, Pereira dos Santos Davi, Ponciano-Silva Marcelo, de Azevedo-Marques, Paulo Mazzoncini, Ferreira de Carvalho, André Ponce de León and Traina Caetano (2016). Endowing a Content-Based Medical Image Retrieval System with Perceptual Similarity Using Ensemble Strategy. Journal of Digital Imaging, 29(1), pp.22–37. doi:10.1007/s10278-015-9809-1.
    [24] Mirasadi Mansoureh Sadat and Foruzan Amir Hossein. (2019). Content-based medical image retrieval of CT images of liver lesions using manifold learning. International Journal of Multimedia Information Retrieval, pp.1–8. doi:10.1007/s13735-019-00179-6.
    [25] Zahra Tabatabaei, Yuandou Wang, Adrián Colomer and Javier Oliver Mol. (2023). WWFedCBMIR: World-Wide Federated Content-Based Medical Image Retrieval. MDPI, pp.1-19.
    [26] Karthik K and Sowmya Kamath S. (2018). A Hybrid Feature Modeling Approach for Content-Based Medical Image Retrieval, IEEE, pp.7–12. doi:10.1109/ICIINFS.2018.8721432.
    [27] Kumar Ashnil, Dyer Shane, Kim Jinman, Li Changyang, Leong Philip H.W, Fulham Michael and Feng Dagan. (2016). Adapting content-based image retrieval techniques for the semantic annotation of medical images. Computerized Medical Imaging and Graphics, pp.1–13. doi:10.1016/j.compmedimag.2016.01.001.
    [28] S. Govindaraju and G. P. Ramesh Kumar. (2016). A Novel Content Based Medical Image Retrieval using SURF Features. Indian Journals of Science and Technology. 9(20), pp.1-8.
    [29] Widmer Antoine, Schaer Roger, Markonis Dimitrios and Muller Henning. (2014). Facilitating medical information search using Google Glass connected to a content-based medical image retrieval system, IEEE, pp.4507–4510. doi:10.1109/embc.2014.6944625.
    [30] Shubham Agrawal, Aastha Chowdhary, Saurabh Agarwala and Veena Mayya. (2022). Content‑based medical image retrieval system for lung diseases using deep CNNs. Springer, pp.1-9.
    [31] Krizhevsky, A. et al. “ImageNet classification with deep convolutional neural networks.” Communications of the ACM 60, 2012.
    [32] He, Kaiming et al. “Deep Residual Learning for Image Recognition.” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
    [33] Simonyan, K. and Andrew Zisserman. “Very Deep Convolutional Networks for Large-Scale Image Recognition.” CoRR abs/1409.1556, 2015.
    [34] Rashad M, Afifi I, Abdelfatah M. RbQE: An Efficient Method for Content-Based Medical Image Retrieval Based on Query Expansion. J Digit Imaging. 2023 Jun;36(3):1248-1261. doi: 10.1007/s10278-022-00769-7. Epub 2023 Jan 26. PMID: 36702987; PMCID: PMC10287886.
    [35] Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. 2009 June. https://doi. org/10.1109/cvpr.2009.5206848.
    [36] Bar Y, Diamant I, Wolf L, Greenspan H. Deep learning with nonmedical training used for chest pathology identification. 2015 Mar. https://doi.org/10.1117/12.2083124.
    [37] Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229. 2013.
    [38] Rey-Otero, Ives and M. Delbracio. “Anatomy of the SIFT Method.” Image Process. Line 4 : 370-396., 2014.
    [39] Bag of Visual Words for Finding Similar Images, Cyrill Stachniss, 2020.
    [40] Mano, Y., & Kusano, K. (2015). A validated LC–MS/MS method of total and unbound lenvatinib quantification in human serum for protein binding studies by equilibrium dialysis. Journal of Pharmaceutical and Biomedical Analysis, 114, 82–87. doi:10.1016/j.jpba.2015.05.008.
    [41] Trojacanec, K., Dimitrovski, I., & Loskovska, S. (2009). Content based image retrieval in medical applications: an improvement of the two-level architecture. IEEE EUROCON 2009. doi:10.1109/eurcon.2009.5167614.
    [42] Chuctaya, H., Portugal, C., Beltran, C., Gutierrez, J., Lopez, C., & Tupac, Y. (2011). M-CBIR: A Medical Content-Based Image Retrieval System Using Metric Data-Structures. 2011 30th International Conference of the Chilean Computer Science Society. doi:10.1109/sccc.2011.18.

Dr. T Bhaskar, Dr. Y. Ramadevi, Dr. Pasam Naga Kavitha, and Padala Sravan (2024), MCBIR: Deep Learning based Framework for Efficient Content Based Image Retrieval System of Medical Images. IJEER 12(4), 1364-1373. DOI: 10.37391/ijeer.120430.