An optimization-driven reinforcement learning with vision transformer generative adversarial network for medical image reconstruction
*Priyanka SharmaCorresponding authorvipri24@gmail.comDepartment of Computer Science and EngineeringPoornima UniversityJaipur, Rajasthan, 303905, IndiaView full profile → , Bright Keswanibright.keswani@poornima.edu.inDepartment of Computer Science and EngineeringPoornima UniversityJaipur, Rajasthan, 303905, IndiaView full profile →
* Corresponding author · click or hover a name for details
- Received:
- 01 Nov 2025
- Published Online:
- 31 Aug 2026
- Article type:
- Research Article
- Language:
- EN
- Article no.:
- JIOS-2210
- Pages:
- 2891–2902
Abstract
The diagnosis of cerebral diseases often requires imaging to assess the state of the brain. Magnetic resonance imaging (MRI) is routinely used to detect brain disorders. However, the obtained MR images often lack sufficient resolution due to restrictions such as patient comfort and prolonged sampling period. Analyzing the low-resolution MRI may result in an erroneous diagnosis. Consequently, there is a need for super-resolution methods to acquire high-resolution MRI images. The Reconstruction-based Super Resolution (SR) approach is a kind of single image SR that reconstructs low-resolution MRI scans into high-resolution pictures. The proposed Vit+TransGAN+RL integrates Vision Transformer (ViT), Transformerbased Generative Adversarial Network (Trans GAN), and Reinforcement Learning (RL), wherein the RL agent dynamically regulates the k-space sampling and reconstruction process by selecting optimal acquisition or refinement actions based on a learned policy that maximizes image quality while minimizing acquisition time. Experimental evaluation demonstrates that the proposed ViT+TransGAN + RL framework achieves 11.47% of MSE, 40.12% of PSNR, 0.7792% of SSIM, 0.1665 of FSIM, -1.4543 of NRMSE, 0.9183 of VIF, 0.8858 of DSC for degradation and 24.65% of MSE, 34.65% of MSE, 0.25 of SSIM, 0.094 of FSIM, -1.45 of NRMSE, 0.445 of VIF and 0.68 of DSC.
Keywords
Subject Classifications
References
[1] A. Al-Kaff, D. Martin, F. Garcia, A. de la Escalera, and J. M. Armingol, “Survey of computer vision algorithms and applications for unmanned aerial vehicles,” Expert Syst. Appl., vol. 92, pp. 447–463 (2018), doi: 10.1016/j.eswa.2017.09.033.
[2] S. Bash, B. Johnson, W. Gibbs, T. Zhang, A. Shankaranarayanan, and L. N. Tanenbaum, “Deep learning image processing enables 40% faster spinal MR scans which match or exceed quality of standard of care: A prospective multicenter multireader study,” Clin. Neuroradiol., vol. 32, no. 1, pp. 197–203 (2022), doi: 10.1007/s00062-021-01121-2.
[3] S. Bash, L. Wang, C. Airriess, G. Zaharchuk, E. Gong, A. Shankaranarayanan, and L. N. Tanenbaum, “Deep learning enables 60% accelerated volumetric brain MRI while preserving quantitative performance: A prospective, multicenter, multireader trial,” AJNR Am. J. Neuroradiol., vol. 42, no. 12, pp. 2130–2137 (2021), doi: 10.3174/ajnr.A7358.
[4] S. Bhuvaji, A. Kadam, P. Bhumkar, S. Dedge, and S. Kanchan, “Brain tumor classification (MRI),” Kaggle Dataset (2020). [Online]. Available: Kaggle – Brain Tumor Classification (MRI).
[5] S. S. Chandra, M. Bran Lorenzana, X. Liu, S. Liu, S. Bollmann, and S. Crozier, “Deep learning in magnetic resonance image reconstruction,” J. Med. Imaging Radiat. Oncol., vol. 65, no. 5, pp. 564–577 (2021), doi: 10.1111/1754-9485.13276.
[6] A. Estler, T.-K. Hauser, A. Mengel, M. Brunnée, L. Zerweck, V. Richter, M. Zuena, M. Schuhholz, U. Ernemann, and G. Gohla, “Deep learning accelerated image reconstruction of fluid-attenuated inversion recovery sequence in brain imaging: Reduction of acquisition time and improvement of image quality,” Acad. Radiol., vol. 31, no. 1, pp. 180–186 (2024), doi: 10.1016/j.acra.2023.05.010.
[7] V. D. Gowda, Y. N. Sunitha, S. V. Chakrasali, K. D. V. Prasad, P. Sarma, and M. Rahman, “Mathematical analysis of wavelet-based multi-image compression in medical diagnostics,” J. Discrete Math. Sci. Cryptogr., vol. 27, no. 2-B, pp. 675–687 (2024), doi: 10.47974/JDMSC-1912.
[8] H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial: Deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1153–1159 (2016), doi: 10.1109/TMI.2016.2553401.
[9] A. Hofmann, M. Klein, D. Rueter, and A. Sauer, “A deep residual neural network for image reconstruction in biomedical 3D magnetic induction tomography,” Sensors, vol. 22, no. 20, Art. no. 7925 (2022), doi: 10.3390/s22207925.
[10] J. Hsieh, E. Liu, B. Nett, J. Tang, J.-B. Thibault, and S. Sahney, A New Era of Image Reconstruction: TrueFidelity™—Technical White Paper on Deep Learning Image Reconstruction, GE Healthcare, Chicago, IL, USA (2019).
[11] J. Huang, S. Wang, G. Zhou, W. Hu, and G. Yu, “Evaluation on the generalization of a learned convolutional neural network for MRI reconstruction,” Magn. Reson. Imaging, vol. 87, pp. 38–46 (2022), doi: 10.1016/j.mri.2021.12.003.
[12] S. H. Kim, Y. H. Choi, J. S. Lee, S. B. Lee, Y. J. Cho, S. H. Lee, S. M. Shin, and J. E. Cheon, “Deep learning reconstruction in pediatric brain MRI: Comparison of image quality with conventional T2-weighted MRI,” Neuroradiology, vol. 65, no. 1, pp. 207–214 (2023), doi: 10.1007/s00234-022-03053-1.
[13] X. Li, H. Zhang, H. Yang, and T. Q. Li, “CS-MRI reconstruction using an improved GAN with dilated residual networks and channel attention mechanism,” Sensors, vol. 23, no. 18, Art. no. 7685 (2023), doi: 10.3390/s23187685.
[14] M. Arvinte, S. Vishwanath, A. H. Tewfik, and J. I. Tamir, “Deep J-Sense: Accelerated MRI reconstruction via unrolled alternating optimization,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Intervent. (MICCAI), pp. 350–360 (2021), doi: 10.1007/978-3-030-87237-3_34.
[15] A. Noonia, D. Thakral, P. Mathur, F. Sheth, H. Shaikh, and A. K. Gupta, “A discrete mathematical model and cryptography for secure medical image analysis: Encrypted chest X-ray classification,” J. Discrete Math. Sci. Cryptogr., vol. 28, no. 5-A, pp. 1473–1486 (2025), doi: 10.47974/JDMSC-2146.
[16] R. Noor, A. Wahid, S. U. Bazai, A. Khan, M. Fang, Syam M.S., U. A. Bhatti, and Y. Y. Ghadi, “DLGAN: Undersampled MRI reconstruction using deep learning based generative adversarial network,” Biomed. Signal Process. Control, vol. 93, Art. no. 106218 (2024), doi: 10.1016/j.bspc.2024.106218.
[17] O. Nitski, S. Nag, C. McIntosh, and B. Wang, “CDF-Net: Cross-domain fusion network for accelerated MRI reconstruction,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Intervent. (MICCAI), pp. 421–430 (2020), doi: 10.1007/978-3-030-59710-8_41.
[18] I. Oksuz, “Brain MRI artefact detection and correction using convolutional neural networks,” Comput. Methods Programs Biomed., vol. 199, Art. no. 105909 (2021), doi: 10.1016/j.cmpb.2020.105909.
[19] A. G. Oskouei, N. Abdolmaleki, A. Bouyer, B. Arasteh, and K. Shirini, “Efficient superpixel-based brain MRI segmentation using multiscale morphological gradient reconstruction and quantum clustering,” Biomed. Signal Process. Control, vol. 100, Art. no. 107063 (2025), doi: 10.1016/j.bspc.2024.107063.
[20] A. S. Rao and K. Mahantesh, “Impact of computer vision based secure image enrichment techniques on image classification model,” J. Discrete Math. Sci. Cryptogr., vol. 26, no. 3, pp. 899–911 (2023), doi: 10.47974/JDMSC-1777.
[21] R. Shaul, I. David, O. Shitrit, and T. R. Raviv, “Subsampled brain MRI reconstruction by generative adversarial neural networks,” Med. Image Anal., vol. 65, Art. no. 101747 (2020), doi: 10.1016/j.media.2020.101747.
[22] Z. Wu, W. Liao, C. Yan, M. Zhao, G. Liu, N. Ma, and X. Li, “Deep learning based MRI reconstruction with transformer,” Comput. Methods Programs Biomed., vol. 233, Art. no. 107452 (2023), doi: 10.1016/j.cmpb.2023.107452.
[23] B. Yacoub, A. Varga-Szemes, U. J. Schoepf, I. M. Kabakus, D. Baruah, J. R. Burt, G. J. Aquino, A. K. Sullivan, J. O. Doherty, P. Hoelzer, and J. Sperl, “Impact of artificial intelligence assistance on chest CT interpretation times: A prospective randomized study,” AJR Am. J. Roentgenol., vol. 219, no. 5, pp. 743–751 (2022), doi: 10.2214/AJR.22.27927.
[24] W. Zhou, H. Du, W. Mei, and L. Fang, “Efficient structurally-strengthened generative adversarial network for MRI reconstruction,” Neurocomputing, vol. 422, pp. 51–61 (2021), doi: 10.1016/j.neucom.2020.09.068.
