This research presents a privacy-preserving approach to chest X-ray image classification utilizing deep learning techniques integrated with Advanced Encryption Standard (AES) encryption and Fully Homomorphic Encryption (FHE) to maintain data confidentiality. The methodology employs various Convolutional Neural Network (CNN) models to classify chest X-rays into three categories: normal, COVID-19 infected, and pneumonia infected, without decrypting the data. The process involves preprocessing the images, determining optimal hyperparameters, and fine-tuning the CNN models through transfer learning. The models evaluated include ResNet50, ResNet101, MobileNetV3 Small, MobileNetV3 Large, DenseNet121, and DenseNet161, with performance metrics such as accuracy, F1-score, precision, recall (sensitivity), and specificity used for comprehensive assessment. Among the tested models, DenseNet161 achieved the highest accuracy of 92.28%, followed by DenseNet121 and ResNet50 with accuracies of 91.41% and 91.26%, respectively. This study demonstrates the feasibility and effectiveness of combining encryption with deep learning for secure and accurate medical image analysis, thereby addressing the critical need for privacy in medical diagnostics.