The integration of AI into medical diagnostics has transformed the detection of gastrointestinal diseases. In this study, we develop a clear mathematical framework for classifying multiple GI conditions. We extend the GastroVision dataset by adding three specific classes—Inflammatory Conditions, Neoplastic Growths and Interventional Findings—and employ three state-of-the-art transfer-learning models (EfficientNetV2, Vision Transformer and Swin Transformer). Outputs from these models are combined using a stacking ensemble; we apply standard data-augmentation techniques and systematically tune each model’s parameters to maximize performance. Our ensemble achieved an overall accuracy of 94.8%, with precision at 92.3%, recall at 93.7% and an F1-score of 93.0%, exceeding current benchmarks. Performance was strong across all categories, with the highest accuracy of 96.1% on Neoplastic Growths. These results demonstrate both the mathematical soundness and practical value of our approach. By linking rigorous model development to clinical applications, this work provides a reliable, extensible framework for deploying AI in complex diagnostic settings and has the potential to improve patient outcomes.