Today’s content delivery over the Internet via Cloud-based Content Delivery Networks (CCDNs) must address critical challenges such as dynamic traffic, congestion, inefficient routing, and latency variations, which are not effectively handled by the traditional Dijkstra’s algorithm due to its static cost assignments. To overcome these challenges, a congestion-aware collaborative Dijkstra optimization approach guided by the Proximal Policy Optimization (PPO) method is proposed. This PPO-Guided Dijkstra integrates graph-theoretic path computation with optimization-driven cost adjustments by considering real-time congestion data. Simulation-based assessments across various network conditions indicate that the proposed model achieves a 33% reduction in latency, a 6.7% improvement in resilience under node failure scenarios, and significantly faster convergence in comparison to conventional and non-collaborative routing strategies. These findings demonstrate the scalable, congestion-aware routing solution for next-generation CCDNs.