The dynamics of biological populations is a complex process that includes selection, mutation, inheritance, migration and random genetic variation, and predicting the dynamics is difficult. The issue of quantifying these interaction mechanisms in a single functional mathematical model is tackled in this work. The goal is to create mathematical models that will enable the description of the evolution of population states and long time behavior of the model. The approach utilized is based on nonlinear differential equations, replicator dynamics, mutation selection equations, equilibrium analysis and numerical simulation. Under various parameter conditions, the stability, convergence, fitness of populations, variation in allele frequencies and evolutionary trajectory are analyzed. Results show that the extent of selection and mutation rates have a significant effect on the establishment of an equilibrium, population diversities, and adaptation. Findings show that the results vary with stable, unstable and coexistence states under different evolutionary scenarios. The innovations are the combination of several evolutionary mechanisms within a tractable computational framework. The study provides quantitative tools for predicting population adaptation, and also concludes that mathematical modeling can provide a characterization of the complex evolutionary dynamics.