In the age of machine learning, Human Exercise Recognition (HER) is an area that has been extensively researched. When it comes to computer vision, action recognition is the process of classifying a human exercise that is seen in a video as belonging to one of a selection of prepared actions.  This paper proposes a new method for recognizing physical activity using a Hybrid Object Boundary Value Deep Sparse Auxiliary Network (HOBV-DSAN). The technique combines object boundary value characteristics with deep sparse learning frameworks to improve the depiction of complicated human motions. Our model captures the global structure as well as the fine-grained motion features of physical activities by integrating boundary-aware representations with a sparsity-driven deep auxiliary network. Experimental results show that the proposed technique outperforms traditional models such as YOLO, Alpha Pose, and Deep Pose on many performance criteria such as Accuracy (95.33%), Sensitivity (99.05%), Specificity (97.47%), and Precision (94.57%). The findings demonstrate the HOBV-DSAN model’s durability and accuracy in detecting a broad range of workouts, making it a suitable option for intelligent physical activity monitoring systems.