The present paper tries to introduce a new process for solving fuzzy singular perturbation problem(SPP, s) with initial condition. This approach depends on on the partially fuzzy neural network to find the numerical solution of the second order of these problems. This system’s trial solution is written as a sum of two parts. The first section meets the fuzzy initial condition and does not have any fuzzy free parameters. The second component consists of a partially fuzzy feed-forward neural network. containing fuzzy adjustable parameters (the fuzzy weights). As a result, the starting condition is fulfilled by construction, and the network is trained to solve the differential equations. When compared to other numerical techniques, this technique proves that neural networks generate solutions with high generalizability and accurateness. A number of examples are given to show the proposed plan.