Open Access
Original Articles
Value and Policy Function Approximations in Infinite-Horizon Optimization Problems
Giorgio Gneccogiorgio.gnecco@dist.unige.itDepartment of Mathematics (Dima)University of GenoaVia Dodecaneso 35, Genova, 16146, ItalyView full profile → , Marcello Sanguinetimarcello@dist.unige.itDepartment of CommunicationsComputer, and System Sciences (Dist)University of GenoaVia Opera Pia 13, 16145 Genova, ItalyView full profile →
* Corresponding author · click or hover a name for details
- Published Online:
- 03 Jun 2013
- Article type:
- Original Articles
- Language:
- EN
- Article no.:
- 1726037X.2008.10698552
- Pages:
- 123–147
Abstract
Suboptimal solutions to infinite-horizon dynamic optimization problems with continuous state are considered. An underlying dynamical system determining the state transition between each stage and the next one is modelled via the constraints (xx+1) ∈ D, t = 0, 1, …, where X is the set to which the state vector belongs and D ⊆ X × X is a correspondence. An error analysis is performed for two cases: approximation of the value function and approximation of the optimal policy function. Structural properties of the dynamic optimization problems are derived, allowing to restrict a priori the approximation to families of functions characterized by certain smoothness properties. The two approximation approaches are compared and the respective pros and cons are highlighted.
Keywords
References
- Aarts, E. and Korst, J.1989 . , John Wiley & Sons .
- Adams, R. A.1975 . , NY: Academic Press .
- Alessandri, A., Sanguineti, M. and Maggiore, M.2002 . Optimization-based learning with bounded error for feedforward neural networks . , 13 : 261 – 273 .
- Barron, A. R.1993 . Universal approximation bounds for superpositions of a sigmoidal function . , 39 : 930 – 945 .
- Bellman, R.1957 . , Princeton: Princeton University Press .
- Benveniste, L. M. and Scheinkman, J. A.1979 . On the differentiability of the value function in dynamic models of economics . , 47 : 727 – 732 .
- Bertsekas, D. P.1997 . A new class of incremental gradient methods for least squares problems . , 7 : 913 – 926 .
- Bertsekas, D. P.2005 . , Vol. 1 , Belmont: Athena Scientific .
- Bertsekas, D. P., Nedic, A. and Ozdaglar, A. E.2003 . , Belmont: Athena Scientific .
- Bertsekas, D. P. and Tsitsiklis, J.1996 . , Belmont: Athena Scientific .
- Blot, J. and Crettez, B.2004 . On the smoothness of optimal paths . , 27 : 1 – 34 .
- Boyd, S. and Vandenberghe, L.2004 . , Cambridge University Press .
- Breiman, L.1993 . Hinging hyperplanes for regression, classification and function approximation . , 39 : 999 – 1013 .
- Cugno, F. and Montrucchio, L.1998 . , Roma: Carocci Editore .
- Dacorogna, B.2004 . , Imperial College Press .
- Darken, C., Donahue, M., Gurvits, L. and Sontag, E.1993 . “ Rate of approximation results motivated by robust neural network learning ” . In , 303 – 309 . New York: The Association for Computing Machinery .
- Ekeland, I. and Turnbull, T.1983 . , Chicago: The University of Chicago Press .
- Gale, D.1967 . On optimal development in a multi-sector economy . , 34 : 1 – 18 .
- Girosi, F. and Anzellotti, G.1993 . “ Rates of convergence for Radial Basis Functions and neural networks ” . In , Edited by: Mammone, R. J.97 – 113 . Chapman & Hall . In
- Giulini, S. and Sanguineti, M.2008 . Approximation schemes for functional optimization problems . , DOI:10.1007/s10957-008-9471-6
- Gnecco, G. and Sanguineti, M.2008 . , DIST Internal Report submitted
- Gnecco, G. and Sanguineti, M.2008 . Approximation error bounds via Rademacher complexity . , 2 : 153 – 176 .
- Gnecco, G. and Sanguineti, M.Suboptimal solutions to dynamic optimization problems via approximations of the policy functions . , to appear
- Goldberg, D. E.1989 . , Addison-Wesley .
- Heinonen, J.2001 . , New York: Springer .
- Kůrková, V. and Sanguineti, M.2001 . Bounds on rates of variable-basis and neural–network approximation . , 47 : 2659 – 2665 .
- Kůrková, V. and Sanguineti, M.2002 . Comparison of worst-case errors in linear and neural network approximation . , 48 : 264 – 275 .
- Kůrková, V. and Sanguineti, M.2005 . Error estimates for approximate optimization by the extended Ritz method . , 18 : 461 – 487 .
- Kůrková, V. and Sanguineti, M.2008 . Geometric upper bounds on rates of variable-basis approximation . , 54 December (DOI: 10.1109/TIT.2008.2006383)
- Montrucchio, L.1987 . Lipschitz continuous policy functions for strongly concave optimization problems . , 16 : 259 – 273 .
- Montrucchio, L.1998 . Thompson metric, contraction property and differentiability of policy functions . , 33 : 449 – 466 .
- Montrucchio, L. and Boldrin, M.1986 . On the indeterminacy of capital accumulation paths . , 40 : 26 – 36 .
- Pinkus, A.1999 . Approximation theory of the MLP model in neural networks . , 8 : 143 – 195 .
- Powell, W. B.2007 . , Hoboken: John Wiley & Sons, Inc .
- Puterman, M. L.1994 . , Wiley .
- Rudin, W.1970 . , McGraw-Hill .
- Smith, J. E. and McCardle, K. F.2002 . Structural properties of stochastic dynamic programs . , 50 : 794 – 809 .
- Sogge, D.1993 . , Cambridge University Press .
- Stein, E. M.1970 . , Princeton: Princeton University Press .
- Stokey, N. L., Lucas, R. E. and Prescott, E.1989 . , MA: Harvard University Press .
- Vapnik, V. N.1998 . , New York: Wiley .
- Venditti, A.2002 . “ Nonlinear dynamics and indeterminacy in multisector growth models ” . In , Lisbon: Portuguese Catholic University .
- Yin, G.1999 . Rates of convergence for a class of global stochastic optimization algorithms . , 10 : 99 – 120 .
- Zhang, F., ed. 2005 . , Springer .
- Zoppoli, R., Sanguineti, M. and Parisini, T.2002 . Approximating networks and extended Ritz method for the solution of functional optimization problems . , 112 : 403 – 439 .
Views: 53Downloads: 60Citations: 2
