Regression Monte Carlo for Impulse Control
Mike Ludkovski1
1 Department of Statistics and Applied Probability, University of California, Santa Barbara CA 93106-3110, USA
MathematicS In Action, Special issue Maths and Industry, Volume 11 (2022) no. 1, pp. 73-90.

I develop a numerical algorithm for stochastic impulse control in the spirit of Regression Monte Carlo for optimal stopping. The approach consists in generating statistical surrogates (aka functional approximators) for the continuation function. The surrogates are recursively trained by empirical regression over simulated state trajectories. In parallel, the same surrogates are used to learn the intervention function characterizing the optimal impulse amounts. I discuss appropriate surrogate types for this task, as well as the choice of training sets. Case studies from forest rotation and irreversible investment illustrate the numerical scheme and highlight its flexibility and extensibility. Implementation in R is provided as a publicly available package posted on GitHub.

Published online:
DOI: 10.5802/msia.18
Classification: 93E25, 65C05, 49N25
Mots-clés : Impulse Control, Statistical Surrogates, Irreversible Investment
Mike Ludkovski 1

1 Department of Statistics and Applied Probability, University of California, Santa Barbara CA 93106-3110, USA
License: CC-BY 4.0
Copyrights: The authors retain unrestricted copyrights and publishing rights 
@article{MSIA_2022__11_1_73_0,
     author = {Mike Ludkovski},
     title = {Regression {Monte} {Carlo} for {Impulse} {Control}},
     journal = {MathematicS In Action},
     pages = {73--90},
     publisher = {Soci\'et\'e de Math\'ematiques Appliqu\'ees et Industrielles},
     volume = {11},
     number = {1},
     year = {2022},
     doi = {10.5802/msia.18},
     language = {en},
     url = {https://msia.centre-mersenne.org/articles/10.5802/msia.18/}
}
TY  - JOUR
AU  - Mike Ludkovski
TI  - Regression Monte Carlo for Impulse Control
JO  - MathematicS In Action
PY  - 2022
SP  - 73
EP  - 90
VL  - 11
IS  - 1
PB  - Société de Mathématiques Appliquées et Industrielles
UR  - https://msia.centre-mersenne.org/articles/10.5802/msia.18/
DO  - 10.5802/msia.18
LA  - en
ID  - MSIA_2022__11_1_73_0
ER  - 
%0 Journal Article
%A Mike Ludkovski
%T Regression Monte Carlo for Impulse Control
%J MathematicS In Action
%D 2022
%P 73-90
%V 11
%N 1
%I Société de Mathématiques Appliquées et Industrielles
%U https://msia.centre-mersenne.org/articles/10.5802/msia.18/
%R 10.5802/msia.18
%G en
%F MSIA_2022__11_1_73_0
Mike Ludkovski. Regression Monte Carlo for Impulse Control. MathematicS In Action, Special issue Maths and Industry, Volume 11 (2022) no. 1, pp. 73-90. doi : 10.5802/msia.18. https://msia.centre-mersenne.org/articles/10.5802/msia.18/

[1] René Aid; Salvatore Federico; Huyên Pham; Bertrand Villeneuve Explicit investment rules with time-to-build and uncertainty, J. Econ. Dyn. Control, Volume 51 (2015), pp. 240-256 | MR | Zbl

[2] Luis H. R. Alvarez A class of solvable impulse control problems, Appl. Math. Optim., Volume 49 (2004) no. 3, pp. 265-295 | DOI | MR | Zbl

[3] Luis H. R. Alvarez Optimal capital accumulation under price uncertainty and costly reversibility, J. Econ. Dyn. Control, Volume 35 (2011) no. 10, pp. 1769-1788 | DOI | MR | Zbl

[4] Luis H. R. Alvarez; Erkki Koskela Optimal harvesting under resource stock and price uncertainty, J. Econ. Dyn. Control, Volume 31 (2007) no. 7, pp. 2461-2485 | DOI | MR | Zbl

[5] Luis H. R. Alvarez; Erkki Koskela Taxation and rotation age under stochastic forest stand value, J. Environ. Econ. Manage., Volume 54 (2007) no. 1, pp. 113-127 | DOI | Zbl

[6] Luis H. R. Alvarez; Jukka Lempa On the optimal stochastic impulse control of linear diffusions, SIAM J. Control Optim., Volume 47 (2008) no. 2, pp. 703-732 | DOI | MR | Zbl

[7] Pablo Azcue; Nora Muler; Zbigniew Palmowski Optimal dividend payments for a two-dimensional insurance risk process, Eur. Actuar. J., Volume 9 (2019) no. 1, pp. 241-272 | DOI | MR | Zbl

[8] Parsiad Azimzadeh; Erhan Bayraktar; George Labahn Convergence of implicit schemes for Hamilton–Jacobi–Bellman quasi-Variational inequalities, SIAM J. Control Optim., Volume 56 (2018) no. 6, pp. 3994-4016 | DOI | MR | Zbl

[9] Matteo Basei Optimal price management in retail energy markets: an impulse control problem with asymptotic estimates, Math. Methods Oper. Res., Volume 89 (2019) no. 3, pp. 355-383 | DOI | MR | Zbl

[10] Erhan Bayraktar; Andreas E. Kyprianou; Kazutoshi Yamazaki Optimal dividends in the dual model under transaction costs, Insur. Math. Econ., Volume 54 (2014), pp. 133-143 | DOI | MR | Zbl

[11] Erhan Bayraktar; Michael Ludkovski Inventory management with partially observed nonstationary demand, Ann. Oper. Res., Volume 176 (2010) no. 1, pp. 7-39 | DOI | MR | Zbl

[12] Christoph Belak; Sören Christensen; Frank Thomas Seifried A general verification result for stochastic impulse control problems, SIAM J. Control Optim., Volume 55 (2017) no. 2, pp. 627-649 | DOI | MR | Zbl

[13] Alain Bensoussan; Benoît Chevalier-Roignant Sequential capacity expansion options, Oper. Res., Volume 67 (2019) no. 1, pp. 33-57 | DOI | MR | Zbl

[14] Alain Bensoussan; R. H. Liu; Suresh P. Sethi Optimality of an (s,S) policy with compound Poisson and diffusion demands: A quasi-variational inequalities approach, SIAM J. Control Optim., Volume 44 (2005) no. 5, pp. 1650-1676 | DOI | MR | Zbl

[15] Abel Cadenillas; Fernando Zapatero Classical and impulse stochastic control of the exchange rate using interest rates and reserves, Math. Finance, Volume 10 (2000) no. 2, pp. 141-156 | DOI | MR | Zbl

[16] Yann-Shin Aaron Chen; Xin Guo Impulse control of multidimensional jump diffusions in finite time horizon, SIAM J. Control Optim., Volume 51 (2013) no. 3, pp. 2638-2663 | DOI | MR | Zbl

[17] Sören Christensen On the solution of general impulse control problems using superharmonic functions, Stochastic Processes Appl., Volume 124 (2014) no. 1, pp. 709-729 | DOI | MR | Zbl

[18] Irmina Czarna; Zbigniew Palmowski De Finetti’s dividend problem and impulse control for a two-dimensional insurance risk process, Stoch. Models, Volume 27 (2011) no. 2, pp. 220-250 | DOI | MR | Zbl

[19] Masahiko Egami A direct solution method for stochastic impulse control problems of one-dimensional diffusions, SIAM J. Control Optim., Volume 47 (2008) no. 3, pp. 1191-1218 | DOI | MR | Zbl

[20] Daniel Egloff; Michael Kohler; Nebojsa Todorovic A dynamic look-ahead Monte Carlo algorithm for pricing Bermudan options, Ann. Appl. Probab., Volume 17 (2007) no. 4, pp. 1138-1171 | MR | Zbl

[21] Brahim El Asri; Sehail Mazid Stochastic impulse control Problem with state and time dependent cost functions, Math. Control Relat. Fields, Volume 10 (2020) no. 4, p. 855 | DOI | MR | Zbl

[22] Brahim El Asri; Sehail Mazid Zero-sum stochastic differential game in finite horizon involving impulse controls, Appl. Math. Optim., Volume 81 (2020) no. 3, pp. 1055-1087 | DOI | MR | Zbl

[23] Salvatore Federico; Mauro Rosestolato; Elisa Tacconi Irreversible investment with fixed adjustment costs: a stochastic impulse control approach, Math. Financ. Econ., Volume 13 (2019) no. 4, pp. 579-616 | DOI | MR | Zbl

[24] Graeme Guthrie Uncertainty and the trade-off between scale and flexibility in investment, J. Econ. Dyn. Control, Volume 36 (2012) no. 11, pp. 1718-1728 | DOI | MR | Zbl

[25] Trevor Hastie; Robert Tibshirani; Jerome Friedman The elements of statistical learning: data mining, inference and prediction, Springer Series in Statistics, Springer, 2009 | DOI

[26] Jianqiang Hu; Cheng Zhang; Chenbo Zhu (s,S) inventory systems with correlated demands, INFORMS J. Comput., Volume 28 (2016) no. 4, pp. 603-611 | MR | Zbl

[27] Michael Kohler A regression-based smoothing spline Monte Carlo algorithm for pricing American options in discrete time, AStA, Adv. Stat. Anal., Volume 92 (2008) no. 2, pp. 153-178 | DOI | MR | Zbl

[28] Francis A. Longstaff; Eduardo S. Schwartz Valuing American options by simulations: a simple least squares approach, Rev. Financ. Stud., Volume 14 (2001), pp. 113-148 | DOI

[29] Mike Ludkovski mlOSP: Towards a Unified Implementation of Regression Monte Carlo Algorithms (2020) (https://arxiv.org/abs/2012.00729)

[30] Bernt Karsten Øksendal; Agnes Sulem Applied stochastic control of jump diffusions, 498, Springer, 2007 | DOI

[31] John Tsitsiklis; Benjamin Van Roy Regression Methods for Pricing Complex American-Style Options, IEEE Trans. Neural Netw., Volume 12 (2001) no. 4, pp. 694-703 | DOI

[32] Christopher K. I. Williams; Carl Edward Rasmussen Gaussian processes for machine learning, MIT Press, 2006

Cited by Sources: