Home | Contents | Submissions, editors, etc. | Login | Search | ECP
 Electronic Journal of Probability > Vol. 6 (2001) > Paper 17 open journal systems 


Stationary Solutions and Forward Equations for Controlled and Singular Martingale Problems

Thomas G. Kurtz, University of Wisconsin, Madison
Richard H. Stockbridge, University of Kentucky


Abstract
Stationary distributions of Markov processes can typically be characterized as probability measures that annihilate the generator in the sense that $int_EAfdmu =0$ for $fin {cal D}(A)$; that is, for each such $mu$, there exists a stationary solution of the martingale problem for A with marginal distribution $ mu$. This result is extended to models corresponding to martingale problems that include absolutely continuous and singular (with respect to time) components and controls. Analogous results for the forward equation follow as a corollary.
Full text: PDF

Pages: 1-52

Published on: January 17, 2001
Bibliography
  1. Bhatt, Abhay G.; Borkar, Vivek S. Occupation measures for controlled Markov processes: characterization and optimality. Ann. Probab. 24 (1996), no. 3, 1531-1562. Math. Reviews number 97i:90105
  2. Bhatt, Abhay G.; Karandikar, Rajeeva L. Invariant measures and evolution equations for Markov processes characterized via martingale problems. Ann. Probab. 21 (1993), no. 4, 2246-2268. Math. Reviews number 95d:60120
  3. Bhatt, Abhay G.; Kallianpur, G.; Karandikar, Rajeeva L. Uniqueness and robustness of solution of measure-valued equations of nonlinear filtering. Ann. Probab. 23 (1995), no. 4, 1895-1938. Math. Reviews number 97a:60061
  4. Bhatt, A. G.; Karandikar, R. L. Evolution equations for Markov processes: application to the white-noise theory of filtering. Appl. Math. Optim. 31 (1995), no. 3, 327-348. Math. Reviews number 95k:60181
  5. Bhatt, Abhay G.; Karandikar, Rajeeva L. Characterization of the optimal filter: the non-Markov case. Stochastics Stochastics Rep. 66 (1999), no. 3-4, 177-204. Math. Reviews number 2000e:60063
  6. Borkar, Vivek S.; Ghosh, Mrinal K. Ergodic control of multidimensional diffusions. I. The existence results. SIAM J. Control Optim. 26 (1988), no. 1, 112-126. Math. Reviews number 89h:93054
  7. Dai, J. G.; Harrison, J. M. Steady-state analysis of RBM in a rectangle: numerical methods and a queueing application. Ann. Appl. Probab. 1 (1991), no. 1, 16-35. Math. Reviews number 92e:60154
  8. Davis, M. H. A.; Norman, A. R. Portfolio selection with transaction costs. Math. Oper. Res. 15 (1990), no. 4, 676-713. Math. Reviews number 92b:90036
  9. Donnelly, Peter; Kurtz, Thomas G. Particle representations for measure-valued population models. Ann. Probab. 27 (1999), no. 1, 166-205. Math. Reviews number 2000f:60108
  10. Echeverrķa, Pedro. A criterion for invariant measures of Markov processes. Z. Wahrsch. Verw. Gebiete 61 (1982), no. 1, 1-16. Math. Reviews number 84a:60088
  11. Ethier, Stewart N.; Kurtz, Thomas G. Markov processes. Characterization and convergence. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons, Inc., New York, 1986. x+534 pp. ISBN: 0-471-08186-8 Math. Reviews number 88a:60130
  12. Jacod, Jean; Shiryaev, Albert N. Limit theorems for stochastic processes. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], 288. Springer-Verlag, Berlin-New York, 1987. xviii+601 pp. ISBN: 3-540-17882-1 Math. Reviews number 89k:60044
  13. Jakubowski, Adam. A non-Skorohod topology on the Skorohod space. Electron. J. Probab. 2 (1997), no. 4, 21 pp. (electronic). Math. Reviews number 98k:60046
  14. Karatzas, Ioannis. A class of singular stochastic control problems. Adv. in Appl. Probab. 15 (1983), no. 2, 225-254. Math. Reviews number 84g:93084
  15. Kurtz, Thomas G. Martingale problems for constrained Markov processes. Recent advances in stochastic calculus. Papers from the Distinguished Lecture Series on Stochastic Calculus held at the University of Maryland, College Park, Maryland, 1987. Edited by John S. Baras and Vincent Mirelli. Progress in Automation and Information Systems. Springer-Verlag, New York, 1990. x+217 pp. ISBN: 0-387-97273-0 Math. Reviews number 94f:93001
  16. Kurtz, Thomas G. A control formulation for constrained Markov processes. Mathematics of random media (Blacksburg, VA, 1989), 139-150, Lectures in Appl. Math., 27, Amer. Math. Soc., Providence, RI, 1991. Math. Reviews number 92h:60122
  17. Kurtz, Thomas G. Random time changes and convergence in distribution under the Meyer-Zheng conditions. Ann. Probab. 19 (1991), no. 3, 1010-1034. Math. Reviews number 92m:60033
  18. Kurtz, Thomas G. Averaging for martingale problems and stochastic approximation. Applied stochastic analysis (New Brunswick, NJ, 1991), 186-209, Lecture Notes in Control and Inform. Sci., 177, Springer, Berlin, 1992. Math. Reviews number 93h:60070
  19. Kurtz, Thomas G. Martingale problems for conditional distributions of Markov processes. Electron. J. Probab. 3 (1998), no. 9, 29 pp. (electronic). Math. Reviews number 99k:60186
  20. Kurtz, T. G.; Ocone, D. L. Unique characterization of conditional distributions in nonlinear filtering. Ann. Probab. 16 (1988), no. 1, 80-107. Math. Reviews number 88m:93146
  21. Kurtz, Thomas G.; Stockbridge, Richard H. Existence of Markov controls and characterization of optimal Markov controls. SIAM J. Control Optim. 36 (1998), no. 2, 609-653 (electronic). Math. Reviews number 99b:93051
  22. Kurtz, Thomas G.; Stockbridge, Richard H. Martingale problems and linear programs for singular control, Thirty-seventh annual Allerton conference on communication, control and computing (Monticello, Ill., 1999), 11-20, Univ. Illinois, Urbana-Champaign, Ill. Math. Reviews number not available.
  23. Royden, H. L. Real analysis. Second edition. Macmillan Publishing Company, New York, 1968. Math. Reviews number 90g:00004 (third ed.)
  24. Shreve, Steven E. An introduction to singular stochastic control. Stochastic differential systems, stochastic control theory and applications (Minneapolis, Minn., 1986), 513-528, IMA Vol. Math. Appl., 10, Springer, New York-Berlin, 1988. Math. Reviews number 89d:93122
  25. Shreve, S. E.; Soner, H. M. Optimal investment and consumption with transaction costs. Ann. Appl. Probab. 4 (1994), no. 3, 609-692. Math. Reviews number 95g:90013
  26. Soner, H. Mete; Shreve, Steven E. Regularity of the value function for a two-dimensional singular stochastic control problem. SIAM J. Control Optim. 27 (1989), no. 4, 876-907. Math. Reviews number 90h:93124
  27. Stockbridge, Richard H. Time-average control of martingale problems: a linear programming formulation. Ann. Probab. 18 (1990), no. 1, 206-217. Math. Reviews number 91b:49029
  28. Stockbridge, Richard H. Time-average control of martingale problems: existence of a stationary solution. Ann. Probab. 18 (1990), no. 1, 190-205. Math. Reviews number 91b:49028
  29. Weiss, A. Invariant measures of diffusions in bounded domains, Ph.D. dissertation, New York University, 1981. Math. Reviews number not avaiable.
















Research
Support Tool
Capture Cite
View Metadata
Printer Friendly
Context
Author Address
Action
Email Author
Email Others


Home | Contents | Submissions, editors, etc. | Login | Search | ECP

Electronic Journal of Probability. ISSN: 1083-6489