|
|
|
| | | | | |
|
|
|
|
|
Compound Poisson Approximation via Information Functionals
|
A. D. Barbour, Angewandte Mathematik, Universität Zürich Oliver Johnson, Department of Mathematics, University of Bristol Ioannis Kontoyiannis, Department of Informatics, Athens University of Economics & Business Mokshay Madiman, Department of Statistics, Yale University |
Abstract
An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Nonasymptotic bounds are derived for the distance between the distribution of a sum of independent integer-valued random variables and an appropriately chosen compound Poisson law. In the case where all summands have the same conditional distribution given that they are non-zero, a bound on the relative entropy distance between their sum and the compound Poisson distribution is derived, based on the data-processing property of relative entropy and earlier Poisson approximation results. When the summands have arbitrary distributions, corresponding bounds are derived in terms of the total variation distance. The main technical ingredient is the introduction of two "information functionals,'' and the analysis of their properties. These information functionals play a role analogous to that of the classical Fisher information in normal approximation. Detailed comparisons are made between the resulting inequalities and related bounds.
|
Full text: PDF
Pages: 1344-1369
Published on: August 31, 2010
|
Bibliography
- Aldous, David. Probability approximations via the Poisson clumping heuristic.
Applied Mathematical Sciences, 77. Springer-Verlag, New York, 1989. xvi+269 pp. ISBN: 0-387-96899-7 MR0969362 (90k:60004)
- Artstein, Shiri; Ball, Keith M.; Barthe, Franck; Naor, Assaf. On the rate of convergence in the entropic central limit theorem.
Probab. Theory Related Fields 129 (2004), no. 3, 381--390. MR2128238 (2006a:94023)
- Artstein, Shiri; Ball, Keith M.; Barthe, Franck; Naor, Assaf. Solution of Shannon's problem on the monotonicity of entropy.
J. Amer. Math. Soc. 17 (2004), no. 4, 975--982 (electronic). MR2083473 (2005e:94047)
- Barbour, A. D.; Chen, Louis H. Y. Stein's method and applications.
Proceedings of the workshop held in Singapore, July 28--August 31,
2003.
Edited by A. D. Barbour and Louis H. Y. Chen.
Lecture Notes Series. Institute for Mathematical Sciences. National
University of Singapore, 5. Published jointly by Singapore University Press, Singapore; and World
Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, 2005. xx+297 pp. ISBN: 981-256-281-8 MR2201882 (2006h:60011)
- Barbour, A. D.; Chen, Louis H. Y.; Loh, Wei-Liem. Compound Poisson approximation for nonnegative random variables via
Stein's method.
Ann. Probab. 20 (1992), no. 4, 1843--1866. MR1188044 (93k:60044)
- Barbour, A. D.; Chryssaphinou, O. Compound Poisson approximation: a user's guide.
Ann. Appl. Probab. 11 (2001), no. 3, 964--1002. MR1865030 (2002k:60029)
- Barbour, A. D.; Holst, Lars; Janson, Svante. Poisson approximation.
Oxford Studies in Probability, 2. Oxford Science Publications.
The Clarendon Press, Oxford University Press, New York, 1992. x+277 pp. ISBN: 0-19-852235-5 MR1163825 (93g:60043)
- Barron, Andrew R. Entropy and the central limit theorem.
Ann. Probab. 14 (1986), no. 1, 336--342. MR0815975 (87h:60048)
- Bobkov, S. G.; Ledoux, M. On modified logarithmic Sobolev inequalities for Bernoulli and Poisson
measures.
J. Funct. Anal. 156 (1998), no. 2, 347--365. MR1636948 (99e:60051)
- Borisov, I. S.; Vorozheĭkin, I. S. Accuracy of approximation in the Poisson theorem in terms of $chisp
2$ distance.
(Russian) Sibirsk. Mat. Zh. 49 (2008), no. 1, 8--22; translation in Sib. Math. J. 49 (2008), no. 1, 5--17 MR2400567 (2009c:60078)
- Čekanavičius, V.; Roos, B. An expansion in the exponent for compound binomial approximations.
Liet. Mat. Rink. 46 (2006), no. 1, 67--110; translation in Lithuanian Math. J. 46 (2006), no. 1, 54--91 MR2251442 (2008k:62042)
- Cover, Thomas M.; Thomas, Joy A. Elements of information theory.
Wiley Series in Telecommunications. A Wiley-Interscience Publication.
John Wiley & Sons, Inc., New York, 1991. xxiv+542 pp. ISBN: 0-471-06259-6 MR1122806 (92g:94001)
- Csiszár, Imre; Körner, János. Information theory.
Coding theorems for discrete memoryless systems.
Probability and Mathematical Statistics. Academic Press, Inc. [Harcourt Brace Jovanovich, Publishers], New
York-London, 1981. xi+452 pp. ISBN: 0-12-198450-8 MR0666545 (84e:94007)
- Diaconis, Persi; Holmes, Susan
Stein's method: expository lectures and applications.
Papers from the Workshop on Stein's Method held at Stanford University,
Stanford, CA, 1998.
Edited by Persi Diaconis and Susan Holmes.
Institute of Mathematical Statistics Lecture Notes---Monograph Series,
46. Institute of Mathematical Statistics, Beachwood, OH, 2004. vi+139 pp. ISBN: 0-940600-62-5 MR2118599 (2005i:62008)
- Erhardsson, Torkel. Stein's method for Poisson and compound Poisson approximation.
An introduction to Stein's method,
61--113, Lect. Notes Ser. Inst. Math. Sci. Natl. Univ. Singap., 4, Singapore Univ. Press, Singapore, 2005. MR2235449
- Harremoës, Peter. Binomial and Poisson distributions as maximum entropy
distributions.
IEEE Trans. Inform. Theory 47 (2001), no. 5, 2039--2041. MR1842536 (2002e:94062)
- Johnson, Oliver. Information theory and the central limit theorem.
Imperial College Press, London, 2004. xiv+209 pp. ISBN: 1-86094-473-6 MR2109042 (2006m:60001)
- Johnson, Oliver. Log-concavity and the maximum entropy property of the Poisson
distribution.
Stochastic Process. Appl. 117 (2007), no. 6, 791--802. MR2327839 (2008k:62012)
- Johnson, Oliver; Barron, Andrew. Fisher information inequalities and the central limit theorem.
Probab. Theory Related Fields 129 (2004), no. 3, 391--409. MR2128239 (2006b:62018)
- Johnson, Oliver; Kontoyiannis, Ioannis;
Madiman, Mokshay.
Log-concavity, ultra-log-concavity and a maximum
entropy property of discrete compound Poisson measures.
Preprint, October 2009. Earlier version online
at {tt arXiv:0805.4112v1}, May 2008.
- Johnstone, Iain M.; MacGibbon, Brenda. Une mesure d'information caractérisant la loi de Poisson.
(French) [An information measure characterizing the Poisson distribution] Séminaire de Probabilités, XXI,
563--573, Lecture Notes in Math., 1247, Springer, Berlin, 1987. MR0942005 (89h:62024)
- Kagan, Abram. A discrete version of the Stam inequality and a characterization of the
Poisson distribution.
J. Statist. Plann. Inference 92 (2001), no. 1-2, 7--12. MR1809692 (2001k:62019)
- Kontoyiannis, Ioannis; Harremoës, Peter; Johnson, Oliver. Entropy and the law of small numbers.
IEEE Trans. Inform. Theory 51 (2005), no. 2, 466--472. MR2236061 (2007j:94025)
- Kontoyiannis, I.; Madiman, M. Measure concentration for compound Poisson distributions.
Electron. Comm. Probab. 11 (2006), 45--57 (electronic). MR2219345 (2007h:60011)
- Le Cam, Lucien. An approximation theorem for the Poisson binomial distribution.
Pacific J. Math. 10 1960 1181--1197. MR0142174 (25 #5567)
- Le Cam, Lucien. On the distribution of sums of independent random variables.
1965 Proc. Internat. Res. Sem., Statist. Lab., Univ. California, Berkeley,
Calif.
pp. 179--202 Springer-Verlag, New York MR0199871 (33 #8011)
- Madiman, Mokshay. Topics in information theory, probability, and statistics.
Thesis (Ph.D.)--Brown University.
ProQuest LLC, Ann Arbor, MI, 2006. 166 pp. ISBN: 978-0542-82065-6 MR2624419
- Madiman, Mokshay; Barron, Andrew. Generalized entropy power inequalities and monotonicity properties of
information.
IEEE Trans. Inform. Theory 53 (2007), no. 7, 2317--2329. MR2319376 (2008k:94024)
- Matsunawa, T. Some strong $varepsilon$-equivalence of random variables.
Ann. Inst. Statist. Math. 34 (1982), no. 2, 209--224. MR0666413 (84c:60044)
- Michel, R.
An improved error bound for the compound Poisson approximation of a
nearly homogeneous portfolio.
ASTIN Bull. 17 (1987), 165--169.
- Romanowska, Malgorzata. A note on the upper bound for the distrance in total variation between
the binomial and the Poisson distribution.
Statistica Neerlandica 31 (1977), no. 3, 127--130. MR0467889 (57 #7739)
- Roos, Bero. Sharp constants in the Poisson approximation.
Statist. Probab. Lett. 52 (2001), no. 2, 155--168. MR1841404 (2002e:60031)
- Roos, Bero. Kerstan's method for compound Poisson approximation.
Ann. Probab. 31 (2003), no. 4, 1754--1771. MR2016599 (2004m:60047)
- Topsøe, Flemming. Maximum entropy versus minimum risk and applications to some classical
discrete distributions.
IEEE Trans. Inform. Theory 48 (2002), no. 8, 2368--2376. MR1930296 (2004c:94059)
- Tulino, Antonia M.; Verdú, Sergio. Monotonic decrease of the non-Gaussianness of the sum of independent
random variables: a simple proof.
IEEE Trans. Inform. Theory 52 (2006), no. 9, 4295--4297. MR2298559 (2008e:60134)
- Vervaat, W. Upper bounds for the distance in total variation between the binomial
or negative binomial and the Poisson distribution.
Statistica Neerlandica 23 1969 79--86. MR0242235 (39 #3568)
- Wu, Liming. A new modified logarithmic Sobolev inequality for Poisson point
processes and several applications.
Probab. Theory Related Fields 118 (2000), no. 3, 427--438. MR1800540 (2002f:60109)
- Yu, Yaming. On the entropy of compound distributions on nonnegative integers.
IEEE Trans. Inform. Theory 55 (2009), no. 8, 3645--3650. MR2598065 (Review)
|
|
|
|
|
|
|
| | | | |
Electronic Journal of Probability. ISSN: 1083-6489 |
|