Volume 2,  Issue 2, 2001

Article 25

BOUNDS FOR ENTROPY AND DIVERGENCE FOR DISTRIBUTIONS OVER A TWO-ELEMENT SET

FLEMMING TOPSOE

DEPARTMENT OF MATHEMATICS
UNIVERSITY OF COPENHAGEN
UNIVERSITETSPARKEN 5
DK-2100 COPENHAGEN, DENMARK.
E-Mail: topsoe@math.ku.dk

Received 6 November, 2000; accepted 6 March, 2001.
Communicated by: F. Hansen


ABSTRACT.   Three results dealing with probability distributions (p,q) over a two-element set are presented. The two first give bounds for the entropy function H(p,q) and are referred to as the logarithmic and the power-type bounds, respectively. The last result is a refinement of well known Pinsker-type inequalities for information divergence. The refinement readily extends to general distributions,  but the key case to consider involves distributions on a two-element set.

The discussion points to some elementary, yet non-trivial problems concerning seemingly simple concrete functions.


Key words:
Entropy, divergence, Pinsker's inequality.

2000 Mathematics Subject Classification:
94A17, 26D15.


Download this article (PDF):

Suitable for a printer:    

Suitable for a monitor:        

To view these files we recommend you save them to your file system and then view by using the Adobe Acrobat Reader. 

That is, click on the icon using the 2nd mouse button and select "Save Target As..." (Microsoft Internet Explorer) or "Save Link As..." (Netscape Navigator).

See our PDF pages for more information.

Other papers in this issue

Power-Monotone Sequences and Fourier Series with Positive Coefficients
J. Nemeth

On Some Fundamental Integral Inequalities and their Discrete Analogues
B.G.  Pachpatte

Subharmonic Functions and their Riesz Measure
Raphaele Supper

A Priori Estimate for a System of Differential Operators
Chikh Bouzar

Improved Inclusion-Exclusion Inequalities for Simplex and Orthant Arrangements
Daniel Q. Naiman and Henry P. Wynn

Monotonic Refinements of a Ky Fan Inequality
Kwok K. Chong

Sub-super Solutions and the Existence of Extremal Solutions in Noncoercive Variational Inequalities
V.K. Le

Refinements of Carleman's Inequality
Bao-Quan Yuan 

Inequalities related to the Chebychev Functional Involving Integrals Over Different Intervals
I. Budimir, P. Cerone, and  J. Pecaric

Some Distortion Inequalities Associated with the Fractional Derivatives of Analytic and Univalent Functions
H.M. Srivastava, Yi Ling and Gejun Bao

Necessary and Sufficient Condition for Existence and Uniqueness of the Solution of Cauchy Problem for Holomorphic Fuchsian Operators
Mekki Terbeche

Bounds for Entropy and Divergence for Distributions over a Two-Element Set
Flemming Topsoe

A Pick Function Related to an Inequality for the Entropy Function
Christian Berg


Other issues

 

2000 School of Communications and Informatics, Victoria University of Technology. All rights reserved.
JIPAM is published by the School of Communications and Informatics which is part of the Faculty of Engineering and Science, located in Melbourne, Australia. All correspondence should be directed to the editorial office.

Copyright/Disclaimer