Issue 2, 2001
BOUNDS FOR ENTROPY AND DIVERGENCE FOR DISTRIBUTIONS OVER A TWO-ELEMENT SET
DEPARTMENT OF MATHEMATICS
UNIVERSITY OF COPENHAGEN
DK-2100 COPENHAGEN, DENMARK.
Received 6 November, 2000; accepted 6 March, 2001.
Communicated by: F. Hansen
Three results dealing with probability distributions (p,q) over a
two-element set are presented. The two first give bounds for the entropy
function H(p,q) and are referred to as the logarithmic and the
power-type bounds, respectively. The last result is a refinement of
well known Pinsker-type inequalities for information divergence.
The refinement readily extends to general distributions,
but the key case to consider involves distributions on a two-element set.
The discussion points to some elementary, yet non-trivial
problems concerning seemingly simple concrete functions.
2000 Mathematics Subject
Download this article (PDF):
Suitable for a printer:
Suitable for a monitor:
|To view these files we
recommend you save them to your file system and then view by using
the Adobe Acrobat Reader.
That is, click on the icon using the 2nd mouse button and
select "Save Target As..." (Microsoft Internet
Explorer) or "Save Link As..." (Netscape
See our PDF pages for more