A Toolbox for Refined Information-Theoretic Analyses with Applications

by Neri Merhav, Nir Weinberger

Overview

This monograph offers a toolbox of mathematical techniques that have been effective and widely applicable in information-theoretic analyses. The first tool is a generalization of the method of types to Gaussian settings, and then to general exponential families. The second tool is Laplace and saddle-point integration, which allow to refine the results of the method of types, and is capable of obtaining various precise asymptotic results.

The third is the type class enumeration method, a principled method to evaluate the exact random-coding exponent of coded systems, which results in the best known exponent in various problem settings. The fourth is a subset of tools aimed at evaluating the expectation of non-linear functions of random variables, either via integral representations, by a refinement of Jensen's inequality via change-of-measure, by complementing Jensen's inequality with a reversed inequality, or by a class of generalized Jensen's inequalities that are applicable for functions beyond convex/concave. Various examples of all these tools are provided throughout the monograph.

Product Details

ISBN-13:	9781638285007
Publisher:	Now Publishers
Publication date:	01/30/2025
Series:	Foundations and Trends(r) in Engineering , #5
Pages:	202
Product dimensions:	6.14(w) x 9.21(h) x 0.43(d)

Table of Contents

1. Introduction
2. Extension of the Method of Types to Continuous Alphabets
3. The Laplace Method of Integration and the Saddle-Point Method
4. The Type Class Enumeration Method
5. Manipulating Expectations of Nonlinear Functions of Random Variables
6. Summary, Outlook and Open Issues
Acknowledgements
Appendices
References