An Introduction to Kolmogorov Complexity and Its Applications

by Ming Li, Paul Vitányi

Overview

This must-read textbook presents an essential introduction to Kolmogorov complexity (KC), a central theory and powerful tool in information science that deals with the quantity of information in individual objects. The text covers both the fundamental concepts and the most important practical applications, supported by a wealth of didactic features.

This thoroughly revised and enhanced fourth edition includes new and updated material on, amongst other topics, the Miller-Yu theorem, the Gács-Kučera theorem, the Day-Gács theorem, increasing randomness, short lists computable from an input string containing the incomputable Kolmogorov complexity of the input, the Lovász local lemma, sorting, the algorithmic full Slepian-Wolf theorem for individual strings, multiset normalized information distance and normalized web distance, and conditional universal distribution.

Topics and features: describes the mathematical theory of KC, including the theories of algorithmic complexity and algorithmic probability; presents a general theory of inductive reasoning and its applications, and reviews the utility of the incompressibility method; covers the practical application of KC in great detail, including the normalized information distance (the similarity metric) and information diameter of multisets in phylogeny, language trees, music, heterogeneous files, and clustering; discusses the many applications of resource-bounded KC, and examines different physical theories from a KC point of view; includes numerous examples that elaborate the theory, and a range of exercises of varying difficulty (with solutions); offers explanatory asides on technical issues, and extensive historical sections; suggests structures for several one-semester courses in the preface.

As the definitive textbook on Kolmogorov complexity, this comprehensive and self-contained work is an invaluable resource for advanced undergraduate students, graduate students, and researchers in all fields of science.

Product Details

ISBN-13:	9783030112974
Publisher:	Springer International Publishing
Publication date:	06/12/2019
Series:	Texts in Computer Science
Edition description:	Fourth Edition 2019
Pages:	834
Product dimensions:	7.01(w) x 10.00(h) x (d)

About the Author

Dr. Paul M.B. Vitányi is a CWI Fellow at the Netherlands National Research Institute for Mathematics and Computer Science (CWI), and a Professor of Computer Science at the University of Amsterdam. Dr. Ming Li is Canada Research Chair in Bioinformatics and University Professor at the University of Waterloo, ON, Canada.

—-

Textbook & Academic Authors Association 2020 McGuffey Longevity Award Winner!

The judges said:

"An Introduction to Kolmogorov complexity and Its Applications has been an outstanding textbook and comprehensive reference for on information complexity for over twenty years. This new edition continues that tradition by laying a terrific foundation in the early chapters for the more advanced theories and concepts that follow. Each new theorem and corollary flows naturally and logically from what came before."

Table of Contents

Preliminaries.- Algorithmic Complexity.- Algorithmic Prefix Complexity.- Algorithmic Probability.- Inductive Reasoning.- The Incompressibility Method.- Resource-Bounded Complexity.- Physics, Information, and Computation.

Editorial Reviews

From the reviews of the second edition:

"We are indeed in the information age and the scientific exploration of information and the laws that govern its behavior has taken center stage in the dramatic development of sciences. Kolmogorov complexity is a central concept and a powerful tool in the understanding of the quantitative nature of information and its processing and transmission. Li and Vitanyi's book beautifully captures the elegance of these ideas, their relevance to more of computer science and their theoretical as well as practical applications. The basic concepts of Kolmogorov complexity should be understood by any technically educated person, and they should be studied by all computer scientists. Li and Vitanyi have provided an ideal book for the exploration of a deep, beautiful and important part of the computer science."

Juris Hartmanis, (Turing Award Winner 1993), NSF, Washington D.C.

"Special attention is paid to the theory underlying inductive inference and its potential applications. The book is likely to remain the standard treatment of Kolmogorov complexity for a long time."

Jorma J. Rissanen, IBM Research, California

"The book of Li and Vitanyi is unexcelled."

Ray J. Solomonoff, Oxbridge Research, Cambridge, Massachusetts

"The book is outstanding . . . the authors did their job unbelievably well...necessary reading for all kinds of readers from undergraduate students to top authorities in the field."

Vladimir A. Uspensky and Alexander K. Shen, Journal of Symbolic Logic

"It is clear that this book will become 'the' Kolmogorov complexity book."

Marius Zimand, Mathematical Reviews

From the reviews of the third edition:

"Kolmogorov complexity, algorithmic information theory, minimum description length, and other information-based disciplines have experienced a phenomenal explosion in the last decade. … is thisthird edition worth reading? Yes, it is. The authors have added an extra 204 pages, distributed throughout the book … . Eight new figures were also added. Most impressively, 301 new references were added, bringing the total to 820. … It is sure to maintain its reputation … ." (Jacques Carette, ACM Computing Reviews, April, 2009)

From the Publisher