Elements of Information Theory

Author: Thomas M. Cover
Publisher: John Wiley & Sons
ISBN: 9781118585771
Release Date: 2012-11-28
Genre: Computers

The latest edition of this classic is updated with new problem sets and material The Second Edition of this fundamental textbook maintains the book's tradition of clear, thought-provoking instruction. Readers are provided once again with an instructive mix of mathematics, physics, statistics, and information theory. All the essential topics in information theory are covered in detail, including entropy, data compression, channel capacity, rate distortion, network information theory, and hypothesis testing. The authors provide readers with a solid understanding of the underlying theory and applications. Problem sets and a telegraphic summary at the end of each chapter further assist readers. The historical notes that follow each chapter recap the main points. The Second Edition features: * Chapters reorganized to improve teaching * 200 new problems * New material on source coding, portfolio theory, and feedback capacity * Updated references Now current and enhanced, the Second Edition of Elements of Information Theory remains the ideal textbook for upper-level undergraduate and graduate courses in electrical engineering, statistics, and telecommunications. An Instructor's Manual presenting detailed solutions to all the problems in the book is available from the Wiley editorial department.

Network Information Theory

Author: Abbas El Gamal
Publisher: Cambridge University Press
ISBN: 9781139503143
Release Date: 2011-12-08
Genre: Technology & Engineering

This comprehensive treatment of network information theory and its applications provides the first unified coverage of both classical and recent results. With an approach that balances the introduction of new models and new coding techniques, readers are guided through Shannon's point-to-point information theory, single-hop networks, multihop networks, and extensions to distributed computing, secrecy, wireless communication, and networking. Elementary mathematical tools and techniques are used throughout, requiring only basic knowledge of probability, whilst unified proofs of coding theorems are based on a few simple lemmas, making the text accessible to newcomers. Key topics covered include successive cancellation and superposition coding, MIMO wireless communication, network coding, and cooperative relaying. Also covered are feedback and interactive communication, capacity approximations and scaling laws, and asynchronous and random access channels. This book is ideal for use in the classroom, for self-study, and as a reference for researchers and engineers in industry and academia.

Information Theory Inference and Learning Algorithms

Author: David J. C. MacKay
Publisher: Cambridge University Press
ISBN: 0521642981
Release Date: 2003-09-25
Genre: Computers

Fun and exciting textbook on the mathematics underpinning the most dynamic areas of modern science and engineering.

A First Course in Information Theory

Author: Raymond W. Yeung
Publisher: Springer Science & Business Media
ISBN: 9781441986085
Release Date: 2012-12-06
Genre: Technology & Engineering

This book provides an up-to-date introduction to information theory. In addition to the classical topics discussed, it provides the first comprehensive treatment of the theory of I-Measure, network coding theory, Shannon and non-Shannon type information inequalities, and a relation between entropy and group theory. ITIP, a software package for proving information inequalities, is also included. With a large number of examples, illustrations, and original problems, this book is excellent as a textbook or reference book for a senior or graduate level course on the subject, as well as a reference for researchers in related fields.

Modern Coding Theory

Author: Tom Richardson
Publisher: Cambridge University Press
ISBN: 9781139469647
Release Date: 2008-03-17
Genre: Technology & Engineering

Having trouble deciding which coding scheme to employ, how to design a new scheme, or how to improve an existing system? This summary of the state-of-the-art in iterative coding makes this decision more straightforward. With emphasis on the underlying theory, techniques to analyse and design practical iterative coding systems are presented. Using Gallager's original ensemble of LDPC codes, the basic concepts are extended for several general codes, including the practically important class of turbo codes. The simplicity of the binary erasure channel is exploited to develop analytical techniques and intuition, which are then applied to general channel models. A chapter on factor graphs helps to unify the important topics of information theory, coding and communication theory. Covering the most recent advances, this text is ideal for graduate students in electrical engineering and computer science, and practitioners. Additional resources, including instructor's solutions and figures, available online: www.cambridge.org/9780521852296.

Detection and Estimation for Communication and Radar Systems

Author: Kung Yao
Publisher: Cambridge University Press
ISBN: 9781139619066
Release Date: 2013-01-17
Genre: Technology & Engineering

Covering the fundamentals of detection and estimation theory, this systematic guide describes statistical tools that can be used to analyze, design, implement and optimize real-world systems. Detailed derivations of the various statistical methods are provided, ensuring a deeper understanding of the basics. Packed with practical insights, it uses extensive examples from communication, telecommunication and radar engineering to illustrate how theoretical results are derived and applied in practice. A unique blend of theory and applications and over 80 analytical and computational end-of-chapter problems make this an ideal resource for both graduate students and professional engineers.

Information Theory and Statistics

Author: Solomon Kullback
Publisher: Courier Corporation
ISBN: 9780486142043
Release Date: 2012-09-11
Genre: Mathematics

Highly useful text studies logarithmic measures of information and their application to testing statistical hypotheses. Includes numerous worked examples and problems. References. Glossary. Appendix. 1968 2nd, revised edition.

Statistical Physics and Information Theory

Author: Neri Merhav
Publisher: Now Publishers Inc
ISBN: 9781601984067
Release Date: 2010
Genre: Technology & Engineering

Statistical Physics and Information Theory is a succinct in-depth review and tutorial of a subject that promises to lead to major advances in computer and communication security

Open Problems in Communication and Computation

Author: Thomas M. Cover
Publisher: Springer Science & Business Media
ISBN: 9781461248088
Release Date: 2012-12-06
Genre: Computers

Thomas M. Cover and B. Gopinatb The papers in this volume are the contributions to a special workshop on problems in communication and computation conducted in the summers of 1984 and 1985 in Morristown, New Jersey, and the summer of 1986 in Palo Alto. California. The structure of this workshop was unique: no recent results. no surveys. Instead. we asked for outstanding open prob~ lems in the field. There are many famous open problems, including the question P = NP?, the simplex conjecture in communication theory, the capacity region of the broadcast channel. and the two·helper problem in information theory. Beyond these well-defined problems are certain grand research goals. What is the general theory of information flow in stochastic networks? What is a comprehensive theory of computational complexity? What about a unification of algorithmic complexity and computational complex ity? Is there a notion of energy-free computation? And if so, where do information theory, communication theory, computer science, and physics meet at the atomic level? Is there a duality between computation and communication? Finally. what is the ultimate impact of algorithmic com plexity on probability theory? And what is its relationship to information theory? The idea was to present problems on the first day. try to solve them on the second day, and present the solutions on the third day. In actual fact, only one problem was solved during the meeting -- El Gamal's prob· lem on noisy communication over a common line.

Information Theory and Evolution

Author: John Avery
Publisher: World Scientific
ISBN: 9789814401241
Release Date: 2012
Genre: Computers

Information Theory and Evolution discusses the phenomenon of life, including its origin and evolution (and also human cultural evolution), against the background of thermodynamics, statistical mechanics, and information theory. Among the central themes is the seeming contradiction between the second law of thermodynamics and the high degree of order and complexity produced by living systems. This paradox has its resolution in the information content of the Gibbs free energy that enters the biosphere from outside sources, as the author will show. The role of information in human cultural evolution is another focus of the book. The first edition of Information Theory and Evolution made a strong impact on thought in the field by bringing together results from many disciplines. The new second edition offers updated results based on reports of important new research in several areas, including exciting new studies of the human mitochondrial and Y-chromosomal DNA. Another extensive discussion featured in the second edition is contained in a new appendix devoted to the relationship of entropy and Gibbs free energy to economics. This appendix includes a review of the ideas of Alfred Lotka, Frederick Soddy, Nicholas Georgiescu-Roegen and Herman E. Daly, and discusses the relevance of these ideas to the current economic crisis. The new edition discusses current research on the origin of life, the distinction between thermodynamic information and cybernetic information, new DNA research and human prehistory, developments in current information technology, and the relationship between entropy and economics.

Mathematical Foundations of Information Theory

Author: Aleksandr I?Akovlevich Khinchin
Publisher: Courier Corporation
ISBN: 9780486604343
Release Date: 1957
Genre: Mathematics

First comprehensive introduction to information theory explores the work of Shannon, McMillan, Feinstein, and Khinchin. Topics include the entropy concept in probability theory, fundamental theorems, and other subjects. 1957 edition.

Information Theory

Author: JV Stone
Publisher: Sebtel Press
ISBN: 9780956372857
Release Date: 2015
Genre: Information theory

Originally developed by Claude Shannon in the 1940s, information theory laid the foundations for the digital revolution, and is now an essential tool in telecommunications, genetics, linguistics, brain sciences, and deep space communication. In this richly illustrated book, accessible examples are used to introduce information theory in terms of everyday games like ‘20 questions’ before more advanced topics are explored. Online MatLab and Python computer programs provide hands-on experience of information theory in action, and PowerPoint slides give support for teaching. Written in an informal style, with a comprehensive glossary and tutorial appendices, this text is an ideal primer for novices who wish to learn the essential principles and applications of information theory.