Information Theory
This unit introduces the Shannon Model of information and the Shannon-Weaver model of information transmission. It shows how the amount of information in a message can be quantified.
Objectives
Upon completion of this lesson, you will be able to
Upon completion of this lesson, you will be able to
- describe different kinds of information
- explain the role of noise
- calculate the information content of a message
- differentiate between subjective, empirical, and logical approaches to probability
- relate the entropy of information
- discuss the value of information
Required Readings
- Wikipedia: Shannon-Weaver Model of Communication
- Intuitive Explanation of Information Entropy
- What is entropy and information gain? An Example of Entropy in Machine Learning & Decision Trees.
Suggested Readings
- Carr, N. (2008). Is Google Making us Stupid?, The Atlantic. July 1, 2008.
- Semantic Conceptions of Information, Stanford Encyclopedia of Philosophy, October 5, 2005.
