Open Educational Resources


Open Educational Resources (OER) are freely accessible, openly licensed documents and media that are useful for teaching, learning, educational, assessment and research purposes.


Course reviews will be shown here


This course explores the ultimate limits to communication and computation, with an emphasis on the physical nature of information and information processing. Topics include: information and computation, digital signals, codes and compression, applications such as biological representations of information, logic circuits, computer architectures, and algorithmic information, noise, probability, error correction, reversible and irreversible operations, physics of computation, and quantum computation. The concept of entropy applied to channel capacity and to the second law of thermodynamics.

34 hours of lectures.

19 lectures.

Problem sets and solutions.


The course is taught from a set of notes written by Prof. Penfield, developed over several years of teaching this course. Each week these readings are supplemented by a plethora of online and print resources. (PDF - 4.4 MB)


Professor Paul L. Penfield, originally from Birmingham, Michigan, received his undergraduate degree from Amherst College, where he majored in physics and worked as the chief engineer on the college radio station. After receiving his Doctorate, Professor Penfield joined the MIT faculty in the Department of Electrical Engineering. From 1989–1999, he served as head of what had become the Department of Electrical Engineering and Computer Science (EECS). During this time, he oversaw the creation of the groundbreaking five-year degree program for the Master of Electrical Engineering.

Seth Lloyd  is a professor of mechanical engineering at the Massachusetts Institute of Technology. He refers to himself as a "quantum mechanic". His research area is the interplay of information with complex systems, especially quantum systems. He has performed seminal work in the fields of quantum computation and quantum communication, including proposing the first technologically feasible design for a quantum computer, demonstrating the viability of quantum analog computation, proving quantum analogs of Shannon's noisy channel theorem, and designing novel methods for quantum error correction and noise reduction.


Creative Commons License
Information and Entropy by Prof. Paul Penfield and Prof. Seth Lloyd is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at

Course content

  • Unit 1: Bits and Codes, Lecture 2

  • Unit 2: Compression, Lecture 1

  • Unit 3: Noise and Errors, Lecture 2

  • Unit 4: Probability, Lecture 1

  • Unit 4: Probability, Lecture 2

  • Unit 5: Communications, Lecture 1

  • Unit 5: Communications, Lecture 2

  • Unit 6: Processes, Lecture 1

  • Unit 7: Inference, Lecture 1

  • Unit 7: Inference, Lecture 2

  • Unit 8: Maximum Entropy, Lecture 1

  • Unit 8: Maximum Entropy, Lecture 2

  • Unit 10: Physical Systems, Lecture 1

  • Unit 10: Physical Systems, Lecture 3

  • Unit 11: Energy, Lecture 1

  • Unit 11: Energy, Lecture 2

  • Unit 12: Temperature, Lecture 1

  • Unit 12: Temperature, Lecture 2

  • Unit 13: Quantum Information, Lecture 1

Interested? Enroll to this course right now.

There is more to learn