ENGR 76: Information Science and Engineering

Images saved using the JPEG format take as little as 10% of the memory needed to store the raw image. The wireless environment that your cellphone uses — 4G, 5G, Wi-Fi and Bluetooth — is noisy and error-prone, and yet we still manage to use wireless transmission to communicate reliably. Our ability to compress video to as little as 0.1% of its raw size, with little perceptible difference, allows us to stream high-quality video over the internet. Your computer's hard drive or solid state drive often suffers errors in storage and has to correct itself. How is this possible? What allows us to compress information and cut through errors in communication and storage? What is information?

In this course, we will learn about the principles and techniques underlying the design of modern information, communication, and decision-making systems. How do we measure how much information we have? How can we represent the same information with less memory? How can we encode information to reliably communicate it over error-prone media? We will introduce the basic notions required to address these questions, and consider applications from those mentioned above, to machine learning, and COVID testing.

Students will get a hands-on appreciation of these concepts by undertaking two main projects. In the first, students will develop a system for compressing images, and explore the issues and trade-offs involved. In the second, students will communicate data over a speaker-to-microphone audio medium, and explore encoding strategies for making this system reliable.

Course Staff

Instructor

Course Assistants

• Surin Ahn (Head CA): surinahn@stanford.edu

• Rashid Al-Abri: rashid@stanford.edu

• Nandita Bhashkar: nanbhas@stanford.edu

• Siddharth Chandak: siddharth.chandak@stanford.edu

• Wei-Ning Chen: wnchen@stanford.edu

• Helen Deng: yiman@stanford.edu

• Edward Kim: edkim36@stanford.edu

• Jay Mardia: jmardia@stanford.edu

• Nathan Marks: nsmarks@stanford.edu

• Tristan Peng: tristan.peng@stanford.edu

• Zachary Shah: zshah9@stanford.edu

• Guilherme David Santos Silva: gdavid@stanford.edu

Times and Places

• Lectures: Tue, Thu 12:00 PM–1:20 PM. STLC 111. Live-streamed on Zoom and recorded; recordings will be available on Canvas after the lecture.

• Project Discussion Sessions: Fri 4:15 PM–5:00 PM on Zoom. Recordings available on Canvas after the session.

• Office Hours: See Canvas.

Prerequisites

The course does not require any specific preparation. We will develop the mathematical language and tools we need from probability theory and Fourier analysis from scratch. The project will involve some coding and CS 106A or equivalent is recommended.

Projects

The course will consist of two projects, which will be divided into weekly subprojects. The projects will explore how concepts taught in class can be used to build real systems for storing and communicating information efficiently. The projects will involve coding and experimentation, with appropriate guidance. The projects will have both structured and open-ended components, allowing you to internalize the concepts and play with new ideas.

The first project will focus on compression of images and audio, which we encounter every day when we use JPEG or MP3 files. You'll learn about compression techniques like Huffman coding, frequency domain transforms, as well as ideas behind lossy compression which allow substantial space savings without perceptible loss in quality.

The second project will focus on communication through a noisy physical medium. We'll work with an audio-based channel, transmitting data from the speaker for your laptop and receiving (distorted) audio through the microphone. We'll then make this system robust to various noise sources in the environment with the use of error correction codes, frequency-domain filtering and machine learning techniques. The project will also introduce basic concepts of data transmission through physical channels, including (de)modulation and synchronization.

Quizzes

We will release a small quiz on Gradescope after each lecture. The quizzes will be untimed and allow multiple submissions.

Grading

70% Projects, 30% Quizzes