ENGR 76: Information Science and Engineering

Images saved using the JPEG format take as little as 10% of the memory needed to store the raw image. The wireless environment that your cellphone uses — 4G, 5G, Wi-Fi and Bluetooth — is noisy and error-prone, and yet we still manage to use wireless transmission to communicate reliably. Our ability to compress video to as little as 0.1% of its raw size, with little perceptible difference, allows us to stream high-quality video over the internet. Your computer's hard drive or solid state drive often suffers errors in storage and has to correct itself. How is this possible? What allows us to compress information and cut through errors in communication and storage? What is information?

In this course, we will learn about the principles and techniques underlying the design of modern information, communication, and decision-making systems. How do we measure how much information we have? How can we represent the same information with less memory? How can we encode information to reliably communicate it over error-prone media? We will introduce the basic notions required to address these questions, and consider various applications ranging from those mentioned above, to machine learning.

Students will get a hands-on appreciation of these concepts by undertaking two main projects. In the first, students will develop a system for compressing images, and explore the issues and trade-offs involved. In the second, students will communicate data over a speaker-to-microphone audio medium, and explore encoding strategies for making this system reliable.

Course Staff

Instructors

• Siddharth Chandak (PI)
  chandaks@stanford.edu

• Ayfer Özgür
  aozgur@stanford.edu

Course Assistants

• Andy Dong

Contact and Communication

Please contact the PI if you have any questions about the course. This includes queries regarding logistics, accommodations, deadlines and the actual content of the course. You can attend his office hours, send an email or make a private post on Ed. We encourage you to first go through this page, the FAQs, and the pinned announcements on Ed before making any individual queries.

All course material will be shared on Canvas and official announcements and communication will happen over Ed. Any technical questions related to the course content should be asked on Ed. All assignment submissions will be done on Gradescope.

Times and Places

• Lectures: Tue, Thu 9:00 AM–10:30 AM. STLC 111. The lectures will not be recorded.

• Problem Set Discussion Section: See Canvas.

• Project Help Sessions: Will be uploaded to Canvas along with each project.

• Office Hours: See Canvas.

Prerequisites

In terms of mathematical background, the course has Math 19 and Math 20 as prerequisites. We will develop the mathematical language and tools we need from probability theory and Fourier analysis from scratch. The projects will involve coding and experience with Python and CS 106A (or equivalent) is required.

Projects

The course will consist of two projects, which will be divided into weekly subprojects. The projects will explore how concepts taught in class can be used to build real systems for storing and communicating information efficiently. The projects will involve coding and experimentation, with appropriate guidance. The projects will have both structured and open-ended components, allowing you to internalize the concepts and play with new ideas.

The first project will focus on compression of images and audio, which we encounter every day when we use JPEG or MP3 files. You'll learn about compression techniques like Huffman coding, frequency domain transforms, as well as ideas behind lossy compression which allow substantial space savings without perceptible loss in quality.

The second project will focus on communication through a noisy physical medium. We will work with an audio-based channel, transmitting data from the speaker for your laptop and receiving (distorted) audio through the microphone. We'll then make this system robust to various noise sources in the environment with the use of error correction codes, frequency-domain filtering and machine learning techniques. The project will also introduce basic concepts of data transmission through physical channels, including (de)modulation and synchronization.

Logistics: Each project will comprise of weekly assignments which build on work done in prior assignments. There will be a total of 8 weekly project assignments - Project 0, Project 1a to 1d and Project 2a to 2c. Project 0 will just be a basic Python setup and tutorial. Each weekly project assignment will be released on Friday and will be due the next Friday. The solutions will be uploaded on Monday, so that they can be used for the next weekly project assignment. There will be no project assignment due in the week of the exam.

Project Help Sessions: A project help session will be uploaded to Canvas along with the project, where the CA will provide initial guidance on how to approach it.

Exam

An in-class exam will be conducted on February 26 (week 8 of the quarter). This exam will cover both the theoretical concepts taught in lectures and the practical concepts covered in project assignments. The questions corresponding to the theoretical concepts will be similar to the ones in mini-PSets and the associated discussion sections.

Mini Problem Sets

We will have weekly mini-PSets on Gradescope. These will cover concepts taught in that week's class and will be released on Thursday. They will be due on Wednesday. These will be untimed and will allow multiple submissions. Students are allowed to collaborate on these problems, and should focus on learning the concepts as similar concepts will be tested in the exam.

PSet Discussion Section: To help students with the problem sets, a discussion section will be held every Monday. Andy (CA) will discuss solutions to problems similar to the ones in the mini-PSet. These sessions will not be recorded, but problems and solutions from these discussion sections will be released on Canvas.

Grading

49% Projects, 20% Mini-PSets, 30% Exam, 1% Attendance (and participation)

Late Policy

• Mini-PSets: We will drop your lowest mini-PSet score. No other extensions and exceptions will be provided.

• Projects: Weekly project assignments will be released each Friday, and they will be due the next Friday. The solutions will be uploaded on Monday. Each student will have a total of 6 late days for Project 0+1 (i.e., 0, 1a, … , 1d) and 4 late days for Project 2 (i.e., 2a, … , 2c). Once these late days are exhausted, any assignments turned in late will be penalized 10% per late day. However, no assignment will be accepted more than 3 days after its due date, at which point the solutions will be released automatically. Any fraction of a late day counts as one late day. These free late days are to be used only in cases of emergency and sickness. As assignments are released and due weekly, we strongly recommend using late days sparingly to avoid falling behind on subsequent deadlines. No exceptions to this policy will be granted to maintain structure and fairness in the class.

Academic Accommodations

If you need an academic accommodation, you should initiate a request with the OAE. We request you to share your accommodation letters with the instructor and the head CAs as soon as possible. Due to the weekly structure of the projects, we cannot provide an extension of more than 3 days to any student under any circumstances. Similarly, only an extension of 1 day will be provided for the mini problem sets. We will grant an extension only if we receive your request before the deadline. Please send an email to the instructor for any communication regarding extensions.