Final Project

The final project of the class requires original work and is intended to give an introduction to the research process. It will require more effort than the first two assignments. We would like you to start early, send e-mail updates often to the teaching staff, and manage the project to finish by the end of the quarter. You may choose to do either:

  • an original research project in the area of computer networking or

  • reproduce a result from a networking paper that hasn't been reproduced in previous iterations of CS244.

For a reproduction project, here are some sample questions your project might try to answer:

  • Does the primary result in the paper hold up? Can you generate a plot or graph that looks the same as one of the main figures in the original paper, and explain where it came from?

  • What kind of “reproduction” is appropriate? Most plots can be reproduced if you simply re-plot the authors’ original data on the same axes! Can the data be generated afresh? Are the assumptions behind the data-generating process, and the way the system was compared with prior work, realistic?

  • What was difficult to reproduce? Were you able to package the contribution of the paper in a way that makes it easier for future “reproducteurs” to reproduce the major results?

  • What happens if you vary a parameter the original experimenter didn’t consider?

  • Having reproduced the primary result, can you now extend or improve the work?

  • Can you find a simplification to the system that still preserves most of the claimed improvements? If so, this can indicate that the original authors are mistaken about why their system outperforms prior work.

See the “Searching for Interesting Papers” section below for ideas on where to look for recent or interesting papers. The previous class projects are on the Reproducing Network Research site.

An original project can be a proof of concept of a novel networked system, an extension of existing work, or any research project that you could realistically extend to a conference-quality paper. Some types of conference papers include:

  • New systems. Questions to answer: how and why did you design the system the way you did? How does the system perform in comparison with prior work in similar areas? Why does it perform as it does (i.e., which design decisions account for what benefits)? What are the system’s weaknesses and limitations?

  • Measurements. What can you learn (about the Internet, about another network, about applications, about people, about spam email, about cryptocurrency, about online video games, about social science) by using the tools of computer networking? What kind of measurement tools or techniques did you use, and what did they tell you about something in the real world? How can somebody reproduce your measurement? What are some weaknesses or potential confounders of your measurement?

  • Any other creative project using the tools or problem domain of computer networking.

Much of this project is intentionally underspecified, and you should pick a project that you have sufficient time and experience to accomplish. Depending on what you propose, your project could require using a profiler, modifying the Linux kernel, implementing a custom simulator, learning a new scripting language, proving an interesting new result, or coordinating a software development effort across multiple group members. You will likely need to read research papers outside of the class reading list or the particular paper you are trying to reproduce. (Check the references of any paper for the most closely related prior work, and read or skim the prior work!)

Your final deliverable will be a 4-6 page report about your project. You will be expected to:

  • Introduce the project, placing it in context, giving an overview, and summarizing your major contributions or findings

  • Discuss prior work and the landscape of the area

  • Explain your methods and/or the design of your system

  • Present the experimental setup in detail (assuming there are experiments, rather than proofs)

  • Describe the results in figures and text

  • Discuss the meaning of the results and what the reader should take away

  • Describe the limitations and weaknesses of your system, measurement, or effort at replication

  • Propose a plan for future work in the area. What would you do next if you had more time?

For reproduction projects, you will post your final writeup on the Reproducing Network Research site (please email the instructors if you would like an exception to this practice).

Be wise and mindful of your (other) deadlines and other commitments before you decide your deliverable.

Deadlines and Deliverables

Project Proposal due Wednesday, May 2 2018, 3pm:

  • Before you submit your proposal, we encourage you to talk to us (e.g. in office hours, or in lecture).

  • Pick either:

    • a paper to reproduce OR

    • an original project idea

  • Write up to one page describing:

    • Who is in your group

    • The particular results you would like to replicate, or the overall goal of your original project.

    • Details about what you will implement or measure

    • A brief outline of incremental steps you will need to do to finish the project as well as a timeline. The goal here is to convince both us and yourself that your project is neither too small nor too big.

  • Submit proposal on gradescope

Intermediate project report due Tuesday, May 22 2018, 11:59pm:

  • Should have the structure of the final report. Please check Piazza for more information.

  • Complete introduction written.

  • Status of the project.

  • Plan for the remaining time

Presentation Proposal (optional) due Sunday, June 3, 2017, 5pm:

  • Submit presentation proposals. More info to come.

Presentations held on Wednesday, June 6 2018:

  • More info to come.

Final Report due Saturday, June 9, 5pm:

  • Mini-paper: 4-6 pages. This is your final report. More info to come.

Regular Check-ins and Meetings

During the course of the assignment, you’ll have one mandatory meeting with the staff around the time the milestone is due. The goal is to make sure you’re on track to complete the assignment. In addition, we will expect you to send an email each week to the staff describing (1) what you did this week and (2) what you need to do next week to stay on track. These emails will not be graded, the idea is to make sure you are making incremental progress. You may also visit the office hours of your TA or any instructor.

Presentations (optional)

More info to come

Grading

You’ll be graded out of 25 points total. The grades will be based on the following:

  • Intermediate progress report

  • Final report

  • Quality of experimental setup and results

  • If you end up doing a presentation, this will only boost your grade.

More than the success of the final outcome, we look for how thorough your efforts have been.

Searching for Interesting Papers

We recommend searching for interesting papers published in recent years in top systems and networking conferences, or by searching through the references section of another paper in the field. Relevant venues include:

  • SIGCOMM: major conference in computer networking

  • NSDI: networking and systems (overlap with OSDI)

  • CoNEXT: second-tier conference in computer networking

  • OSDI: major conference in computer systems (sometimes including networking)

  • SOSP: alternates years with OSDI

  • USENIX ATC: second-tier conference in computer systems (sometimes including networking)

  • HotNets: upcoming work and position papers in networking. Intended to be exciting, half-baked proposals.

  • IMC: top-tier conference in Internet measurement

Other places would be the website of authors whose papers you’ve read in class and liked.