Project 2: Raft Revisions and Replicated Shell

This project has two components. First, you will revise the code you wrote for Project 1 based on the feedback you received in code reviews. Second, you will add a persistent log to your Raft implementation and use the replicated log to build a simple replicated state machine that executes shell commands.

Code revisions

Modify and refactor your Raft implementation based on the feedback you received from code reviews. I expect you to fix all of the issues that I raised in my code review (check with me if you don't think it is possible or reasonable to deal with some of my requests). My overall review comments include a few larger tasks as well. You should also incorporate most or all of the suggestions you received from other students. If you get conflicting comments from different reviewers, you can decide how to handle them (but check with me before going against any of my suggestions). You will need to make many small changes, such as improving comments, but most of you will also need to make significant changes to some of your interfaces, which will have more global ramifications.

Your overall goal should be to make your code as clean and simple as possible. Now that you've gotten feedback and had a chance to hear how other groups solved the same problems, you should have lots of ideas for how to improve your design. For example, you should refactor your abstractions to provide the simplest possible interfaces to your classes, with the best possible information hiding. Also, make your code as easy to read as possible; to do this you will need good documentation as well as a clean class structure.

Before refactoring your existing code, I recommend doing at least a rough design for the new features described below: their needs may suggest additional changes to make in the structure of your code.

Create a file changes in the top-level directory of your repository, which briefly describes the most significant ways in which you improved your Project 1 code for Project 2 (what you changed, at a high level, and why you did it). This file can be relatively short (10-20 lines): it's intended to make it easier for reviewers to find the most important changes.

Replicated shell

In addition to revising your code based on the Project 1 reviews, you will also add new features to your Raft implementation; this will give you a opportunity to apply some of the design lessons you learned during Project 1. In this project you will create a simple replicated shell: when a user types shell commands in a client application, they will be forwarded to the Raft state machine, replicated in all of the logs, and then executed as shell commands on each of the Raft servers. The new code for this project must include the following:

A mechanism for storing log entries in one or more files. The contents of each log entry will be a string of any length. In addition to the string, you must also store a term number with each log entry. Your design must be efficient enough to support logs containing millions of log entries and update rates of 100-1000 new entries per second. You may not use an existing database system or storage system for this: you must build your functionality on the basic C/C++ file I/O facilities such as fopen, std::iostreamstd::fstream. If you are not sure whether it's OK to use a particular library in your implementation, check with me.
Servers must support a new request, which will be issued by clients to send shell commands for replication and execution. Each of these requests will contain the string contents for the new log entry (a shell command). The request will return a string result back to the client once the entry has been replicated and executed (see below for details).
Leaders must replicate their log entries to followers as described in the Raft paper. All log entries must be safely stored in a file before a response is generated for an AppendEntries request. Use log information to implement the election restriction described in Section 5.4.1 of the Raft paper. Your mechanism for log replication must be efficient enough to support a fairly high volume of log messages.
Leaders must determine when log entries are committed, and they must propagate commitIndex to followers as described in the Raft paper.
Once a server knows that a log entry is committed, it should treat the contents of the entry as a shell command and execute it by invoking bash with the -c option. Each server should execute the command in its private working directory. Log entries must be executed in order according to their log indices, and only one command should execute at a time. All servers must execute each committed command, not just the leader. When the leader receives a client request, it should add the command to its log, wait until the log entry has been committed and the command has been executed, and then return to the client any output generated by the command.
Each server must store persistent information about which log entries it has executed. When a server restarts after a crash, it must execute any committed entries that were not executed before the crash, but it must not reexecute commands that already executed.
Write a simple client application that executes a loop reading a line from standard input, sending a request to the Raft leader to replicate and execute the command, and printing the results returned by the leader. You can pass a list of server names to the client using the same approach used for servers (e.g., using command-line arguments). You will need to implement a mechanism for clients to identify the cluster leader, since only the leader can accept new log entries. If the leader crashes, the client must find the new leader, once it has been elected, and retry the request with that leader. Note that leaders can crash in the middle of executing a client request as well as between requests.

Additional Notes and Requirements

This project must be implemented in C++.
Before starting on Project 2, merge your project1 branch back into the master branch of your repository. You can do this by closing the pull request for Project 1 (select the "Merge commit" option).
Create a new branch, project2, and do all of your work for this project on that branch.
For the main class that implements your log storage, create an initial version of the file that contains top-level declarations and interface comments, but no method bodies. Commit this version of the file as a separate commit on the project2 branch, and tag that commit commentsBeforeCode2. Make sure that the message for this commit also includes the name of the file. I recommend that you write comments before code for all your files, but I will only require it for this one file.
You can test your replicated shell using simple shell commands such as "echo foo", "echo foo bar > test.out" and "cat test.out". Since each server runs in a different working directory, you should be able to see a separate copy of the file appear in each server's directory.
You do not need to implement the mechanisms described in the Raft paper for cluster membership changes, log compaction, snapshots, or full linearizability. The replicated shell you implement will not be a perfect replicated state machine: some shell commands, such as "pwd" or "hostname", may behave differently on different servers, and it is possible for a shell command to get executed twice under some conditions.

Submitting Your Project

To submit your project, push all of your changes to GitHub on the project2 branch and then create a pull request. The base for the pull request should be your master branch (which now contains all of your work up through Project 1) and the comparison branch should be the head of your project2 branch. Use "Project 2" as the title for your pull request. If your project is not completely functional at the time you submit, describe what is and isn't working in the comments for the pull request.

Late Days