The lab checkoff sheet for all students can be found right here.

Get Started

Before starting, go ahead and clone the lab6 folder, which contains the code for the file-server program discussed in Problem 1. This is the main problem you'll be focusing on in section - the second problem is extra, only if there's time.

git clone /usr/class/cs110/repos/lab6/shared lab6

Problem 1: File Server

(Est. 40min)

In class, we built a server that used our subprocess function to run an external program. Using most of the same code (minus the subprocess, timing, caching, and JSON functionality), we can write a file server that sends files from our file system to a client, and also produces directory listings if the client requests a directory. Many World Wide Web servers have this ability built-in, and it is a quick way to access files on your web server. If you cloned the repository, you have all of the code for the file server.

In order to request a file, the client requests the server name and port number, followed by the file path, which has its root wherever the server is running. e.g. http://myth57:13133/filename.

The code for getFilename is shown below:

static string getFilename(iosockstream& ss) {
    string method, path, protocol;
    ss >> method >> path >> protocol;
    string rest;
    getline(ss, rest);
    cout << "\tPath requested: " << path << endl;
    if (path == "/") {
        // serve current directory
        return (".");
    }
    size_t pos = path.find("/");
    return pos == string::npos ? path : path.substr(pos + 1);
}

• The function populates the method, path, and protocol variables, but only uses the path variable. What is the purpose of reading in data we won't use?
• What is the purpose of the getline(ss, rest) statement?
• Can you think of any security vulnerabilities in the code above? Hint: what if you ran the server in a directory located at ~/yourHome/SecureServer. Is there any way for the client to be malicious?

The three functions shown below determine whether the file requested by the client is a file or a directory, and format the fileContents appropriately. If the name requested is a directory, the opendir and readdir functions are used to populate an html listing, complete with hyperlinks. For actual files, the file is sent in plain text without any extra formatting.

/** 
 * Function: listDir
 * ------------------------------
 *  Populates fileContents with a directory listing in HTML
 */
static void listDir(string& fileContents, const string& directoryPath, bool& isHTML) {
    isHTML = true;

    /* Create a list of all entries in this directory, in (filename, path) pairs
     * e.g. (myfile.txt, samples/mydir/files/myfile.txt).
     */
    vector<pair<string, string>> containedFiles;

    DIR *dir = opendir(directoryPath.c_str());
    if (dir != NULL) {
        struct dirent *ent = readdir(dir);

        // Loop through every entry in this directory
        while (ent != NULL) {
            string entryFileName = string(ent->d_name);
            string pathToFile;

            if (entryFileName == ".") {
                pathToFile = ".";
            } else if (entryFileName == "..") {
                // remove up to the final slash 
                size_t lastSlashIndex = directoryPath.rfind("/");
                if (lastSlashIndex != string::npos) {
                    pathToFile = directoryPath.substr(0, lastSlashIndex);
                } else {
                    pathToFile = "";
                }
            } else {
                pathToFile = directoryPath + "/" + entryFileName;
            }

            // Add a new (filename, path) pair
            containedFiles.push_back(make_pair(entryFileName, pathToFile));

            ent = readdir(dir);
        }

        closedir(dir);

        // Fill fileContents with HTML for these pairs (omitted)
        makeHTMLForDirectoryContents(fileContents, directoryPath, containedFiles);
    } else {
        /* could not open directory */
        fileContents = "<h1>Could not open directory.</h1>\r\n";
    }

}

/** 
 * Function: showFile
 * ------------------------------
 *  Populates fileContents with the file contents 
 */
static void showFile(string& fileContents, const string& fileName, bool& isHtml) {
    ifstream t(fileName);
    stringstream buffer;
    buffer << t.rdbuf();
    fileContents = buffer.str();
    isHtml = false;
}

/**
 * Function: getFileContents
 * ------------------------------
 *  If fileName is a file, populate fileContents with
 *  the file's data. If fileName is a directory,
 *  populate fileContents with a directory listing
 */
static void getFileContents(string& fileContents, const string& fileName, bool& isHTML) {
    struct stat s;
    if (stat(fileName.c_str(), &s) == 0) {
        if (s.st_mode & S_IFDIR) {
            //it's a directory
            listDir(fileContents, fileName, isHTML);
        }
        else if (s.st_mode & S_IFREG) {
            //it's a file
            showFile(fileContents, fileName, isHTML);
        } else {
            fileContents = "<h1>Requested name was not a file or directory.</h1>\r\n";
            isHTML = true;
            return;
        }
    } else {
        fileContents = "<h1>File \"" + fileName + "\" was not found.</h1>\r\n";
        isHTML = true;
        return;
    }
}

• Explain how the path to the file is generated for the “..” file. Why does it need to be handled as a special case?
• Assuming that the client is using a web browser, some files that the client might request could actually be html files, yet we are displaying them as raw text files. How might you modify showFile to set the isHTML flag appropriately for html files?

The sendResponse function is shown below:

static void sendResponse(iosockstream& ss, const string& payload, bool isHTML) {
    ss << "HTTP/1.1 200 OK\r\n";
    if (isHTML) {
        ss << "Content-Type: text/html; charset=UTF-8\r\n";
    } else {
        ss << "Content-Type: text/plain; charset=UTF-8\r\n";
    }
    ss << "Content-Length: " << payload.size() << "\r\n";
    ss << "\r\n";
    ss << payload << flush;
}

• Notice that we use the isHTML bool we populated in the file-handling functions to tell the client's browser what type of data we are sending back. The “Content-Type” string, known as a “media type” or “MIME type” is an important identifier for the Internet. We have already used the application/javascript see here type for the scrabble solver server, but there are many other types. You can find an overwhelming list here.
• Why must we declare the character set along with the content type?
• What is the purpose of the flush on the last line?

You can try out the server by running it and then requesting files using your web browser (if you aren't on the Stanford campus, you should use the Stanford VPN so you can access the myth machines with your browser). There are two directories (subdir and anotherdir) in samples with files in them to test going up and down directories.

Solution

• Populating method, path and protocol: We must read in the method and protocol information, because we are getting a well-formatted stream of data. We can't simply pick and choose the data we read in, and we must read all of it. So, we read in the method and protocol and then discard them.
• getline(ss, rest): There is always a blank line after the protocol that we need to read in.
• Security vulnerabilities: A malicious client could pass in an absolute file path that starts with two slashes, such as //usr/class/cs110/, or (worse), //afs/ir/users/y/h/yourHome, and this server does not protect against any non-desired access. Once your server is accessible to the network, anyone on the network can access it.
• Handling ..: Because we want to go back to the previous directory, we need to remove the last part of the path. We do a reverse-find for “/” and this tells us where the last slash is. We then use the string::substr(0,pos) function to get the string up to the last slash. Why is this necessary? If you remove the special case and lump .. in with regular filenames, the HTML link rendered will be e.g. samples/... When the user clicks on it that directs them to samples/ (which is correct). The problem is that then, the HTML link for .. will be e.g. samples//.., which doesn’t work in a webpage; if you click on it it takes you to samples/, so you’re stuck. In other words, if we’re at a non-root-URL ending in /, the .. link won’t work. For that reason, when processing .. we need to do the / removal. Alternatively, we could not have a special case, but just not add a / when building the URLs if there already is one. E.g. pathToFile = directoryPath + (directoryPath[directoryPath.length() - 1] == '/' ? "" : "/") + entryFileName;
• Setting isHTML to true: We could check if the extension for the file is .html, and then set the isHtml flag to true.
• Character set: Because the world doesn't run on ASCII any more, and we want the ability to send characters from any language (including emojis 🏄).
• flush: Streams are often buffered, which means that they don't always send the data immediately. If we didn't flush, then there would be a deadlock condition where the data wouldn't get sent and the client would wait until timeout.

Problem 2: Networking, Client/Server, Request/Response

(Est. 15min)

• Explain the differences between a pipe and a socket.
• Explain how system calls are a form of client/server and request/response.
• Describe how networking is just another form of function call and return. What “function” is being called? What are the parameters? And what's the return value? Assume HTTP is the operative protocol.
• Describe the network architecture needed for:
  • Email servers and clients
  • Peer-to-Peer Text Messaging via cell phone numbers
  • Skype
• As it turns out, each of the three network applications above all make use of custom protocols. Which ones could have relied on HTTP and/or HTTPS instead of custom protocols?

Solution

• Pipe vs. socket: Fundamentally, a pipe is a unidirectional communication channel and a socket is a bidirectional one. Pipes also are only used to communicate within a given system while sockets are used to communicate over IP, almost always between different hosts.
• System calls as client/server and request/response: the system call is providing a clear service to the user program, which is the client of that service. The request protocol is one we've seen in prior materials (e.g. assign3's trace) (populate %rax with an opcode, additional registers with arguments required of that system call, and so forth), and the response protocol has also been established (success or failure [with side effects] expressed via single return value in %rax).
• Networking as function call and return: the client requires some computation to be performed in another context, and in this case that context is provided on another machine as opposed to some other function on the same machine. The function being called is the URL (where the function lives, and which particular service is relevant, e.g. http://cs110.stanford.edu/cgi-bin/gradebook), the parameters are expressed via text passed from client to server, and the return value is expressed via text passed from server to client.
• Network architectures:
  • Email servers and clients: Most email clients and servers speak IMAP over a secure connection. IMAP is similar to HTTP, except that the request and response protocol is optimized for the selection of a mailbox, a digest of all emails in that mailbox, the ability to create and delete mailboxes, the ability to mark an email as read, and the ability, of course, to send an email. (Curious how you can securely telnet to, say, imap.gmail.com? Read this.)
  • Peer-to-Peer Text Messaging via cell phone numbers: By default, the cell service provider intercepts all messages via a centralized farm of servers and forwards messages (with images, emoji, etc) on to the intended recipient. In some cases (e.g. two iPhones in conversation over wifi), Apple mediates instead of, say, Verizon. For an accessible introduction to the actual protocol used by early SMS implementations, read this.
  • Skype: Same principle as SMS/text messaging, except that persistent connections between clients need to be maintained. This Wikipedia segment does a nice job explaining what Skype does, without going in to the weeds. If you like going into weeds, then this is a really, really well written technical piece explaining how it all works. If you take CS144, this last article is a reading assignment.
  • Using HTTP or HTTPS: even though it might have been cumbersome, all of them could have. HTTP/HTTPS is a fairly generic grammar that allows side effects, and everything needed for email, SMS, and video chat could, in principle, be codified via HTTP. However, custom protocols are generally constructed to optimize for common operations (as with email messages that need to be deleted) and/or the need for persistent connections (as with video conferencing).

Checkoff Questions Solutions

• Problem 1: Explain how the path to the file is generated for the “..” file. Why does it need to be handled as a special case? Because we want to go back to the previous directory, we need to remove the last part of the path. We do a reverse-find for “/” and this tells us where the last slash is. We then use the string::substr(0,pos) function to get the string up to the last slash.
• Problem 1: Suppose a user might want to keep a directory inside of a directory private. Explain how you might change the file server to allow users to mark certain directories as off limits. One way would be to have the user place a file (something like .private) in those directories, and then have the server look for that file in any directory it finds. If it exists in the directory, the server can return an error (such as 403 Forbidden) when asked to display the directory.
• Problem 2a: Explain the differences between a pipe and a socket. Fundamentally, a pipe is a unidirectional communication channel and a socket is a bidirectional one. Pipes also are only used to communicate within a given system while sockets are used to communicate over IP, almost always between different hosts.
• Problem 2b: Describe how networking is just another form of function call and return. What “function” is being called? What are the parameters? And what's the return value? Assume HTTP is the operative protocol. the client requires some computation to be performed in another context, and in this case that context is provided on another machine as opposed to some other function on the same machine. The function being called is the URL (where the function lives, and which particular service is relevant, e.g. http://cs110.stanford.edu/cgi-bin/gradebook), the parameters are expressed via text passed from client to server, and the return value is expressed via text passed from server to client.