Lab Solution 6: Networking
The lab checkoff sheet for all students can be found right here.
Get Started
Before starting, go ahead and clone the lab6
folder, which contains the code for the file-server
program discussed in Problem 3.
git clone /usr/class/cs110/repos/lab6/shared lab6
Problem 1: Hello Server Short Answers
Est. 25min.
Consider the following server called hello-server
:
static const size_t kNumWorkers = 3;
static pid_t workerPIDs[kNumWorkers];
static void shutdownAllServers(int unused) {
for (size_t i = 0; i < kNumWorkers; i++) {
cout << "Shutting down server with pid " << workerPIDs[i] << "." << endl;
kill(workerPIDs[i], SIGINT);
waitpid(workerPIDs[i], NULL, 0);
}
cout << "Shutting down orchestrator." << endl;
exit(0);
}
static void handleRequest(int client) {
sockbuf sb(client);
iosockstream ss(&sb);
ss << "Hello from server (pid: " << getpid() << ")" << endl;
}
static void runServer(int server) {
cout << "Firing up hello server inside process with pid " << getpid() << "." << endl;
while (true) {
int client = accept(server, NULL, NULL);
cout << "Request handled by server (pid " << getpid() << ")." << endl;
handleRequest(client);
}
}
int main(int argc, char *argv[]) {
int server = createServerSocket(33334);
for (size_t i = 0; i < kNumWorkers; i++) {
if ((workerPIDs[i] = fork()) == 0) runServer(server);
}
signal(SIGINT, shutdownAllServers);
runServer(server); // let orchestrator process be consumed by same server
return 0;
}
If we launch the above executable on myth64
, hit it 12 times by repeatedly typing telnet myth64 33334
at the prompt from a myth65
shell, and then press ctrl-c
on myth64
, we get the following:
myth64$ ./hello-server
Firing up hello server inside process with pid 19941.
Firing up hello server inside process with pid 19940.
Firing up hello server inside process with pid 19942.
Firing up hello server inside process with pid 19943.
Request handled by server (pid 19940).
Request handled by server (pid 19942).
Request handled by server (pid 19941).
Request handled by server (pid 19943).
Request handled by server (pid 19940).
Request handled by server (pid 19942).
Request handled by server (pid 19941).
Request handled by server (pid 19943).
Request handled by server (pid 19940).
Request handled by server (pid 19942).
Request handled by server (pid 19941).
Request handled by server (pid 19943).
^CShutting down server with pid 19941.
Shutting down server with pid 19942.
Shutting down server with pid 19943.
Shutting down orchestrator.
Based on the code and test run you see above, answer each of the following short answer questions.
- Note the very first line of the server’s
main
function creates a server socket that listens to 33334 for incoming network activity. Why, after thefor
loop withinmain
, are all of the child processes also listening to port 33334 through the same server socket descriptor? - One way to describe port numbers is as "virtual process IDs". What might be meant by that, and why are virtual process IDs needed with networked applications?
- What happens if the call to
exit(0)
is removed from the implementation ofshutdownServers
and we send aSIGINT
to the orchestrator process in the suite of 4hello-server
servers? - If the call to the
signal
function is moved to reside above thefor
loop inmain
instead of below it, does that impact our ability to close down the full suite of 4hello-server
servers? - The above program doesn’t close the server sockets in
shutdownServers
. Describe a simple way to ensure a server socket is properly closed withclose()
before the surrounding server executable exits.
Solution
- The child processes are also listening on the same port because server sockets are descriptors, they are replicated across
fork
boundaries as traditional descriptors are, and they’re all bound to the same session/resource in the file entry table. - The reference of port numbers as "virtual process IDs" refers to the fact that we need an exposed port number to remain constant and map to a bona fide pid internally, much as virtual addresses map to physical ones. We can control the port number, but we can’t control the chosen pid.
- If the
exit
is removed, the three child servers are killed, as they rely on the defaultSIGINT
handler, which interrupts (i.e. terminates) the process. The parent process continues on without children. - If
signal
is moved above the loop, we can still close all four by sendingSIGINT
to the parent. Now all servers respond to the handler, but we don’t check the return values ofkill
orwaitpid
, so if those calls fail or are redundant, the handlers still run without crashing (even if there are morecout
statements now). - To close a server socket here, we can store the value of
server
as a global so that handlers can access it. Addclose(server)
just before theexit
call, and installSIGINT
handlers in child servers that callclose(server)
,exit(0)
, and nothing else.
Problem 2: Networking, Client/Server, Request/Response
(Est. 15min)
- Explain the differences between a pipe and a socket.
- Explain how system calls are a form of client/server and request/response.
- Describe how networking is just another form of function call and return. What “function” is being called? What are the parameters? And what's the return value? Assume HTTP is the operative protocol.
- Describe the network architecture needed for:
- Email servers and clients
- Peer-to-Peer Text Messaging via cell phone numbers
- Skype
- As it turns out, each of the three network applications above all make use of custom protocols. Which ones could have relied on HTTP and/or HTTPS instead of custom protocols?
Solution
- Pipe vs. socket: Fundamentally, a pipe is a unidirectional communication channel and a socket is a bidirectional one. Pipes also are only used to communicate within a given system while sockets are used to communicate over IP, almost always between different hosts.
- System calls as client/server and request/response: the system call is providing a clear service to the user program, which is the client of that service. The request protocol is one we've seen in prior materials (e.g. assign3's
trace
) (populate%rax
with an opcode, additional registers with arguments required of that system call, and so forth), and the response protocol has also been established (success or failure [with side effects] expressed via single return value in%rax
). - Networking as function call and return: the client requires some computation to be performed in another context, and in this case that context is provided on another machine as opposed to some other function on the same machine. The function being called is the URL (where the function lives, and which particular service is relevant, e.g. http://cs110.stanford.edu), the parameters are expressed via text passed from client to server, and the return value is expressed via text passed from server to client.
- Network architectures:
- Email servers and clients: Most email clients and servers speak IMAP over a secure connection. IMAP is similar to HTTP, except that the request and response protocol is optimized for the selection of a mailbox, a digest of all emails in that mailbox, the ability to create and delete mailboxes, the ability to mark an email as read, and the ability, of course, to send an email. (Curious how you can securely telnet to, say, imap.gmail.com? Read this.)
- Peer-to-Peer Text Messaging via cell phone numbers: By default, the cell service provider intercepts all messages via a centralized farm of servers and forwards messages (with images, emoji, etc) on to the intended recipient. In some cases (e.g. two iPhones in conversation over wifi), Apple mediates instead of, say, Verizon. For an accessible introduction to the actual protocol used by early SMS implementations, read this.
- Skype: Same principle as SMS/text messaging, except that persistent connections between clients need to be maintained. This Wikipedia segment does a nice job explaining what Skype does, without going in to the weeds. If you like going into weeds, then this is a really, really well written technical piece explaining how it all works. If you take CS144, this last article is a reading assignment.
- Using HTTP or HTTPS: even though it might have been cumbersome, all of them could have. HTTP/HTTPS is a fairly generic grammar that allows side effects, and everything needed for email, SMS, and video chat could, in principle, be codified via HTTP. However, custom protocols are generally constructed to optimize for common operations (as with email messages that need to be deleted) and/or the need for persistent connections (as with video conferencing).
Problem 3: File Server
(Est. 40min)
In class, we built a server that used our subprocess function to run an external program. Using most of the same code (minus the subprocess, timing, caching, and JSON functionality), we can write a file server that sends files from our file system to a client, and also produces directory listings if the client requests a directory. Many World Wide Web servers have this ability built-in, and it is a quick way to access files on your web server. If you cloned the repository, you have all of the code for the file server.
In order to request a file, the client requests the server name and port number, followed by the file path, which has its root wherever the server is running. e.g. http://myth57:13133/filename
.
The code for getFilename
is shown below:
static string getFilename(iosockstream& ss) {
string method, path, protocol;
ss >> method >> path >> protocol;
string rest;
getline(ss, rest);
cout << "\tPath requested: " << path << endl;
if (path == "/") {
// serve current directory
return (".");
}
size_t pos = path.find("/");
return pos == string::npos ? path : path.substr(pos + 1);
}
- The function populates the
method
,path
, andprotocol
variables, but only uses thepath
variable. What is the purpose of reading in data we won't use? - What is the purpose of the
getline(ss, rest)
statement? - Can you think of any security vulnerabilities in the code above? Hint: what if you ran the server in a directory located at
~/yourHome/SecureServer
. Is there any way for the client to be malicious?
The three functions shown below determine whether the file requested by the client is a file or a directory, and format the fileContents
appropriately. If the name requested is a directory, the opendir
and readdir
functions are used to populate an html listing, complete with hyperlinks. For actual files, the file is sent in plain text without any extra formatting.
/**
* Function: listDir
* ------------------------------
* Populates fileContents with a directory listing in HTML
*/
static void listDir(string& fileContents, const string& directoryPath, bool& isHTML) {
isHTML = true;
/* Create a list of all entries in this directory, in (filename, path) pairs
* e.g. (myfile.txt, samples/mydir/files/myfile.txt).
*/
vector<pair<string, string>> containedFiles;
DIR *dir = opendir(directoryPath.c_str());
if (dir != NULL) {
struct dirent *ent = readdir(dir);
// Loop through every entry in this directory
while (ent != NULL) {
string entryFileName = string(ent->d_name);
string pathToFile;
if (entryFileName == ".") {
pathToFile = ".";
} else if (entryFileName == "..") {
// remove up to the final slash
size_t lastSlashIndex = directoryPath.rfind("/");
if (lastSlashIndex != string::npos) {
pathToFile = directoryPath.substr(0, lastSlashIndex);
} else {
pathToFile = "";
}
} else {
pathToFile = directoryPath + "/" + entryFileName;
}
// Add a new (filename, path) pair
containedFiles.push_back(make_pair(entryFileName, pathToFile));
ent = readdir(dir);
}
closedir(dir);
// Fill fileContents with HTML for these pairs (omitted)
makeHTMLForDirectoryContents(fileContents, directoryPath, containedFiles);
} else {
/* could not open directory */
fileContents = "<h1>Could not open directory.</h1>\r\n";
}
}
/**
* Function: showFile
* ------------------------------
* Populates fileContents with the file contents
*/
static void showFile(string& fileContents, const string& fileName, bool& isHtml) {
ifstream t(fileName);
stringstream buffer;
buffer << t.rdbuf();
fileContents = buffer.str();
isHtml = false;
}
/**
* Function: getFileContents
* ------------------------------
* If fileName is a file, populate fileContents with
* the file's data. If fileName is a directory,
* populate fileContents with a directory listing
*/
static void getFileContents(string& fileContents, const string& fileName, bool& isHTML) {
struct stat s;
if (stat(fileName.c_str(), &s) == 0) {
if (s.st_mode & S_IFDIR) {
//it's a directory
listDir(fileContents, fileName, isHTML);
}
else if (s.st_mode & S_IFREG) {
//it's a file
showFile(fileContents, fileName, isHTML);
} else {
fileContents = "<h1>Requested name was not a file or directory.</h1>\r\n";
isHTML = true;
return;
}
} else {
fileContents = "<h1>File \"" + fileName + "\" was not found.</h1>\r\n";
isHTML = true;
return;
}
}
- Explain how the path to the file is generated for the “..” file. Why does it need to be handled as a special case?
- Assuming that the client is using a web browser, some files that the client might request could actually be html files, yet we are displaying them as raw text files. How might you modify
showFile
to set theisHTML
flag appropriately for html files?
The sendResponse
function is shown below:
static void sendResponse(iosockstream& ss, const string& payload, bool isHTML) {
ss << "HTTP/1.1 200 OK\r\n";
if (isHTML) {
ss << "Content-Type: text/html; charset=UTF-8\r\n";
} else {
ss << "Content-Type: text/plain; charset=UTF-8\r\n";
}
ss << "Content-Length: " << payload.size() << "\r\n";
ss << "\r\n";
ss << payload << flush;
}
- Notice that we use the
isHTML
bool we populated in the file-handling functions to tell the client's browser what type of data we are sending back. The “Content-Type” string, known as a “media type” or “MIME type” is an important identifier for the Internet. We have already used theapplication/javascript
see here type for the scrabble solver server, but there are many other types. You can find an overwhelming list here. - Why must we declare the character set along with the content type?
- What is the purpose of the
flush
on the last line?
You can try out the server by running it and then requesting files using your web browser (if you aren't on the Stanford campus, you should use the Stanford VPN so you can access the myth
machines with your browser). There are two directories (subdir
and anotherdir
) in samples
with files in them to test going up and down directories.
Solution
- Populating
method
,path
andprotocol
: We must read in themethod
andprotocol
information, because we are getting a well-formatted stream of data. We can't simply pick and choose the data we read in, and we must read all of it. So, we read in themethod
andprotocol
and then discard them. getline(ss, rest)
: There is always a blank line after the protocol that we need to read in.- Security vulnerabilities: A malicious client could pass in an absolute file path that starts with two slashes, such as
//usr/class/cs110/
, or (worse),//afs/ir/users/y/h/yourHome
, and this server does not protect against any non-desired access. Once your server is accessible to the network, anyone on the network can access it. - Handling
..
: Because we want to go back to the previous directory, we need to remove the last part of the path. We do a reverse-find for “/” and this tells us where the last slash is. We then use thestring::substr(0,pos)
function to get the string up to the last slash. - Setting
isHTML
totrue
: We could check if the extension for the file is.html
, and then set theisHtml
flag totrue
. - Character set: Because the world doesn't run on ASCII any more, and we want the ability to send characters from any language (including emojis 🏄).
flush
: Streams are often buffered, which means that they don't always send the data immediately. If we didn't flush, then there would be a deadlock condition where the data wouldn't get sent and the client would wait until timeout.
Checkoff Questions Solutions
- Problem 1: Note the very first line of the server’s
main
function creates a server socket that listens to 33334 for incoming network activity. Why, after thefor
loop withinmain
, are all of the child processes also listening to port 33334 through the same server socket descriptor? The child processes are also listening on the same port because server sockets are descriptors, they are replicated acrossfork
boundaries as traditional descriptors are, and they’re all bound to the same session/resource in the file entry table. - Problem 2: Describe how networking is just another form of function call and return. What “function” is being called? What are the parameters? And what's the return value? Assume HTTP is the operative protocol. the client requires some computation to be performed in another context, and in this case that context is provided on another machine as opposed to some other function on the same machine. The function being called is the URL (where the function lives, and which particular service is relevant, e.g. http://cs110.stanford.edu), the parameters are expressed via text passed from client to server, and the return value is expressed via text passed from server to client.
- Problem 3: Explain how the path to the file is generated for the “..” file. Why does it need to be handled as a special case? Because we want to go back to the previous directory, we need to remove the last part of the path. We do a reverse-find for “/” and this tells us where the last slash is. We then use the
string::substr(0,pos)
function to get the string up to the last slash.
Icons by Piotr Kwiatkowski