Due date: Mon Apr 04 11:59 pm - Hard deadline: Mon Apr 04 11:59 pm
Assignment by Julie Zelenski
[Quick Links: Advice page, Grading]
Completing this assignment will give you valuable practice with
This introductory exercise is intended to acclimate you to the unix environment and bring out your inner software testing ninja. Your task is to poke and prod a given program to determine where it meets its goals and where it falls short. The specification for the program is given in the Assignment 1 Reassemble writeup. The program makes a good-faith effort at the correct functionality, but it has been deliberately seeded with a set of flaws, one in each of these areas:
After identifying the four bugs in your program, you'll prepare a report detailing what you found.
First, ensure you understand the expected behavior. Read the assignment 1 writeup and experiment with the sample executable at /afs/ir/class/cs107/samples/assign0/reassemble_soln. If unsure of what is expected (e.g. should this input be rejected? what is the correct merge of these fragments?), use the sample solution as a guide.
Next, familiarize yourself with use of our sanity check tool. This handy tool automates comparison of your program's output to the sample. The default sanity check runs a few simple cases. You can also provide your own custom cases to run more comprehensive tests. Custom sanity check is your friend -- take some time now to get comfortable using it and you'll enjoy a payoff on this and all subsequent assignments.
Now, let's get on to bug hunting!
Usage refers to how a program is invoked, i.e. the command entered by the user to start the program. The program is expected to check for an appropriate sequence of suitable arguments and start the program running (when usage is valid) or respond with an informative message and exit gracefully (when not). Find the bug in your program's usage handling by invoking it with a variety of valid and invalid arguments to identify a usage whose handling doesn't match the spec.
For fragment read, the program should accept read well-formed inputs and reject malformed inputs with a clear and helpful message. Brainstorm the range of possible inputs, both valid and invalid, and try these inputs to find those that are mishandled during read. As a testing aid, you can invoke the program with the optional -r flag which causes it to read fragments, print them out, and stop, without attempting reassembly. Using reassemble -r allows you to focus your testing on fragment read in isolation.
There are a variety of ways the align/merge logic might be a bit off and you'll likely find this bug to be the most challenging to narrow in on. The symptom of the logic bug will be that the program is properly invoked (no usage issue) on a well-formed input (no read issue), yet reassembly produces wrong output or exhibits a runtime misbehavior (crash, hang, premature exit). Each logic bug can be reproduced during the align/merge of a single pair, so your minimal test input should be winnowed down to two fragments. Consider what fragment pairs might be tricky to align/merge and craft small targeted inputs to test each. Identify the underlying pattern by examining what is common to those cases affected by the bug. A more roundabout strategy to identify a minimal test case is to start with a complex input that exhibits a problem (e.g. one of the large sample files) and repeatedly cut it down and re-test until you have narrowed in on the pair of fragments at the heart.
Lastly, you'll need to bust out Valgrind to find the lurking memory error. Valgrind is your go-to for finding and resolving memory troubles and we want you to practice with this important tool right away. Read the cs107 guide to valgrind for an introduction. Your program contains a lurking memory error that shows up during a proper program invocation (no usage bug) on a well-formed input (no read bug) that reassembles into the correct result (no logic bug). Despite the program seeming to have run perfectly, the Valgrind report shows a memory error. Run some Valgrind trials to observe how/whether the reported issue changes with different inputs. Your Valgrind bug report should specifically answer these questions:
Edit the files readme.txt and custom_tests to document your findings. For each of your four bugs:
Describe the trigger as precisely as possible, i.e. isolated to the critical feature that distinguishes the non-working cases from the working. The test case should be one specific minimal example that reproduces the bug, but your readme should indicate how to create infinitely more such inputs based on the underlying pattern.
Assume the audience for your bug report is the original developer, thus there is no need for long-winded commentary that repeats the program specification or argues for why crashing is obviously undesirable. Stick to the key facts! Consider:
Each bug report can be documented in 1-3 sentences. The shell command wc -w readme.txt will count the total number of words in your readme file. Four solid and concise reports can total under 250 words. Your grader will thank you :-)
Test cases should be prepared in the form used by custom sanity check. Your repo contains custom_tests and three fragment files (one each for read, logic, and memory). Edit these files to configure a test case for each of your bugs. Running custom sanity check should exhibit failures for the usage, read, and logic bugs. Your memory test case will succeed under sanity check (because it doesn't use Valgrind), but that same case run under Valgrind will show the failure.
Start a habit of making regular commits (cs107 guide to Mercurial) to track your progress and be sure to submit for grading when finished.
The assignment is graded out of 25 points. The readme.txt is worth 20 points: 5 points per category for correctly identifying the bug and providing a bug report that is informative, accurate, and concise. The remaining 5 points are for your completed custom_tests and input files that demonstrate your bugs.
If you can't nail down a particular bug, you can earn partial credit for describing your efforts and what you learned thus far. For example, providing one example input to trigger the bug but without identifying the broader underlying pattern is worth about half credit for that category.
The starter project contains a buggy reassemble, readme.txt, and custom_tests and fragment files for use with custom sanity check. The project is distributed as a Mercurial repository. Clone the starter project from your cs107 repository using the command
hg clone /afs/ir/class/cs107/repos/assign0/$USER assign0
The $USER shell variable automatically expands to your sunet id. If there is no repo for your sunet, this means you were not enrolled in Axess when repos were created, please register asap and send email to cs107@cs asking us to manually create a repo for your sunet id.
The directory /afs/ir/class/cs107/samples/assign0 contains a fully-correct sample solution and some fragment files. You can access these via their full path (e.g. /afs/ir/class/cs107/samples/assign0/reassemble_soln), but it is a bit unwieldy. Your repo has slink, which is a symbolic link to this shared samples directory. Use slink to more easily refer to those files (e.g. slink/reassemble_soln) and avoid retyping the full path.
After you have cloned, immediately edit and commit a simple change:
readme.txt to add your nameIf this succeeds, your environment appears properly configured and you're ready for bug hunting. If not, resolve the issue before going any further: review the Mercurial guide, post a question to forum/email, or come to office hours.
Don't miss out on our companion page of advice and hints for this assignment: Go to advice/FAQ page
After you are finished with your work and have committed all your changes, you must follow the submit instructions to hand in your work. Submit is distinct from commit! You commit as you are working to take snapshots of your progress and submit when finished to send your work to the staff for grading. If you only commit, but never submit, we never see your work and will score the empty repo as a zero.
We recommend you do a trial submit well in advance of the deadline to familiarize yourself with the process and allow time to work through any snags. You can replace that submission with a subsequent one if desired.
No late submissions are accepted on this assignment. The deadline is firm without exception. Don't miss this chance to snap up some quick points and start your quarter off right!
How did it go for you? Review the post-task self-check.