CS 5963/6963: Applied Software Security Testing


Final Project (collected via Canvas)

Instructions: Using your skills from Labs 1–3, team up in groups of no more than four students to hunt down bugs in a real-world application of your choice! Upon selecting a target application, your team will need to figure out how to (1) harness it, (2) fuzz it, and (3) triage any discovered bugs. You may select any target you like (e.g., software APIs, video games, emulators), provided that it has not been fuzzed before—or has demonstrably not yet been fuzzed effectively.

Halfway through the semester, your team will present a 5-minute project proposal to the class outlining your chosen target, your proposed approach, and the significance of your work. At the semester's end, you will prepare and deliver a 15-minute 5-minute final presentation alongside a final report outlining your ultimate approach, findings, and any discovered bugs.

Heilmeier's Catechism will serve as the high-level rubric for your proposal, presentation, and report—so be ready to explain why your project idea matters! But most importantly, get creative and have fun, and report any bugs you find along the way!

Project signup sheet is available: here (accessible only via your @gcloud.utah.edu account).

Lab Exercises (collected via Canvas)

Instructions: There will be three introductory fuzzing exercises that will count for 45% of your course grade (15% each).

Unless otherwise indicated, you must work solo. You may consult general reference material, but you may not collaborate with other students. The material you turn in must be entirely your own work, and you are bound by the Student Code.

Assignment Deadline (by 11:59PM)
Wednesday, February 7
Overview: In preparation for the semester project, this lab will familiarize you with AFL++ (the world's most popular and extensible fuzzing platform). You will test several of AFL's features in fuzzing an application of your choice.

Learning Objectives:

  • Compiling a C/C++ program
  • Inserting AFL++’s instrumentation
  • Initiating fuzzing with AFL++
  • Interpreting AFL++’s results


Your task: Select three of AFL++'s user-configurable features and evaluate their impacts on fuzzing:

  • What led you to explore these fundamental features and why?
  • How do these features impact speed, coverage, and crash discovery?
  • Do certain features work better in tandem, or individually?
  • Do these features perform as you expected, or unexpectedly?
  • As a target application, you are welcome to use the FuzzGoat fuzzing benchmark, or select a program from any other publicly-available fuzzing benchmark corpora (e.g., FoRTE-FuzzBench, HexHive's Magma).


Other Notes:

  • Information on AFL++'s available features can be found in its documentation.
  • Linux is recommended. You are welcome to use the Lubuntu VM from CS 4440.
  • For issues troubleshooting AFL++, you can ask for help on the Course Piazza, or reach its authors via GitHub or the #aflplusplus-issues-questions channel in the Awesome Fuzzing Discord. It is recommended that you start early.


Recommended Readings:


What to Submit:

Submit a 1–3 page report detailing your experimental findings. There are no "right" or "wrong" answers—your work will be assessed by your overall effort. You have full creative liberty—feel free to use images, tables, etc.

Wednesday, February 14
Overview: In this lab, you will leverage AddressSanitizer (ASAN)—today's state-of-the-art memory error detection tool—to perform post-fuzzing crash triage for crashing test cases you found during your Lab 1 fuzzing campaign.

Learning Objectives:

  • Compiling a C/C++ program with ASAN
  • Generating and deduplicating ASAN reports
  • Interpreting and triaging fuzzer-found crashes


Your task: Replay the fuzzer-found crashing test cases you found during Lab 1 on an ASAN-instrumented version of your target program, and collect and analyze their resulting crash report information:

  • What sorts of bugs did your fuzzer find?
  • Are you seeing lots of "unique" crashes map to only a few bugs?
  • Or lots of bugs with a few crashes each?
  • What issues do you see, and what insights do you have?


Other Notes:

  • AFL's fuzzer-found crashes are stored in sub-directory within its output folder. Locate this before proceeding!
  • See LiveOverflow's blog and the Readings below for tips on using ASAN for post-processing fuzzer-found crashes.
  • Instead of re-running test cases by hand one-by-one, you can save lots of time by scripting (e.g., with Bash, Python).
  • Didn't find any crashes in Lab 1? Perform some quick fuzzing on the FuzzGoat benchmark (it has lots of crashes).
  • For issues troubleshooting ASAN, you can ask for help on the Course Piazza, or on the #crash-analysis channel in the Awesome Fuzzing Discord. It is recommended that you start early.


Recommended Readings:


What to Submit:

Submit a 1–3 page report detailing your experimental findings. There are no "right" or "wrong" answers—your work will be assessed by your overall effort. You have full creative liberty—feel free to use images, tables, etc.

Wednesday, February 28
Overview: In this lab, you will learn the art of writing fuzzing harnesses for a real-world software API (libArchive).

Learning Objectives:

  • Writing and testing API fuzzing harnesses
  • Compiling and linking fuzzing harnesses
  • Understanding harnessed vs. application fuzzing


Your task: Write an AFL-friendly harness (i.e., processes file inputs using AFL's @@ directive):

  • What libArchive API functions does your harness include, and why?
  • How did you test whether your harness is correct?
  • Did you harness perform as expected, or unexpectedly?
  • Work together! You may team up in groups of no more than three, or work solo.


Other Notes:


Recommended Readings:


What to Submit:

Submit a 1–3 page report detailing your experimental findings. There are no "right" or "wrong" answers—your work will be assessed by your overall effort. You have full creative liberty—feel free to use images, tables, etc.

Paper Presentations (collected via Canvas)

Instructions: Select one paper from the course Schedule to present to the class in a 15–20 minute presentation, totaling 10% of your course grade. Following each presentation will be a 5–10 minute audience discussion.

Presenters: your responsibility is to teach us the paper:

  • Outline its context, pros and cons, main contributions, summary, and key assumptions.
  • Prepare a short slide deck (you are free get "inspired" from existing presentations).
  • Deliver a 15–20 minute presentation, with a 5–10 minute audience discussion to follow.
  • After your presentation, submit your slides in PDF format to Canvas by 11:59PM.


Audience: you are not required to read the paper. However, you must participate in the class discussion of the paper.

Paper signup sheet is available: here (accessible only via your @gcloud.utah.edu account).