Fuzzing

A5 Fuzzing / Fuzz Testing

In this assignment, you will learn the concept of fuzzing, create fuzzed inputs for a program, and evaluate your fuzzer with coverage information.

Fuzzing/Fuzz Testing

This is a way to generate test cases automatically in order to uncover unexpected program behaviour (e.g., crashes, memory leaks, or other types of bugs).  We often try to generate random inputs to test a program automatically. Depending on whether the tester is aware of the program structure, there could be different types of fuzzing:

• Blackbox fuzzing
  • It treats the program as a black box and is unaware of the internal program structure. For example, a random testing tool that generates inputs at random is considered a black box fuzzer.
• Whitebox fuzzing
  • It leverages program analysis to increase code coverage systematically or reach specific critical program locations. For example, one can run the program (starting with some random inputs), gather constraints on inputs at conditional statements, and use a constraint solver to generate new test inputs.
• Greybox fuzzing
  • It tests the program with partial knowledge of its internal workings. For example, a grey box fuzzer could leverage coverage feedback from other instrumentations or libraries to learn how to reach deeper into the program. If a generated input increases coverage, it will be learned by the fuzzer to improve further fuzzing.

Mechanism

A tester tries to break into the system or application with the help of random data values (i.e., fuzz). In this methodology, generally, coding errors & security vulnerabilities are explored by feeding invalid or random inputs to the system or software application. It may be seen as an automated or semi-automated process, where significant defects, mainly security gaps and crashes, potential memory leaks, etc., are revealed to fix them.

Tools

Many fuzzing tools are widely used in industry and academia.

• Chaos Monkey is a tool developed by Netflix, responsible for randomly terminating instances in production to ensure that engineers implement their services to be resilient to instance failures.
• OSS Fuzz is an open-source fuzzing framework from Google. It aims to make common open-source software more secure and stable by combining modern fuzzing techniques with scalable, distributed execution.
• AFL is a security-oriented fuzzer that employs a novel type of compile-time instrumentation and genetic algorithms to automatically discover test cases that trigger new internal states in the targeted binary.

In this assignment, we will use Python and fuzzingbook library to learn and practice the basic ideas of fuzz / fuzzy testing. If interested, you can look into those tools when you have time.

Requirements

Please ensure you have Python installed on your computer (try to install the same version).

• OS: macOS/Windows 10 (untested on other OS)
• Python >= 3.7 (the example code will not directly work on Python 2. x because of the difference in syntax)

Note: Only the above environment is tested by us. Not sure if other versions work well.

After installing Python, you can install fuzzingbook library and matplotlib library in your environment (you will need them later) by using pip:

pip install fuzzingbook
pip install matplotlib

Experiments

Still trying to install fuzzingbook, but then I need to get graphviz. To get graphviz I need svn. Kill me just.

git clone https://github.com/uds-se/fuzzingbook

pip install fuzzingbook

Part A

Part B

Part C

Report

Note: Please check the “Tasks” section of each part, and answer the questions as required. The percentage following each question indicates how many marks this question accounts for the assignment. Wrap your report and code in one zip file(YourName_YourID.zip).

Question 1.1 (0.5%): Paste the screenshot of the output of Part A below.

Question 1.2 (0.5%): What is the type of fuzzer in Part A? Briefly explain why.

Question 2.1 (1%): Paste the screenshot of the output of Part B below.

Question 2.2 (1%): What is the type of fuzzer in Part B? Briefly explain why.

Question 3.1 (2%): Paste the screenshot of the graph you generated to compare the coverage information of Random Fuzzer and Mutation Fuzzer below.(1%) Upload the code as q3.py.(1%)

Question 3.2 (1%): What is the type of mutation fuzzer in this part? Briefly explain the reason.