Fuzzing
First learned about it for a compiler interview. Also seen in SE465.
What is Fuzzing?
Fuzzing is a software testing technique that involves providing invalid, unexpected, or random data as input to a computer program. The goal is to discover vulnerabilities, bugs, or crashes in the targeted software by observing its behavior under unexpected or malicious input.
Fuzzing is also known as fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program.
The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks.
An effective fuzzer generates semi-valid inputs that are “valid enough” that is they are not directly rejected by the parser, but do create unexpected behaviours deeper in the program and are “invalid enough” to expose corner cases that have not been properly dealt with.
The goal of fuzzing is to detect unknown vulnerabilities or bugs. Fuzzing revels potential bugs through unintended or anomalous behaviours in the application being fuzzed such as crashes, infinite loops, or other behaviours a user or developer may consider “bad”.
It usually does this by mutating the inputs fed into the program in hopes for further code coverage, so each nook and cranny of the program can be exposed to this arbitrary input. The goal is to claim that a given program is robust enough to perform as intended or to find the bugs in the program so the developer can remediate them.
How does fuzzing work?
Uses
Typically, a fuzzer distinguishes between crashing and non-crashing inputs in the absence of specifications and to use a simple and objective measure. Crashes can be easily identified and might indicate potential vulnerabilities (e.g., denial of service or arbitrary code execution). However, the absence of a crash does not indicate the absence of a vulnerability. For instance, a program written in C may or may not crash when an input causes a buffer overflow. Rather the program’s behavior is undefined.
Browser security
Modern browser undergo extensive fuzzing.
From SE465
Fuzzing / Fuzz Testing
A way to generate test cases automatically in order to uncover unexpected program behaviour (e.g., crashes, memory leaks, or other types of bugs). We often try to generate random inputs to test a program automatically. Depending on whether the tester is aware of the program structure, there could be different types of fuzzing.
Types of fuzzing:
- Blackbox fuzzing
- It treats the program as a black box and is unaware of the internal program structure. For example, a random testing tool that generates inputs at random is considered a black box fuzzer.
- Whitebox fuzzing:
- It leverages program analysis to increase code coverage systematically or reach specific critical program locations.
- For example, one can run the program (starting with some random inputs), gather constraints on inputs at conditional statements, and use a constraint solver to generate new test inputs.
- Greybox fuzzing
- It tests the program with partial knowledge of its internal workings.
- For example, a grey box fuzzer could leverage coverage feedback from other instrumentations or libraries to learn how to reach deeper into the program. If a generated input increases coverage, it will be learned by the fuzzer to improve further fuzzing.
Mechanism
A tester tries to break into the system or application with the help of random data values (i.e., fuzz). In this methodology, generally, coding errors & security vulnerabilities are explored by feeding invalid or random inputs to the system or software application. It may be seen as an automated or semi-automated process, where significant defects, mainly security gaps and crashes, potential memory leaks, et., are revealed to fix them.
Tools
Many fuzzing tools are widely used in industry and academia.
- Chaos Monkey
- A tool developed by Netflix, responsible for randomly terminating instances in production to ensure that engineers implement their services to be resilient to instance failures.
- OSS Fuzz
- Is an Open Source fuzzing framework from Google. It aims to make common open-source software more secure and stable by combining modern fuzzing techniques with scalable, distributed execution.
- AFL
- A security-oriented fuzzer that employs a novel type of compile-time instrumentation and genetic algorithms to automatically discover test cases that trigger new internal states in the targeted binary.
In this assignment, we will use Python and fuzzingbook library to learn and practice the basic ideas of fuzz / fuzzy testing. If interested, you can look into those tools when you have time.