A Closer Look to Fuzz Testing

Fuzz Testing remains an important part of any application or system QA process. Sometimes known as “fuzzing,” it is a software testing technique aimed at scrambling any input to an application — file formats, UI devices such as a keyboard and mouse, network protocol data, etc. — in an attempt to break its functionality. It sees wide use in security testing, among other QA scenarios.

We recently talked about Microsoft’s Project Springfield which is a Cloud-based fuzz testing application. No matter the audience for your company’s application development efforts, fuzz testing needs to be part of your QA arsenal. Let’s take a closer look at this vital part of the software testing process.

Breaking an Application’s Inputs

Vetting how an application handles its input — no matter the source — is the primary goal of fuzz testing. Considering the wide use SQL injection techniques in hacking, any program needs to be able to either restrict the kinds of data sent to it or properly recognize and deal with nefarious inputs. Just verifying that an app doesn’t crash isn’t good enough; the security of the entire computing infrastructure hosting a hacked application is at risk.

While primarily used for black-box testing, where invalid data is sent into an application, fuzz testing techniques are also leveraged in white-box testing to truly test how the source code handles faulty input data. It is also leveraged to see how if an app properly deals with any memory issues, including those dreaded crash-causing leaks.

Automation is primarily used for fuzz testing, although QA engineers and developers need to be closely involved in the process. Automated routines are able to combine fuzz and stress testing to great effect. It helps in ensuring proper code coverage, as well as finding bugs not yet defined by a test case. 

Types of Fuzzers

There are two common types of fuzz testing applications, commonly known as fuzzers. Mutation fuzzers take existing input data patterns and modify or “mutate” them to create a variety of test data to vet the application’s inputs. On the other hand, generation fuzzers create the input data from scratch based upon specifications; these are sometimes known as specification-based fuzzers.

Leveraging both types helps ensure the complete testing of any application’s inputs.

Additional Resources on Fuzz Testing

While many existing QA frameworks include some fuzz testing functionality, there are also a variety of commercial and open source projects focused specifically on fuzzing. Of course, Project Springfield from Microsoft warrants another mention, especially if your organization builds Windows or Azure applications. Redmond is actively looking for partners to help spread the world about Springfield.

Defensics from Codenomicon is a commercial suite of security testing applications based on fuzz testing technology. Known for its support for over 270 network protocols, file formats, and interfaces, Defensics promises to be vital part of any QA shop’s test tool arsenal. It was the QA application that discovered the infamous Heartbleed vulnerability in 2014.

Shops interested in an open source fuzzing solution need to check out the Sulley Fuzzing Framework, downloadable on GitHub. This Python-based tool is fully automated and vets input data from network protocols, file formats, and command-line arguments. A tutorial is available on Scribd.

If your team isn’t already fuzzing as part of your QA process, use these resources to learn about the technique to ensure you deliver more secure applications to your customers.

Stay tuned to the Betica Blog for additional dispatches from the worlds of QA and software development.