Black Box, Grey Box, White Box Testing

Security+ Objectives 1.4

Black Box, Grey Box, White Box testing: What Differences?

There are several ways to conduct penetration tests. If you’re considering penetration testing for your network, you’ll likely choose either black, white, or gray box testing. Each method has merits, so it’s helpful to understand the difference between these tests in order to decide which route is right for your organization.

Black Box Testing

Black box testing is a way to test a system with no access to information about the system being tested. The tester has no knowledge of the system, the source code, or the system architecture. Since this is the approach most attackers will take, black box testing replicates the method a hacker would use to try to get into the system.

Here are some of the advantages of black box pen testing:

♦ Since knowledge of the programming language isn’t necessary, the tester doesn’t have to be an expert
♦ The tester documents inconsistencies between the actual system and the specs
♦ It’s performed from an outsider’s perspective, not the system designer’s
♦ It’s reproducible
♦ It’s efficient on larger systems

These are some disadvantages of black box pen testing:

♦ The tests are difficult to design
♦ The results can be overestimated
♦ It’s unable to test all software properties
♦ Uncovering bugs and vulnerabilities can take longer than with other tests
♦ It may not be thorough
♦ Testers are unable to test specific segments of code, such as complex areas that are more prone to errors
♦ There’s a chance of repeating testing already performed by the programmer

White Box Testing

White box testing is also known as clear box testing, glass box testing, structural testing, and transparent box testing. This method of testing software checks the internal structure of an application. The tester has knowledge and access to the source code and the system architecture.

These are advantages of white box pen testing:

♦ It makes sure all independent paths of a module have been checked
♦ It verifies all logical decisions along with their values
♦ It checks syntax and uncovers typographical errors
♦ It finds design errors due to the difference between the code design and actual implementation
♦ It’s often faster at finding bugs and vulnerabilities than black box testing
♦ The testing coverage is usually more complete
♦ It finds errors in “hidden” code
♦ It approximates partitioning done by execution equivalence
♦ It helps in optimizing code
♦ It helps to remove extra lines of code that can introduce hidden flaws

The disadvantages of white box pen testing include:

♦ The testing is more difficult to design
♦ It requires specialized knowledge and tools.
♦ Lack of access to a running system makes it difficult to find defects based on a misconfigured system or issues that only exist when the system is deployed
♦ It’s more expensive
♦ It’s difficult to find hidden errors in every part of source code
♦ It usually requires modifying the program, changing values to force execution paths, or generating a complete range of inputs to test a function

Gray Box Testing

Gray box testing combines elements of black box and white box testing. It simulates an attack by a hacker who has gained access to the network infrastructure documents. The tester has some knowledge of the system being tested, which is usually limited to design documents and architecture diagrams.

Advantages of gray box pen testing include:

♦ It combines the benefits of black box and white box testing
♦ Greater knowledge of the target system can uncover more significant vulnerabilities with less effort
♦ It can better approximate advantages some attackers may have
♦ It’s non-intrusive and unbiased, since the tester doesn’t have access to source code
♦ Testing is performed from the user’s perspective, not the designer’s
♦ There’s no need to provide internal information about the program’s operations and functions

Some disadvantages of gray box pen testing:

♦ There’s limited ability to go over source code and test coverage
♦ There’s a chance of repeating testing already performed by the programmer
♦ It can’t test every possible input stream due to time constraints
♦ It’s unsuitable for algorithm testing

Which Test You Should Choose?

Deciding which testing methodology to adopt depends on the goals of the test. White is best for uncovering semantic errors at the beginning of the lifecycle. Black is ideal for situations where you either don’t have the source code or you want to view the application from an attacker’s viewpoint. Gray provides the most comprehensive software assurance program.

No matter which type of testing your organization ultimately selects, it’s important to have skilled testers perform the tests and analyze the results.