How to Reevaluate the Testing Efficiency by Understanding What a Testing Framework Can Offer

What is a testing framework?

A testing framework is a set of guidelines used for creating and designing the testing process. Depending on the type of the project, the team needs to choose the most suitable testing techniques they could apply to cover all testing levels.

Being a process under constant refinement, whose goal is to pass through multiple testing levels and types, we can spot some advantages, such as:

• decreasing the manual testing execution time;
• lowering the risks of errors;
• reducing the development costs (for bug fixing, maintenance costs);
• improving the testing speed and efficiency.

Let’s go deeper and understand what each component of the testing framework means and who is responsible for ensuring that the guidelines are followed.

Testing techniques

Using several types of testing techniques will ensure the product is complete, secure, and efficient.

There are two main categories of testing techniques and a hybrid one:

• black-box
• white-box
• grey-box

The techniques’ name is relevant for the depth of the access in the internal structure of the application.

The Black box technique does not imply access to the internal structure of the application, focusing on external expectations when internal behavior is unknown. Also called ‘functional testing’, its main characteristic is the focus on the user experience and specification analysis. The black box technique is usually used by the QA engineers as part of feature testing but can also be used by developers. As part of the acceptance testing, the black box testing can be done by the client’s team or by the end-user itself.

The White box technique consists of creating test scenarios based on reviewing the internal structure and process of the component or the system, also called ‘clear box testing’. The scope of the white box testing is to verify the entire flow by understanding the input and the process performed to receive the expected output. Usually, this technique is used to test the code implementation and to make sure that there are no flows that can negatively impact software security and performance. Having access to the internal application structure is the privilege of the development team and, therefore, the development team is responsible for applying this technique in the development cycle.

The Grey box technique, which stands in the middle of the road between black box testing and white box testing, implies a partial access to the structure of the internal application. Gray box testing gives the ability to test both sides of an application, the presentation layer, as well as the code part. It is primarily useful in integration to the system testing. In this type of test, the QA engineer knows the role of the system and its functionalities, and also knows (though not extensively) its internal mechanisms (especially the internal data structure and the algorithms used).

Testing levels

Tests are frequently grouped by where they are added in the software development process, or by the level of specificity of the test. Generally, there are four recognized levels of testing:

• Unit testing
• Integration testing
• System testing
• Acceptance testing

1. The purpose of unit testing is to validate each unit of the software code and the way in which it performs. The main properties of unit tests ” – small scope, done by the programmer himself, and fast – meaning that they can be run very frequently when programming”, as Martin Fowler mentions, provide instant feedback. They are the first quality gate, being widely used in TDD and BDD approaches.
2. Integration testing aims to test different parts of the system, combined to assess if they work correctly together. By testing the units in groups, any faults in the way units interact can be identified. During Integration testing, a service layer of the application is tested. Developers themselves or independent QA engineers are the ones performing Integration Testing.
3. The purpose of system testing is to evaluate the compliance of a certain system with the specified requirements. System testing is the first testing level that carries out the task of testing a software product as a complete and integrated software.
4. Acceptance testing is the fourth and last level of software testing performed after System testing and before making the system available for actual use. The purpose of this test is to evaluate the compliance of a certain system with the business requirements and assess whether it is acceptable for delivery.

Testing types

A software testing type is defined as a set of activities with a defined test objective and test strategy. There are two main categories: functional testing and non-functional testing.

Functional testing focuses on making sure that the products meet the requirements from the business perspective. The most used types are:

• Smoke testing
• Regression testing
• Cross-platform testing and cross-browsing testing

Non-functional testing targets the way in which the product behaves in different conditions, rather than the specific behavior of that system. As part of the non-functional testing there are:

• Security testing
• Performance testing
• Install Testing
• Recovery Testing

Let’s mix them and find who is responsible

Between the testing techniques, levels, and types there is a strong interconnection. For defining the role, we need to understand what is the intersection between the three of them.

Keep in mind that automating the tests for each of the techniques and testing types will not change the testing technique or/and type itself! The focus of the technique/type should not change!

Conclusion

Changing the way of working will always be questioned, and sometimes it will be hard for all the team members to be fully engaged. What we need to keep in mind is always trying to understand the change and see if the invested effort can be paid back later. Adopting and creating a testing framework will raise concerns at the beginning, but in time, if the team gives a try, it will pay off all the effort and costs.