Exhaustive Testing is impossible due to its complex and time-consuming nature.
Almost every QA engineer, intern, or senior will certainly accept this fundamental testing principle. Even if you have plenty of time, the principle will still prove true. Because quality is not always about having enough time, but more about having sufficient resources, expertise, a space for discussions, and analyzing different points of view.
We guess the one superpower all QA engineers would like to have is a fresh pair of eyes for every single day. With time, we are often accustomed to some application patterns that are not user-friendly and even incorrect. How to cope with such fallacies?
Some companies find Crowdtesting as a solution. But is there an easier method to solve this problem with your in-house team?
Ask this question to any QA Engineer in Flux Technologies, and everyone would answer: “GTS” (a.k.a Group Testing Session).
What is Group Testing Session?
Group Testing Session is a hybrid of focused groups (a research technique aimed to collect data by identifying and exploring how people think and behave), a peer-testing (a way of evaluating work performed by a coworker), and an Alpha testing (first end-to-end testing of a product to ensure it meets the business requirements and functions correctly. Usually, internal employees perform this test).
The method is used to keep the QA eyes fresh and incorporate a multi-level testing process to ensure the maximum quality of any application.
It is a small group of QA engineers (from 3 to 10) with different expertise and product knowledge levels. They all sit together and interact with the application in order to check its functionality and what can be improved in terms of usability. During the past couple of years, we have done several group sessions (many more to come) and would like to share our experience in this.
The GTS can be conducted with various goals in mind. Either it is requested by the product’s dedicated QA Engineer(s) or Project Manager.
Here are some cases when GTS can be requested:
- Tested features need double validation
- New flow and/or design is implemented and requires usability testing
- To ensure the system is ready for release
- Gathering improvement ideas on an existing user flow, and more.
Phase 1: Preparation
After the request is approved, the preparation phase starts. During this phase, a load of work should be done. As a rule, QA engineers of the application (after confirming with the product/project manager) perform preparation activities, which include:
- Prioritization of the functionalities or flows that need to be tested: What part of the application should be under test?
- Preparation of test data: Are there any accounts or specific data that need to be set up beforehand?
- Preparation of checklists/scenarios: Are there specific use cases that need to be validated in GTS?
- Prioritizing and preparing devices based on the hardware requirements: Should web applications be tested on desktop or mobile devices?
- Preparation of logging and note-taking methods and templates: How to store the results whether they should be grouped or submitted separately?
- Defining resources needed for the GTS: How many QA engineers should participate in GTS?
- Defining the time: How many hours should be spent on testing so it doesn’t violate exhaustive testing and pesticide paradox principles?
- Scheduling the session: When and where the GTS should take place? Are all participants invited and informed correspondingly?
Phase 2: The Process
During the GTS, the facilitator describes the flow and gives minimal instructions to the participants to reduce extra communication during the process. Participants may ask questions only if they are run blockers, and the answer can unblock them. In this stage, it is essential to fix every unexpected behavior of the testable item.
Moreover, during the process, note-taking and logging are crucial. Besides the notes, participants can also include screenshots and video recordings in their reports. Because every little detail matters—and descriptions should be as detailed as possible.
Additionally, participants can use different testing types and styles during the session, such as exploratory, ad-hoc, error-guessing, and intuitive testing.
Phase 3: Wrap up
After the main testing, we summarize our notes (including improvements, bugs, etc.). This process will not be painful and time-consuming if you already take descriptive notes during the main testing phase.
However, for more efficiency, we perform these actions right after the GTS when information is still in our short-term memory. Otherwise, chances are we can lose or forget it. Subsequently, we pass all the gathered information to the GTS facilitators according to the pre-defined format.
Phase 4: Post-Testing Activities
After collecting reports from all participants, the final activities come to the stage. Firstly, you will have to analyze the results carefully. Afterward, participants should remove the duplicates or treat them as indicators of frequently reported cases. Now group the similar issues and improvements. Lastly, exclude the risks and issues.
After the final report is generated with all issues and suggested improvements, the facilitator passes the GTS report to PM or PO for prioritizing the items and including them in the development phase.
Some Pros and Cons
The GTS is a self-made process so that it will be refined over time. It has both advantages and disadvantages. Here are some of the crucial pros and cons we found in GTS:
Some Essential Tips + Do’s and Don’ts
- Generally, intuitive testing takes place during the GTS. But still, make sure the participants have sufficient product knowledge—especially when dealing with tools that need specific training and guidance.
- Never underestimate the importance of preparation. GTS is useful when you prepare it effectively. However, you don’t need to over-plan the process; always leave room for creativity and intuition.
- Always include screenshots and recordings in your reports, as it can be a blessing for a product team.
- Take notes during the testing and include them in your reports.
- Not all features need GTS. So, carefully choose the application or feature for GTS to avoid wasting time finding already known issues. Focus on the parts that need usability validation.
- You can also find some functional bugs during GTS, but do not expect full system testing and critical bugs from GTS. Moreover, make sure the application has few functional issues when conducting GTS.
