Exploratory Testing is a black box testing technique which lays out the freedom to testers to think out of the box with a minimal dependency on pre-designed answers and step on a process of investigation and discovery to improve the quality of application under test. In layman terms, Exploratory testing is said to be a detailed series of testing by exploring the platform, designing the test cases, executing those immediately and maintaining the results. The thumb rule of exploratory testing is : minimum planning and maximum execution. Exploratory testing is a crucial aspect of software testing as this is not a script based test, it involves the art of being creative and productive. Nowadays, this type of testing is preferable as this doesn’t define a fixed set of methodologies which a tester has to follow, the tester is completely free to fly on his own path to play with an application and identify the potential bugs. This is actually a simultaneous process that carries out the activities of test design and test execution without formally documenting the test cases and test conditions.
Like every other testing technique, exploratory testing also has some pros and cons, let’s have a quick look at those.
Pros:
- As this type of testing doesn’t require any extensive test planning, the testing can even be done when requirement documents are not easily available.
- It is more efficient in short term projects where minimal planning is required and important bugs are found quickly.
- This testing technique uncovers the maximum defects when compared with usual regression testing as the regression testing is only performed according to a fixed set of test cases.
- This type of testing can also be done by the management team and stakeholders since this doesn’t require any kind of scripting.
- In an agile environment where continuous deployments are done, exploratory testing is a perfect testing style that can share the insights in a limited timeframe.
Cons:
- As exploratory testing is a crucial part of testing, it is more dependent on the tester’s prior skillset, knowledge and experience.
- As no formal documentation is made before performing exploratory testing, this might create a negative impact in long-term projects.
- It is difficult to review the test cases later which creates a possibility of missing the critical bugs.
Techniques of Exploratory Testing
When and How to use Exploratory Testing in testing workflow
Exploratory Testing is said to be done in early phases of Software Development Life Cycle(SDLC). In the agile development environment where the sprints are short and software builds are released very frequently, exploratory testing plays a vital role in discovering and reporting bugs and getting them resolved within the same sprint cycle. The experience gained by testers from exploratory testing can be valuable in later stages of testing and to design in-depth test cases. Once the test cases are designed from such investigation and observation, they can further be planned to be automated to get added in the regression suite.
Till now we are aware that exploratory testing is all about exploring. Exploratory testing is not only done to understand the product functionalities but also to understand the customer requirements. For quality benchmarking, it is important to navigate the software according to the end user perspective. Though there is no defined process for exploratory testing, the below depiction will define a structure of exploratory testing commonly adopted by testers.
Real Time Example of Exploratory Testing
Example #1
What we see is what we believe, right? You might still be wondering how exploratory testing looks like. Well, it looks like wandering out in an app and choosing your own path to fly without depending on someone’s direction. That means, there are no pre-designed test cases and testers have some sort of quick description which defines what exactly needs to be tested.
Let’s take an example of a food delivery app and list few testing modules that strikes in our mind:
- Login
- Search and filter
- Outlets nearby
- Restaurant selection
- Adding food items to cart
- Modifying cart
- Promo codes
- Payment Gateway
- Delivery tracking
In case of product exploratory testing, it is a good practise to start the testing from the initial module like login page and then move on to the next relevant module. Following this testing workflow would also cover the testing according to the end user perspective. Always remember, speed and accuracy are the most important factors in such applications.
Since the food delivery app would be having multiple restaurants and dishes, it is important for an exploratory tester to test with different test data. The login page and payment gateway is a more concerned aspect in terms of security and is important to be tested with multiple test scenarios. However, being creative and analytical is something that exploratory testing requires the most.
Example #2
Let’s take another example of exploratory testing using pCloudy Certifaya AI-Bot.
The new trend of intelligent automation has proposed the new way of exploratory testing. The concept of AI Testbot has been introduced to reduce the burden on the QA team.
In exploratory testing, AI Testbot investigates and reports bugs in case of functionality crashes or unexpected errors pop-ups. For each test iteration, it has a power to store execution logs, video recordings and screenshots for future references.
pCloudy’s Certifaya features a testbot that allows a tester to upload an application that needs to be tested and sit back tight until the test report is handed over. The key feature of this testbot is that it not only performs exploratory testing but also the cross device testing on different device variants. Once an application is uploaded and a session is triggered, the smart bot crawls inside the application without any human intervention and digs for each random corner case with a motive to uncover maximum bugs. As soon as the Certifaya session gets completed, an insightful report gets mailed to the team to document the test results.
Certifaya Report
The comprehensive report includes scenarios performed by the testbot along with logs and snapshots. The functionality performance score is also wrapped up in the report in the form of charts and graphs like battery chart, memory chart, CPU chart in addition to each frame rendering time.
In agile methodology where software is released in multiple small iterations and developers have to wait for rapid feedback from testers, the testbot plays a vital role in performing quick exploratory testing for new software versions and smoke testing for the entire build deployed.
Myths about Exploratory Testing
Let us discuss about some misconception related to exploratory testing that mislead a tester in his way to become an expert:
Achieving success in exploratory testing with test automation
Many people have a question in mind, can exploratory testing be automated?
We can achieve success in testing with a combination of exploratory testing and test automation, however, these two approaches cannot replace each other.
After being aware of the exploratory testing approach, do we still think that sense of creativity be automated? Can human curiosity be automated? Or random inventions automated? I don’t think so.
Though these kinds of activities can be partially done by a testbot but it is actually not possible via test automation. Common things that we can automate in flow of exploratory testing is to generate random test data, functional test execution, output logging, developing reports and sharing insights with the development team.
In an agile development model, the process structure is defined in such a way that the testing can be done parallely with development. Once the new build is deployed on a testing environment, testers get a go ahead from dev to start off the testing.
To start exploratory testing, initial minimal plans are developed and testing is continued accordingly. Each edge case, critical case, flaky test case and other anonymous doubtable test cases are recorded separately so that they can be retested.
It is very complex and time consuming for the QA team to retest such test cases everytime a new build is deployed in a testing environment. Hence, once the exploratory testing of a particular module comes to an end, a new plan is developed to automate such test cases which can have direct impact on the performance of the production environment. In this way, we can achieve success in exploratory testing with test automation which further leads to better test coverage by covering all corner cases and uncovering all major defects and serving continuous improvement in quality.
Wrapping Up
Without any doubt, exploratory testing brings out the creativity of a tester and has an extreme effect in improving the product quality. Investing too much time in exploring and discovering edge case scenarios might affect the regular testing. It is important to maintain a healthy balance between the two. The kind of reporting done in exploratory testing matters a lot. The tester may find out only one bug in exploratory testing but the way the reporting is done can change the decision made by the management to enhance the growth of the product.
Originally published at https://www.pcloudy.com.
