Best practices of exploratory testing

What determines the effectiveness of exploratory testing?

Factors affecting the ad hoc testing success include:

Knowledge

Especially theory, meaning the knowledge of given technologies or tools, and such. How to write SQL queries so that a manual tester can quickly extract the data they need for testing from a given product the database, for example. Another example: you’re testing an application based on the Salesforce CRM Platform, but you don’t know about object relationships. In this situation, you would not know that having a master-detail object relationship, when deleting the parent record, meaning Cars group, all the child records are also deleted – Make along with the Model assigned to them.

Experience

As time goes by, we gain more and more practice through the trial-and-error method by working on different projects and using different implementation approaches. It would be hard for a novice tester, without application documentation, to quickly put together action and prioritize test areas.

Let me describe it with an example. We have a mobile app for couriers to assign shipment statuses. The application works offline and transmits statuses to a web application for dispatchers. It’s worth remembering that not all apps always work online, and that’s why you should first check whether the data from the mobile app will be transferred to the web app after synchronization is triggered.

Without this, the warehouse inventory will not match the real shipments number and dispatchers will not know that the courier has already taken a shipment and delivered it to the customer.

Besides, let’s remember that our life experience is also useful. A person who used to sell online finds it easier to test platforms for setting up an online store, such as Shoper, PrestaShop, or Magento, because they understand the business needs better. Similarly, a person who previously worked as a courier will have basic business knowledge that is helpful in testing applications related to this industry.

Intuition

Think of the work of a detective who, while conducting an investigation, hits upon a clue that leads to the case solution. Similarly, a manual tester using a selected data or steps combination finds a mistake. I remember a situation when I was testing an application for couriers. When creating a shipment for courier pick-up in one-minute intervals for the smallest and largest sizes, the application worked correctly, but for non-standard ones, the creation form closed without displaying the shipment on the list.

Figure: Factors indicating exploratory testing effectiveness

Getting familiar with the application through tester improvisation

In a perfect world, every product should have up-to-date documentation, but I know from experience that this is rarely the case. 😉 If we work on the project from the beginning of the application development, we are up-to-date and don’t need the documentation that much. The hard part begins when we get to test an application that was developed before we joined the project. Especially in migration projects, when a company sells a given application to another company – then you can’t count on up-to-date documentation. Changes are often made along the way, and time is of the essence; everything is done at the expense of creating documentation because the new owner wants to benefit from the new application as soon as possible.

I remember when I worked on a major project at a time when we transferred the product to a new owner. The documentation was outdated, the instructional videos didn’t show the basics, just more advanced processes. On top of that, there were deadlines pushed to the limit and daily status meetings where the expectation for the fastest possible results was emphasized. I was responsible for creating test cases for one application area.

This was a great opportunity to use exploratory testing. I started by looking at what technology the application is built on and what the application’s purpose is. The next step was to dissect potential keywords based on our experience in defining test cases – create a service technician, create a work order, for example. Then, I ran tests based on the above keywords using different data and combinations. When I had doubts or found a potential bug, I consulted with a business analyst or a business person who previously used that application’s area daily. With time, getting answers to my questions, and knowing what an error and a correct operation are, I learned the area of the application for which I was responsible. As a result, within the deadline, I created test cases that met the customer’s expectations.

Exploratory testing – when to say: enough is enough?

In the case of ad hoc testing, we need to answer one question: How much time can we devote to such testing? If we have an application that is not complicated, an hour is enough to learn the product area and start writing test cases, without wasting time on preparing documentation.

What about when a large application is involved? I remember that in such a situation, the test team leader would announce the day and the time frame in which we were to perform exploratory testing. There was no time for regression testing based on test cases, and the customer expected us to often check if the product was still working properly. For example, I and the other testers had two hours for such tests, between 11 a.m. and 1 p.m., on Monday and Friday. Every tester on the team ran tests during this time, and any errors were recorded in an online excel. After the allotted time, even if there was still something left to test, we had to stop and join a video call to discuss the bugs found during the allotted time slot.

Exploratory testing can be used in both small and large applications. It is one of the methods to quickly onboard the tester into the project. I hope that this article, as well as the previous ones, helps you to better understand functional tests and use them in practice during your testing career.