Exploratory testing – Why is it important?

First, let’s clarify – what is exploratory testing? The simplest definition is that Exploratory Testing is an approach (not a technique), where you apply your product learnings, creativity and wisdom to come up with new test scenarios.
But why do we need to do this if we’re already doing a good level of scripted testing? We’re writing test scenarios in each story, running them on different builds until they all pass, and we’re also running them in regression to make doubly sure that the product is still working.
Sound good and thorough – what’s the point in doing more software testing on top of that?
Well, there are a few good reasons to do exploratory testing in addition to the regular, scripted testing. Certainly there are a number of benefits to doing scripted testing; this is a very organized and maintainable approach, but it has its limitations as well. Let’s have a look at a few of them:
Software programs can fail in many ways
Scripting all the possible failure conditions is impossible. Software can fail in many different ways, e.g. different configuration settings, unpredictable inputs, environmental state, network conditions, etc. A case in point — recently it was determined that a configuration setting (leading to software failure) was the cause of a debacle within the air travel industry. The problem was tracked down to software settings that, if adjusted, stopped the deletion of data and eventually caused the system to crash as a result of a memory leak. (See my blog on what the FAA’s “Flypocalypse” debacle teaches us about test automation best practices.)
Scripts get stale quickly
Once your scripts have been run and tested, running the same script again and again is effectively trying to find the same issue again and again! Now the scripts are only good to find the impact of the other changes. That’s another reason why it is a good idea to have test automation so that the machines can look after finding the impact issues, and human minds can be used to find new issues.
Product risk profiles evolve over time
The product risk profile keeps evolving throughout the lifetime of the product, whereas test scripts are mostly written while implementing (or sometimes designing) a particular story. The continuous evolving risk throughout the product mandates continuous impact analysis and maintenance which takes a significant amount of time and effort.
Everyone is different
Different people make different mistakes, and likewise different people can think of different conditions to find the problems with a system. In this way, the same set of tests can be proven limited. Exploratory testing fills that gap and allow us to leverage expertise and human intelligence over time.
Continual change of surroundings
Some factors are out of our control. The environment around us keeps changing. For example – new security vulnerabilities are being uncovered every day. Exploratory testing around the Bigger Picture can help guard us against this and keep our approach fresh and relevant.
So, the challenge is to find NEW errors, not to look for the same thing over and over and over again!
Remember — with a script, you’ll miss the same things every time.