Exploratory Testing

Exploratory testing (ET) is simultaneous learning, test design, and test execution [1]. Unlike ad-hoc testing, ET operates within predefined testing parameters — it uses structured heuristics, charters, and sessions to guide the tester’s investigation [2].

ET is not the absence of structure but a different relationship between test design and execution: the tester’s cognitive engagement during testing replaces upfront test case documentation.

ET vs. Ad-Hoc Testing

“Don’t mix ET with Session Based Testing. The plainest definition of exploratory testing is test design and test execution at the same time. ET is an approach, not another testing technique.” — Industry practitioner [3]

Aspect Ad-Hoc Testing Exploratory Testing
Planning None Charter, goals, heuristics
Structure Random, unguided Structured by techniques and sessions
Documentation None Session notes, defect logs
Reproducibility Low Medium (via session reports)
Skill dependency Low High (domain knowledge, test design intuition)

What Does ET Look Like in Practice?

Imagine you are testing a coupon system for an e-commerce site. A scripted tester would receive pre-written test cases: “Apply valid coupon → verify 10% discount,” “Apply expired coupon → verify error message,” and so on.

An exploratory tester receives a charter instead:

Explore the coupon system using unusual input combinations to discover pricing errors.

During a 90-minute session, the tester might:

  1. Apply a valid coupon — it works. But what if the tester applies it twice? The discount doubles. Bug #1.
  2. Try a coupon on a sale item — the coupon stacks with the sale, creating a negative price. Bug #2.
  3. Add items to cart, apply coupon, then remove items — the discount stays but the subtotal drops below zero. Bug #3.
  4. Open two browser tabs, apply different coupons in each — both apply. Bug #4.

None of these scenarios were in any test plan. The tester discovered them by observing the system’s behavior, forming hypotheses (“What if coupons stack?”), and testing those hypotheses immediately. This is the core of ET: learning drives test design in real time.

After the session, the tester writes up the four bugs with reproduction steps and notes which areas were explored — creating accountability without requiring upfront test case documentation.

The Exploration Spectrum

ET and scripted testing are not binary alternatives but endpoints of a continuum [4]:

flowchart LR
    L1["Freestyle<br>Tester gets only<br>the test object"]
    L2["High Exploration<br>Charter with<br>high-level goals"]
    L3["Medium Exploration<br>Charter adds starting<br>points and info"]
    L4["Low Exploration<br>Charter contains<br>detailed activities"]
    L5["Fully Scripted<br>Steps and test<br>data specified"]

    L1 --> L2 --> L3 --> L4 --> L5

    style L1 fill:#c8e6c9,stroke:#388e3c
    style L2 fill:#dcedc8,stroke:#689f38
    style L3 fill:#fff9c4,stroke:#f9a825
    style L4 fill:#ffe0b2,stroke:#ef6c00
    style L5 fill:#ffccbc,stroke:#e64a19

Each level has distinct strengths [4]:

Factor Higher Exploration Lower Exploration
Defect detection Better —
Time efficiency Better —
Tester motivation Higher Lower
Adaptability to change Better —
Traceability — Better
Conformance verification — Better
Novice-friendly — Better

Why Exploratory Testing?

The empirical evidence across five controlled experiments consistently shows [5], [6], [7]:

Finding Evidence
Matches scripted testing in defect detection No significant difference in 4 of 5 experiments
4-6x more efficient when total effort counted ET: 4.58h vs. TCT: 19.47h for equivalent results [6]
Fewer false positives TCT produces 2x more false defect reports [5]
88% industry adoption Mainstream practice, not academic theory [3]

However, ET consistently achieves lower systematic coverage than scripted testing [8], creating a fundamental trade-off between efficiency and coverage assurance.

Key Topics

Techniques and Heuristics

Structured approaches to guide exploration:

  • Whittaker’s Tourist Metaphor (18+ tours across 5 districts)
  • Hendrickson’s heuristics (CRUD, Goldilocks, Follow the Data)
  • Dynamic application of test design techniques

Session-Based Test Management

Managing ET with accountability:

  • Charter + time box + reviewable result + debriefing
  • Metrics and reporting
  • Degrees of exploration decision framework

Empirical Evidence

Comprehensive effectiveness data:

  • Five controlled experiments comparing ET vs. scripted testing
  • Industrial case studies and defect detection rates
  • The role of tester knowledge and experience

ET in Agile Context

ET fits naturally into agile testing cycles, supplementing scripted regression and progression tests at every level [9]. Agile values rapid feedback and adaptation — exactly the strengths of ET.

Agile Practice ET Role
Sprint testing Quick exploration of new features
Regression Supplement automated regression with exploratory sessions
User stories Explore edge cases beyond acceptance criteria
Bug hunting Focused sessions on risk areas

Quick Reference

Parameter Value Source
ET session length 1-2 hours [10]
Industrial defect rate 4.8-8.7 defects/hour [1]
Industry adoption 88% of professionals [3]
Efficiency advantage 4-6x over scripted [6], [7]
Tool support 75% have none [3]

References

  1. J. Itkonen and K. Rautiainen, “Exploratory testing: a multiple case study,” in International Symposium on Empirical Software Engineering (ISESE), 2005. doi: 10.1109/ISESE.2005.1541817.
  2. C. Kaner, J. Bach, and B. Pettichord, Lessons Learned in Software Testing. Wiley, 2002.
  3. D. Pfahl, H. Yin, M. V. Mäntylä, and J. Münch, “How is exploratory testing used? A state-of-the-practice survey,” in ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 2014, pp. 1–10. doi: 10.1145/2652524.2652531.
  4. A. N. Ghazi, K. Petersen, E. Bjarnason, and P. Runeson, “Exploratory Testing: One Size Doesn’t Fit All,” 2017.
  5. J. Itkonen, M. V. Mäntylä, and C. Lassenius, “Defect Detection Efficiency: Test Case Based vs. Exploratory Testing,” in International Symposium on Empirical Software Engineering and Measurement (ESEM), 2007, pp. 61–70. doi: 10.1109/ESEM.2007.56.
  6. J. Itkonen and M. V. Mäntylä, “Are test cases needed? Replicated comparison between exploratory and test-case-based software testing,” Empirical Software Engineering, vol. 19, no. 2, pp. 303–342, 2014, doi: 10.1007/s10664-013-9266-8.
  7. W. Afzal, A. N. Ghazi, J. Itkonen, R. Torkar, A. Andrews, and K. Bhatti, “An experiment on the effectiveness and efficiency of exploratory testing,” Empirical Software Engineering, vol. 20, no. 3, pp. 844–878, 2015, doi: 10.1007/s10664-014-9301-4.
  8. S. M. A. Shah, U. Alvi, Ç. Gencel, and K. Petersen, “Comparing a Hybrid Testing Process with Scripted and Exploratory Testing: An Experimental Study with Practitioners,” in International Conference on Product-Focused Software Process Improvement (PROFES), 2014. doi: 10.1007/978-3-319-06862-6_13.
  9. L. Crispin and J. Gregory, Agile Testing: A Practical Guide for Testers and Agile Teams. Addison-Wesley, 2009.
  10. J. Bach, “Session-Based Test Management,” Software Testing and Quality Engineering, vol. 2, no. 6, pp. 32–37, 2000.

Disclaimer: AI is used for text summarization, polishing and explaining. Authors have verified all facts and claims. In case of an error, feel free to file an issue.

Table of contents

This site uses Just the Docs, a documentation theme for Jekyll.