Threat Modeling: 81% Precision, But Only 36% Recall
Threat modeling is a systematic process for identifying security threats to a system before they are exploited. Shostack (2014) argues it should be a basic professional skill — “like version control” — not a specialist activity [1]. Yet empirical studies show that current practice is at the lowest maturity level, with significant room for improvement [2].
Shostack’s 4-Step Framework
Shostack distills threat modeling into four questions [1]:
| Step | Question | Technique |
|---|---|---|
| 1 | What are we building? | Data Flow Diagrams (DFDs) |
| 2 | What can go wrong? | STRIDE categories |
| 3 | What are we going to do about it? | Mitigate, eliminate, transfer, or accept |
| 4 | Did we do a decent job? | Validate completeness and consistency |
flowchart LR
Q1["1. What are<br>we building?<br>(DFDs)"]
Q2["2. What can<br>go wrong?<br>(STRIDE)"]
Q3["3. What to<br>do about it?<br>(mitigate/accept)"]
Q4["4. Did we do<br>a decent job?<br>(validate)"]
Q1 --> Q2 --> Q3 --> Q4
Q4 -.->|"iterate"| Q1
style Q1 fill:#e8f5e9,stroke:#019546,color:#282828
style Q2 fill:#c8e6c9,stroke:#019546,color:#282828
style Q3 fill:#a5d6a7,stroke:#019546,color:#282828
style Q4 fill:#81c784,stroke:#019546,color:#282828
The process starts with Data Flow Diagrams showing trust boundaries, data stores, processes, and external entities. Threats are then systematically elicited by applying STRIDE categories to each DFD element.
STRIDE Threat Categories
Microsoft’s STRIDE framework provides a mnemonic for six threat categories [3]:
| Category | Threat | Property Violated | Example |
|---|---|---|---|
| Spoofing | Pretending to be someone else | Authentication | Stolen credentials |
| Tampering | Modifying data or code | Integrity | Man-in-the-middle attack |
| Repudiation | Denying an action occurred | Non-repudiation | Unsigned transactions |
| Information Disclosure | Exposing data to unauthorized parties | Confidentiality | Heartbleed (CVE-2014-0160) [4] |
| Denial of Service | Making a system unavailable | Availability | WannaCry ransomware [5] |
| Elevation of Privilege | Gaining unauthorized access | Authorization | Buffer overflow → root |
Shostack candidly acknowledges that STRIDE “makes a lousy taxonomy” — it was designed as a mnemonic to prompt thinking, not a rigorous classification system [1]. Categories overlap, and real attacks often span multiple categories.
STRIDE Empirical Effectiveness
Scandariato et al. (2015) conducted the most rigorous empirical evaluation of STRIDE to date, with 57 participants applying STRIDE to a web application [6]:
| Metric | Value | Implication |
|---|---|---|
| Precision | 81% | Most identified threats are valid |
| Recall | 36% | 64% of actual threats are missed |
| Productivity | 1.8 valid threats/hour | Drops to 0.9 with documentation overhead |
| Main weakness | Spoofing, tampering under-reported | Participants focus on familiar categories |
What This Means
STRIDE is good at confirming threats you identify (81% precision) but poor at finding all threats (36% recall). This means:
- STRIDE should not be the only threat identification technique
- Teams should combine STRIDE with attack trees, LINDDUN (privacy), or other complementary methods
- The 1.8 threats/hour rate means a moderately complex system requires significant time investment
Threat Modeling Maturity
Yskout et al. (2020) proposed a 5-level maturity model and found that most organizations are at Level 1 (ad-hoc) [2]:
| Level | Name | Description |
|---|---|---|
| 1 | Ad-hoc | Informal, inconsistent, individual expertise |
| 2 | Repeatable | Documented process, basic templates |
| 3 | Defined | Standardized across organization, training |
| 4 | Managed | Measured, tracked, continuously improved |
| 5 | Optimized | Automated, self-adaptive, integrated into CI/CD |
Key Findings
| Finding | Value |
|---|---|
| Average project TM effort | 69 hours (SD=32) |
| Threat elicitation share | Only 1/3 of total effort |
| Research focus | Disproportionately on elicitation |
| Adoption criteria | 6: model-based, traceable, systematic, business-integrated, context-aware, scalable |
The gap between research focus (mostly on threat elicitation techniques) and practitioner needs (equal time on modeling, analysis, and review) suggests that advancing maturity requires tools and processes for the entire TM lifecycle, not just better threat identification.
12 Threat Modeling Methods
Shevchenko et al. (2018) surveyed 12 threat modeling methods at the SEI/CMU [7]:
| Method | Focus | Strengths | Limitations |
|---|---|---|---|
| STRIDE | Per-element threats via DFDs | Most mature, systematic | Scalability, 36% recall |
| PASTA | Risk-centric, 7-stage process | Business context | Complex, requires expertise |
| LINDDUN | Privacy threats | Complements STRIDE for GDPR | Privacy-only scope |
| Attack Trees | Decompose attack goals | Visual, intuitive | Manual, no tool support |
| CVSS Scoring | Vulnerability severity | Standardized, quantitative | Scores may vary across experts |
| Trike | Requirements-based, risk model | Formal risk assessment | Limited adoption |
The survey’s key conclusion: no single best method — selection depends on project needs, team expertise, and regulatory context [7]. STRIDE remains the most mature and widely adopted, but organizations should consider combining methods for comprehensive coverage.
OWASP Top 10
The OWASP Top 10 catalogues the most critical web application security risks, updated periodically based on industry data [8]:
| # | Risk | Case Study |
|---|---|---|
| A01 | Broken Access Control | Target: HVAC vendor credentials → 40M cards (missing reference) |
| A02 | Cryptographic Failures | Heartbleed: missing bounds check → memory leak [4] |
| A03 | Injection | Equifax: Apache Struts RCE → 147M records [9] |
| A04 | Insecure Design | SolarWinds: no threat model for build process [10] |
| A05 | Security Misconfiguration | Default credentials, unnecessary services |
| A06 | Vulnerable Components | Log4Shell: JNDI lookup → RCE in 93% of clouds [11] |
| A07 | Authentication Failures | Credential stuffing, session fixation |
| A08 | Software & Data Integrity | SolarWinds: compromised build pipeline [10] |
| A09 | Logging & Monitoring Failures | WannaCry: patch available 2 months before attack [5] |
| A10 | Server-Side Request Forgery | Cloud metadata endpoint abuse |
The Top 10 is not a testing methodology but a prioritization guide — it helps teams focus security effort on the most impactful risk categories.
CVSS: Scoring Vulnerability Severity
The Common Vulnerability Scoring System quantifies vulnerability severity on a 0-10 scale using three metric groups [12] [13]:
| Metric Group | What It Measures | When Applied |
|---|---|---|
| Base | Intrinsic vulnerability characteristics (attack vector, complexity, impact on CIA) | At disclosure |
| Temporal | How the threat evolves over time (exploit maturity, patch availability) | Over time |
| Environmental | Organization-specific context (asset criticality, existing mitigations) | Per deployment |
Severity Scale (CVSS v3.1)
| Rating | Score Range | Example |
|---|---|---|
| None | 0.0 | Informational finding |
| Low | 0.1–3.9 | Minor information disclosure |
| Medium | 4.0–6.9 | XSS requiring user interaction |
| High | 7.0–8.9 | SQL injection in authenticated endpoint |
| Critical | 9.0–10.0 | Log4Shell (10.0), Heartbleed (7.5), EternalBlue (8.1) |
Log4Shell (CVE-2021-44228) scored CVSS 10.0 — the maximum — because it required no authentication, could be triggered remotely, and provided full system compromise [11].
CVSS scores may be inconsistent across experts [12]. The Base score captures technical severity but not business impact — a CVSS 10.0 vulnerability in an internal tool may matter less than a CVSS 6.0 in a payment system.
References
- A. Shostack, Threat Modeling: Designing for Security. Wiley, 2014.
- K. Yskout, T. Heyman, D. Van Landuyt, L. Sion, K. Wuyts, and W. Joosen, “Threat Modeling: From Infancy to Maturity,” in ICSE-NIER 2020, 2020.
- Microsoft, “STRIDE Threat Classification.” 2009. Available at: https://docs.microsoft.com/en-us/previous-versions/commerce-server/ee823878(v=cs.20)
- National Institute of Standards and Technology, “CVE-2014-0160: OpenSSL Heartbleed.” 2014. Available at: https://nvd.nist.gov/vuln/detail/CVE-2014-0160
- National Institute of Standards and Technology, “CVE-2017-0144: Windows SMB Remote Code Execution (EternalBlue).” 2017. Available at: https://nvd.nist.gov/vuln/detail/CVE-2017-0144
- R. Scandariato, K. Wuyts, and W. Joosen, “A Descriptive Study of Microsoft’s Threat Modeling Technique,” Requirements Engineering, vol. 20, no. 2, pp. 163–180, 2015, doi: 10.1007/s00766-013-0195-2.
- N. Shevchenko, T. A. Chick, P. O’Riordan, T. P. Scanlon, and C. Woody, “Threat Modeling: A Summary of Available Methods,” Carnegie Mellon University Software Engineering Institute, 2018.
- OWASP, “OWASP Top 10 - 2021.” 2021. Available at: https://owasp.org/Top10/
- National Institute of Standards and Technology, “CVE-2017-5638: Apache Struts 2 Remote Code Execution.” 2017. Available at: https://nvd.nist.gov/vuln/detail/CVE-2017-5638
- Cybersecurity and Infrastructure Security Agency, “Alert AA20-352A: Advanced Persistent Threat Compromise of Government Agencies,” CISA, 2020. Available at: https://www.cisa.gov/news-events/cybersecurity-advisories/aa20-352a
- National Institute of Standards and Technology, “CVE-2021-44228: Apache Log4j Remote Code Execution (Log4Shell).” 2021. Available at: https://nvd.nist.gov/vuln/detail/CVE-2021-44228
- P. Mell, K. Scarfone, and S. Romanosky, “A Complete Guide to the Common Vulnerability Scoring System Version 2.0,” FIRST, 2007.
- Forum of Incident Response and Security Teams, “Common Vulnerability Scoring System v3.1: Specification Document.” 2019. Available at: https://www.first.org/cvss/specification-document
Disclaimer: AI is used for text summarization, polishing and explaining. Authors have verified all facts and claims. In case of an error, feel free to file an issue.