Test & Evaluation

This section outlines the key components that have been identified for the Test & Evaluation dimension as well as maturity levels within the components. These components and levels form the basis for assessing how threat informed an organization’s T&E program is. This assessment can be conducted using the companion spreadsheet published with this white paper.

Type of Testing

Are cybersecurity tests focused on helping defenders improve against prioritized threats?

  1. None

  2. Security Control / Risk Assessment (reactive, compliance-focused)

  3. Vulnerability Assessment / Penetration Test (reactive, threat-focused)

  4. Adversary Emulation (proactive, threat-focused) 1 2

  5. Purple Teaming (proactive, threat-focused, collaborative)

Frequency of Testing

Do your tests keep pace with changing adversaries and defended technologies?

  1. None

  2. Annual

  3. Semi-Annual

  4. Monthly

  5. Continuous

Test Planning

Are tests coordinated and prioritized on the most relevant threat behaviors?

  1. None

  2. Ad hoc

  3. Deliberately planned and scoped, informed by Threat Actor or prioritized TTPs 3

  4. Collaboratively planned with Defenders, focused on known gaps and validating coverage

  5. Collaboratively planned with Defenders, linked to organizational Metrics or KPIs

Test Execution

Does testing cover adversary TTPs in addition to traditional IOCs?

  1. None

  2. Scanners or other tooling, not threat-focused

  3. Commodity tooling, IOC-focused

  4. Commodity tooling, TTP-focused, minimum 1 implementation of a technique 4

  5. Commodity or Custom tooling, TTP-focused, multiple (including evasive 5 ) implementations of a technique

Test Results

How effectively do test results cause improvements in defensive measures?

  1. None

  2. Results generated

  3. Results generated, leadership interest, actions taken

  4. Results formally tracked; findings drive detection improvements and architectural changes

  5. Results formally tracked; findings drive organizational programs, hiring, training, and other significant investments

References

1

https://caldera.mitre.org/

2

https://github.com/center-for-threat-informed-defense/adversary_emulation_library

3

https://mitre-engenuity.org/cybersecurity/center-for-threat-informed-defense/our-work/attack-flow/

4

https://mitre-engenuity.org/cybersecurity/center-for-threat-informed-defense/our-work/micro-emulation-plans/

5

https://posts.specterops.io/reactive-progress-and-tradecraft-innovation-b616f85b6c0a