Test & Evaluation

This section outlines the components that have been identified for the Test & Evaluation dimension as well as the levels within the components. These components and levels form the basis for assessing how threat informed an organization’s T&E program is.

Type of Testing

What type of security tests are conducted to evaluate defensive measures?

  1. Reactive, compliance-focused, e.g. security control assessment

  2. Reactive, IOC-focused, e.g. vulnerability assessment

  3. Proactive, threat-focused, e.g. adversary emulation

  4. Proactive, threat-focused, collaborative, e.g. Purple Team

Frequency of Testing

How frequently are security tests conducted?

  1. Annual or Ad hoc

  2. Semi-Annual

  3. Monthly

  4. Continuous

Test Planning

Are tests coordinated and prioritized on the most relevant threat behaviors?

  1. No formal planning or ad hoc planning

  2. Deliberately planned and scoped, informed by threats

  3. Collaboratively planned with defenders, focused on known gaps and validating coverage

  4. Collaboratively planned with defenders, linked to organizational metrics or KPIs

Test Execution

Does testing cover adversary TTPs in addition to traditional IOCs?

  1. Not threat-focused, e.g. scanners

  2. IOC-focused, e.g. commodity tooling

  3. TTP-focused, single procedures of techniques

  4. TTP-focused, multiple procedures of techniques, custom tooling

Test Results

How effectively do test results cause improvements in defensive measures?

  1. Results generated

  2. Actions taken with internal team, e.g. playbooks updated

  3. Results formally tracked; findings drive detection improvements and architectural changes

  4. Results formally tracked; findings drive organizational programs, hiring, training, and other significant investments