Is there a standard exercise evaluation guide we can download?

Not one that fits. Federal doctrine is explicit that guides are built for the organization's own plans, policies and procedures, because each organization has its own targets and critical tasks. A borrowed guide evaluates somebody else's.

Do we need an evaluator as well as a facilitator?

For a small session one person can do both badly. Running the room takes full attention and so does recording it, so the two jobs compete. If you only have one person, record the session and write it up afterwards rather than trying to do both live.

Does a carrier want to see an evaluation guide?

No. What a carrier can use is the report the evaluation produced: what was run, who was there, what was decided, what was found, and what is being done about it. The guide is the instrument, and the carrier forms we have read never ask for the instrument.

Exercise evaluation guide: how HSEEP scores a tabletop, and how a carrier reads it

An exercise evaluation guide is the form an evaluator fills in while the exercise runs. Federal doctrine defines it as A document that captures information specifically related to the evaluation requirements developed by the exercise planning team (FEMA, Homeland Security Exercise and Evaluation Program doctrine, 2020). It is a data collection instrument, not a score out of ten. FEMA’s HSEEP course does have evaluators give each target on the guide a rating, from Performed without Challenge (P) to Unable to be Performed (U) (FEMA, HSEEP course L0146, Module 5, Exercise Evaluation, 2026), but those rate performance against your own targets rather than against a universal standard.

What is in one?

Whatever the exercise is trying to find out. The doctrine says the guide provides evaluators with a standardized tool to guide data collection and capture performance results, and immediately adds that because each organization has its own targets and critical tasks, guides are developed specific to their plans, policies, procedures, and protocols (FEMA, Homeland Security Exercise and Evaluation Program doctrine, 2020).

That second half is the part to take seriously. A guide written for somebody else evaluates somebody else’s plan. The objectives at the top of yours should be the two or three things you actually wanted to learn, written before the scenario and not adjusted afterwards.

So how is a tabletop scored?

Against your own objectives, by comparison. In the doctrine’s description of a functional exercise, evaluators observe behaviors and compare them against established plans, policies, procedures, and standard practices (FEMA, Homeland Security Exercise and Evaluation Program doctrine, 2020).

Comparison is the whole method. The question is never whether the room performed well in the abstract. It is whether what they did matched what the plan said they would do, and where it did not, why.

Evaluation then assesses the ability to meet the objectives and capabilities by documenting strengths, areas for improvement, capability performance, and corrective actions in the report and its improvement plan (FEMA, Homeland Security Exercise and Evaluation Program doctrine, 2020).

Four outputs, and notice that the first is strengths. An evaluation that records only problems has not been done properly.

What does the evaluator actually do?

During	After
Watches, and writes. Does not participate, does not prompt, does not rescue the room	Turns observations into findings written so somebody absent can follow them
Records the decision, who made it, on what information, and when	Pairs each finding with a corrective action, an owner and a date
Notes what the plan said would happen, next to what happened	Drafts the report while the detail is still recoverable

There is a second input. Doctrine also provides for a participant feedback form, A document that provides a mechanism to collect input from all participants following an exercise and supports the development of an After-Action Report (AAR) and Improvement Plan (IP) (FEMA, Homeland Security Exercise and Evaluation Program doctrine, 2020).

Worth doing. It collects input from everyone, including anyone who did not speak up in the room.

Does a carrier read any of this?

Not the guide. The report.

An underwriter is unlikely to be evaluating your exercise program against federal doctrine. The carrier forms we have read ask whether the plan was tested, and the report also shows who was in the room and that what came out of it went somewhere. What the doctrine contributes is the discipline: FEMA’s summary of it says SMART ensures that corrective actions are developed to be Specific, Measurable, Achievable, Relevant, and Time-bound (FEMA, Homeland Security Exercise and Evaluation Program (HSEEP) 2020 Revision information sheet, 2020).

Use the guide because it makes the report better. Do not send it to anybody.