Tabletop exercise or penetration test: which does your renewal actually ask about?

A penetration test asks whether somebody can get into your systems. A tabletop exercise asks what your people would do once somebody has. They answer different questions, they produce different documents, and every framework that covers both treats them as separate obligations. If your renewal application asks whether your incident response plan has been tested, a penetration test report does not answer it, however recent and however good.

What is each one, precisely?

NIST Special Publication 800-115, through the CSRC glossary, defines penetration testing as Security testing in which evaluators mimic real-world attacks in an attempt to identify ways to circumvent the security features of an application, system, or network (NIST Computer Security Resource Center glossary, citing NIST SP 800-115, 2008).

Evaluators. Attacks. Systems. It is hands-on-keyboard work against technology.

A tabletop exercise, in the same body’s guidance, has personnel meet to discuss their roles during an emergency and their responses to a particular emergency situation, and is discussion-based only and does not involve deploying equipment or other resources (NIST Special Publication 800-84, Guide to Test, Training, and Exercise Programs for IT Plans and Capabilities, 2006).

Discussion. Roles. Responses. Nobody touches anything.

One of those is easy for a client to picture and one is not, which is part of why they get confused. Told they are getting a tabletop exercise, participants routinely assume their technical staff will be logged into the security tooling reading logs. It has to be explained that this is one scenario on one screen with everyone in the room working the problem out loud. A penetration test needs no such explanation, because everyone can imagine somebody trying to break in.

How do they compare on what they actually prove?

	Penetration test	Tabletop exercise
The question it answers	Can our systems be broken into, and how	What would our people do once they have been
Who takes part	Testers, and usually one internal contact	The people who would run the response, by role
What is being stressed	Technology	Decisions, authority and communication
Typical output	A findings report with severity ratings and remediation steps	An after-action report with decisions, gaps and owners
Fails when	A vulnerability is found	Nothing fails. Gaps surface
Answers “have you tested your IR plan?”	No	Yes, with a dated record
Answers “do you test your systems?”	Yes	No
Can be run internally	Yes, if qualified and organisationally separate	Yes

The two rows in the middle of that table are the whole argument. A penetration test can be excellent and tell you nothing about whether your CFO knows who authorises a ransom conversation, or whether anyone has decided who speaks to a journalist. An exercise can be excellent and tell you nothing about whether your remote access is exposed.

Do the standards actually treat them separately?

Yes, consistently, and the cleanest illustration is a single regulation naming both.

New York’s financial services rule requires penetration testing from both inside and outside the information systems’ boundaries by a qualified internal or external party at least annually (NYDFS, 23 NYCRR 500.5(a)(1), 2023). Separately, in a different section, it requires covered entities to test their incident response and recovery plans periodically, but at a minimum annually, with all staff and management critical to the response (NYDFS, 23 NYCRR 500.16(d), 2023).

Two separate requirements, two separate sections, two separate cadence clocks. Nobody drafting that rule thought one covered the other.

The payment card standard does the same thing structurally, placing penetration testing under its requirement about testing systems and networks, and incident response plan testing under a different requirement entirely. Different control families, different evidence.

What does Canada’s own data say about the confusion?

Something genuinely revealing, and it is not an opinion.

Statistics Canada runs the national survey on cyber security and cybercrime. Its questionnaire asks businesses which activities they undertake to identify cyber security risks, and the answer options include Penetration testing, undertaken by an employee and Penetration testing, undertaken by an external party (Statistics Canada, Canadian Survey of Cyber Security and Cybercrime questionnaire, survey 5244, 2025).

The same questionnaire asks nothing at all about tabletop exercises, simulations, drills, or testing an incident response plan. It asks whether a business continuity plan exists. It never asks whether anyone has tried it.

So Canada’s national statistical agency treats penetration testing as a measurable, countable activity, and does not measure plan testing at all. That is why there is no Canadian figure for how many firms run exercises: nobody has ever asked the question.

It also explains a certain amount of the confusion in the market. The technical test is the one that gets counted, so it is the one that gets budgeted.

Where do red teaming and purple teaming fit?

Alongside the penetration test, not alongside the exercise.

A red team engagement is an adversary simulation: testers pursue an objective across your environment over weeks, usually without the defending team knowing, to see whether the attack is detected and how the response performs in practice. A purple team engagement runs the same activity with the attacking and defending sides working together in the open, so the defenders learn as it happens.

Both are technical, both involve real systems, and both are considerably more expensive than either a standard penetration test or a tabletop exercise. Both also produce something a tabletop cannot: evidence about whether your detection actually fires.

What none of them produces is a record of a leadership team making decisions, because the executives are usually not involved and, in a red team, are often deliberately not told. A room that has never discussed who authorises a public statement will not have discussed it during a red team either.

The practical hierarchy for a fifty to three hundred person firm: know whether you can be broken into, know whether you would notice, and know what you would do about it. Penetration testing answers the first. Red and purple teaming answer the second. A tabletop exercise answers the third, and it is the cheapest of the three by a wide margin.

Which should you do?

Both, eventually, and the sequencing depends on what you already know.

If you have never had anybody try to break in, do that first. Marsh’s analysis of controls against claims data puts network hardening, endpoint detection and response, and logging and monitoring above incident response planning in effectiveness, with incident response planning in the top five (Marsh McLennan, Incident response planning emerges as key cybersecurity control, 2025). The technical side is not the junior partner here.

If you already have a technical programme and have never rehearsed a decision, the exercise will find more. Most mid-market firms we encounter are in this position: reasonable tooling, a written plan nobody has stress-tested, and no idea who would make the notification call.

And if what you are doing is filling in a renewal application, read the questions rather than reasoning from your security programme. The form asks about them separately because they are separate. A strong answer on one does nothing for the other.

The one overlap worth noticing

A penetration test finding is a good scenario seed.

If your last test showed that a particular system is exposed, running the tabletop against exactly that scenario turns an abstract finding into a rehearsed response, and it makes the exercise unarguably relevant to your real risk rather than generic. It is the cheapest way to make the two pieces of work reinforce each other instead of sitting in separate folders.