Ask three vendors for "security testing" and you will get three proposals that differ by an order of magnitude in price and share almost no methodology. That is not because anyone is being dishonest. It is because "security testing" covers at least three distinct exercises that answer genuinely different questions — and the industry has been sloppy about saying which is which.
The choice matters more than the vendor. A red team bought at the wrong moment produces an expensive report confirming what a much cheaper exercise would have told you in week one. Here is the honest version of what each one is.
The one-line version
- Vulnerability scan asks: "Do we have any known, catalogued weaknesses?"
- Penetration test asks: "Can a skilled attacker break this specific system, including in ways no catalogue lists?"
- Red team asks: "Would we notice, and could we respond, if a determined adversary came after us for real?"
Note that only the third one is a test of your people and processes. The first two test your technology. That distinction drives everything below.
Vulnerability scanning
Automated tooling compares your systems against a database of known vulnerabilities and misconfigurations, then reports matches. It runs in hours, costs little per run, and should be continuous rather than occasional.
What it is genuinely good for: patch hygiene at scale, catching regressions, satisfying the baseline evidence most compliance frameworks ask for, and giving you a trend line. If you are not scanning continuously, start there before considering anything else on this page.
Where it stops: a scanner can only find weakness classes someone has already described and encoded. Business-logic flaws, authorisation gaps, and findings that are only dangerous in combination are structurally invisible to it. We covered why in detail in the five blind spots every vulnerability scanner shares. The practical failure mode is a clean report read as "we are secure" when it means "we match no known signatures."
Penetration testing
A skilled human — usually with a defined scope, a time box, and your knowledge — attempts to compromise a specific system and documents what worked, how far it went, and how to fix it. Typically one to three weeks per engagement.
What it is genuinely good for: everything the scanner cannot reach. A tester can be briefed that "a user must never be able to approve their own refund" and then spend a day trying to do exactly that. They can chain three low-severity findings into one real compromise and show you the path. They can reason about your particular architecture rather than a generic one. When people say a scan "missed" something serious, a pen test is almost always the exercise that would have caught it.
Where it stops: it is scoped and announced. Your defenders usually know it is happening, which means it measures the strength of the system but not the alertness of the organisation. It is also a point-in-time artefact — a pen test result describes the code and configuration as they were that fortnight.
Our penetration testing engagements are built around that scoping conversation, because a badly scoped test is the most common way this money gets wasted.
Red teaming
A goal-oriented adversary simulation: rather than "test this application," the objective is something like "obtain access to the payment approval system," with wide latitude on route and usually without the defending team being told. Runs for weeks to months.
What it is genuinely good for: testing detection and response as an organisational capability. A red team's most valuable output is frequently not "we got in" — it is "we were inside for eleven days and the only alert that fired was closed as a false positive." That is a finding about your SOC operations, your telemetry coverage, and your escalation paths, and no amount of application testing will surface it.
Where it stops — and this is the part vendors under-explain: a red team assumes you already have detection worth testing. If your logging is incomplete or nobody is watching the alerts overnight, you do not need an expensive adversary to prove it; you already know, and the engagement will simply confirm it at high cost. Red teaming also gives deliberately incomplete coverage: the team stops at the first successful route to the objective, which means large parts of your estate go untouched by design.
We scope red team operations with that prerequisite stated openly, and we will say so if we think you are not ready for one yet.
Side by side
| Vulnerability scan | Penetration test | Red team | |
|---|---|---|---|
| Question answered | Any known weaknesses? | Can this system be broken? | Would we detect and respond? |
| Primary subject | Technology | Technology | People and process |
| Typical duration | Hours, continuous | 1–3 weeks | Weeks to months |
| Coverage | Broad, shallow | Scoped, deep | Narrow, goal-driven |
| Defenders told? | Yes | Usually | Usually not |
| Finds logic flaws? | No | Yes | Only if on the path |
| Tests your alerting? | No | Barely | Yes — that is the point |
| Relative cost | Low | Medium | High |
A maturity-ordered way to choose
These are not competing purchases; they are roughly sequential capabilities. The order that wastes the least money:
- If you are not scanning continuously — start there. It is the cheapest coverage per pound available and everything else assumes it.
- If scanning is clean but nobody has tested your logic or authorisation — buy a penetration test. This is where most organisations should be spending, and most of the genuinely serious findings we see come out of this tier.
- If you have a functioning SOC, real telemetry, and rehearsed escalation — a red team will tell you something you do not already know. Before that point, it mostly will not.
- If you want to improve detection rather than grade it — ask for a purple-team exercise instead: the offensive and defensive sides work together in the open, running known techniques while your team tunes detections against them in real time. It is often the highest learning-per-pound option, and it is badly under-bought because it does not produce a dramatic "we got in" narrative.
Three questions worth asking any vendor
- "Which of these am I actually buying?" If a proposal uses all three terms interchangeably, that is a signal about how the work will be run.
- "What will you not cover?" Every one of these exercises has a defined blind spot. A vendor who can name theirs precisely understands their own methodology.
- "What does the retest look like?" A finding you never verify as fixed is a finding you still have. Remediation verification should be in the scope from the start, not sold back to you later.
The most expensive security testing mistake is not buying the cheap option. It is buying the sophisticated option before the basics are in place, then reading a confirmation of the obvious as a sophisticated result.
If you are not certain which tier fits where you are, that assessment conversation is free and usually short — talk to an engineer, or read how we structure the underlying controls in zero trust for high-stakes platforms.
All testing described here is conducted only against systems the client owns or is explicitly authorised to assess, under written scope.
Discussion 0
Sign in or create a free account to comment and vote.
No comments yet. Be the first to share your thoughts.