How to measure penetration testing team performance

The calendar is full, utilisation looks healthy, and reports are still waiting for review. If you run a pentest team, that combination deserves a closer look: the booking figures tell you little about where the work is getting stuck.

A queue caused by a lack of reviewer time needs a different response from repeated rewrites caused by missing evidence. Useful measures help you tell those situations apart, so you can spend time on the right problem.

You can start with the question you need answered and the records you already have. Understanding one delay is a better starting point than collecting numbers for every part of the service at once.

Use the framework to choose where to look

The Conversec Pentest Team Performance Framework groups operational evidence into five dimensions:

Dimension	Question it answers	Example indicators
Delivery flow	Does work move predictably from scope to closure?	Scope readiness, blocked days, report turnaround
Quality	Is work ready at the expected review point?	First-pass acceptance, material QA defects, review cycles
Client friction	Where does ambiguity create avoidable work?	Clarification events, severity disputes, escalations
Capacity resilience	How dependent is delivery on rescue work?	Senior intervention, unbilled delivery load, reviewer queues
Consultant development	Is ownership becoming repeatable across the team?	Independently demonstrated delivery responsibilities

Read related results together. A shorter turnaround can conceal a quality problem if reviewers have started passing reports with unsupported conclusions. Additional senior help may also explain an apparent improvement without making the normal workload sustainable.

The Pentest Team Maturity Scorecard serves a different purpose. It is a self-assessment across seven areas of team practice. Use it to identify where to investigate, then use the framework to choose operational measures. A scorecard result does not convert into a framework score.

If you need another pair of eyes on the problem, a Pentest Team Diagnostic adds a focused review of reports, QA comments and delivery records with your team. Its findings explain what the sample supports and what still needs checking; further implementation support is a separate choice.

The five dimensions and measures are Conversec’s way of looking at delivery. The CREST Defensible Penetration Test guidance provides context for objectives, scoping, competent practitioners and reporting. It does not define or endorse these measures.

Define the measure before collecting it

A measure becomes useful when everyone means the same thing by it. Write down what you are counting or timing, where the information comes from and who checks it. Agreed exclusions and recorded changes to the definition make later comparisons easier to trust.

For example, the framework defines report turnaround as the median number of business days from the planned end of testing to the client-ready draft. Measuring from actual testing completion instead would exclude some delay. Measuring to final sign-off would include a different part of the process. Those results should not be compared as if they describe the same thing.

For QA measures, agree what constitutes a material defect. A wrong impact claim needs different treatment from a reviewer’s preference about sentence order. The QA standard gives the team a basis for making that distinction.

Use the same definitions across reporting periods. If a definition changes, mark the change alongside the results so the apparent trend is not mistaken for a delivery improvement.

Work through one delivery problem

Imagine reviewing ten completed engagements. Four reports needed material correction at first review, leaving six that passed without it. Three engagements needed unplanned senior help, and median report turnaround was four business days. These are illustrative figures to work through, rather than client results or a benchmark.

The numbers give you a reason to look at the review records. Suppose those records show that testers captured evidence without enough account context. Senior reviewers then had to reconstruct what happened before approving the findings. You now have a specific problem to work on.

That supports a specific change: require an evidence check before testing access ends. Give the delivery lead responsibility for arranging it on relevant engagements. Keep the existing report quality requirements.

At the next agreed review, compare turnaround and first-pass acceptance with the earlier period. Check whether the same evidence defect recurred and whether the added check consumed more time than expected. Include senior support in that assessment.

If quality holds and reconstruction work falls, continue the change and gather more results. If reports still spend most of their time waiting for a reviewer, investigate reviewer availability next. The initial change may address a real defect without resolving the main delay.

Interpret costs and client questions carefully

Include informal support when recording engagement effort. The framework’s unbilled delivery load measures unplanned non-billable recovery hours as a share of recorded engagement hours. It does not treat every non-billable activity as a failure. Planned mentoring and ordinary review have a place in the service allocation.

Client clarification needs a record of its cause. A remediation engineer asking about an unusual implementation choice may be using the report well. Repeated requests to identify the affected endpoint suggest that the report lacks necessary detail.

Apply the same care to severity disputes. Record whether new information changed the assessment or whether the original reasoning was unclear. A disputed rating is not, by itself, evidence of a poor finding.

Use the results in a delivery review

Show the number of engagements behind each rate. Note unusual work and changes in reviewer availability. Small samples can move sharply, so investigate a result before treating it as a stable trend.

For junior development, retain examples of responsibilities completed independently at the required quality level. Fewer QA comments may support that assessment, but could also reflect easier assignments or less thorough review. Discuss the actual work with the reviewer.

Finish the delivery review with something the team can act on: one change, someone to take it forward and a date to look at the results. At the next review, check whether the work improved and whether the change made anything else harder. Another measure earns its place when it answers a question the existing records leave open.