A quality management system without indicators is little more than a statement of intent. What turns quality into a governable discipline are metrics and KPIs (key performance indicators) — they translate processes into comparable figures, set targets and trigger action whenever reality drifts from the plan. This article explains how to build a genuinely useful quality scorecard, which indicators actually matter, how to frame them within ISO 9001:2015, and which mistakes cause a dashboard full of charts to be ignored when decisions need to be made.
Why measure: from control to improvement
Clause 9.1 of ISO 9001:2015, "Monitoring, measurement, analysis and evaluation," requires the organisation to determine what needs measuring, when, and how to analyse the results. Collecting data is not enough: the standard demands that those data be converted into performance evaluation and into inputs for the management review process (clause 9.3). Measuring well is the prerequisite for the continual improvement articulated in clause 10.
The fundamental distinction is between a metric (any value that is measured) and a KPI (the metric that is linked to a target and that, when it deviates, demands action). A scorecard with fifty metrics and no associated targets is not a scorecard — it is noise. Selecting a small number of meaningful KPIs, anchored to the quality strategy, is what gives the dashboard real value.
Leading indicators versus lagging indicators
Quality KPIs fall into two broad families. Lagging indicators measure outcomes that have already occurred: defects per million opportunities (DPMO), return rates, cost of poor quality, and customer complaints. They are reliable but arrive late. Leading indicators anticipate the outcome: number of audits completed, preventive actions opened, training delivered, or process capability. A well-balanced scorecard combines both: leading indicators to allow timely correction, lagging indicators to confirm the result.
The essential quality KPIs
Although every industry has its own, there is a common core worth monitoring. The defect rate, or non-conformities per unit produced, directly measures the output of the process. The cost of poor quality (CoPQ) aggregates the costs of internal failures, external failures, appraisal, and prevention — and typically reveals figures that surprise management. The average time to close non-conformities indicates how quickly the system corrects problems. Customer satisfaction, measured by NPS or structured surveys, closes the loop with the market. And the effectiveness of corrective actions (the percentage of non-conformities that do not recur) measures whether the system is genuinely learning.
Process capability: Cp and Cpk
In manufacturing processes, the capability indices Cp and Cpk are the technical KPIs par excellence. Cp compares the width of the specification limits with the natural spread of the process, while Cpk adds the effect of centring relative to the target value. A Cpk of 1.33 is considered the minimum acceptable in many sectors, and 1.67 or higher corresponds to highly capable processes. Statistical process control (SPC) using control charts allows the distinction between common-cause variation (inherent to the process) and special-cause variation (a specific cause that must be investigated), preventing over-adjustment that actually worsens quality.
The mistake of adjusting the process after every random variation has a name in the quality literature: tampering, illustrated by Deming's famous funnel experiment. Reacting to common-cause variation as though it were special cause increases total dispersion rather than reducing it. That is why a well-designed scorecard does not simply show the latest value of an indicator but presents it on a control chart with calculated limits, so that whoever is responsible can see at a glance whether a deviation warrants investigating a specific cause or whether it is simply part of the natural noise of the process.
The cost of poor quality: the KPI that convinces management
Of all indicators, the one with the greatest impact on the board of directors is the cost of poor quality (CoPQ), because it translates quality into business language: money. It comprises four categories. Prevention costs (training, preventive maintenance, robust design) and appraisal costs (inspections, testing, audits) are voluntary investments. Internal failure costs (rework, scrap, downtime) and external failure costs (returns, warranties, customer loss, reputational damage) are the consequences of not having invested in time.
The relationship between these categories follows the well-known 1-10-100 rule: fixing a defect at the design stage costs one unit, fixing it in production costs ten, and resolving it once it has reached the customer costs one hundred. Measuring CoPQ and breaking it down by category makes the business case that investing in prevention is not a cost but the most profitable lever in the system. It is common for the first serious CoPQ measurement to reveal figures equivalent to a significant percentage of turnover, transforming quality into a profitability conversation rather than a compliance exercise.
| Characteristic | Metric | KPI |
|---|---|---|
| Associated target | Not necessarily | Always |
| Action on deviation | Optional | Mandatory |
| Linked to strategy | Sometimes | Always |
| Example | Number of units produced | Cpk ≥ 1.33 on the critical process |
| Review frequency | Variable | Defined and periodic |
Building the scorecard using the SMART framework
Each KPI must be defined against SMART criteria: specific (what exactly is measured and with which formula), measurable (with a reliable and automatable data source), achievable (a realistic target based on the baseline), relevant (linked to a quality or business objective), and time-bound (with a horizon and a measurement frequency). The data card for each indicator must document the formula, the data source, the owner, the target, the alarm threshold and the review frequency. Without that card, the same KPI gets calculated differently depending on who is looking at it.
Implementation steps
- Start from the quality objectives: each KPI must serve a stated objective; if it does not serve any, it has no place on the dashboard.
- Establish the baseline: measure the current state before setting targets, so that goals are realistic rather than arbitrary.
- Automate data capture: integrate the indicator with the ERP, MES or management system to avoid error-prone manual spreadsheets.
- Define thresholds and owners: each KPI should have an owner who acts when the alarm threshold is crossed.
- Review in the management review: bring the scorecard to clause 9.3 so that improvement decisions are driven by data, not impressions.
Common mistakes in quality scorecards
The most widespread mistake is the vanity metric: dashboards that display impressive numbers that nobody uses to decide anything. The second is measuring what is easy to capture rather than what is important, because the relevant data is harder to collect. The third is the perverse effect of a poorly designed indicator: if speed of closing non-conformities is rewarded, they will be closed without analysing the root cause, and the same problems will recur. The fourth is never revisiting the set of KPIs: a scorecard that never evolves ends up measuring processes that have already changed.
Frequently asked questions
How many KPIs should a quality scorecard have?
Enough to cover the critical objectives and as few as possible so that they are genuinely reviewed. A typical range is five to fifteen KPIs per level of responsibility. Beyond that, attention disperses and the dashboard stops being used for decisions.
Does ISO 9001 require specific KPIs?
It does not prescribe specific indicators. Clause 9.1 requires the organisation to determine what to measure and to analyse the results, but leaves the choice of appropriate KPIs to the organisation according to its context. What the standard audits is the coherence between objectives, measurement and improvement actions.
What is the difference between Cp and Cpk?
Cp measures the potential capability of the process assuming it is perfectly centred, while Cpk accounts for the actual centring relative to the target value. A process can have a good Cp and a poor Cpk if it produces parts within tolerance but shifted towards one limit.
How often should KPIs be reviewed?
It depends on the indicator: operational ones may be monitored daily or by shift, tactical ones monthly, and strategic ones at the management review, at least once a year. Each KPI data card should specify its own review frequency.
Conclusion
A quality scorecard only delivers value when every figure has a target, an owner, and a threshold that triggers action. The key is not the number of indicators but the balance between leading metrics that allow timely correction and lagging metrics that confirm the result — all of it anchored to clause 9.1 of ISO 9001 and feeding the management review. Measuring the cost of poor quality and the true capability of processes, rather than merely impressive-looking numbers, is what turns the dashboard into a decision-making tool. At Summum Calidad we design scorecards with documented KPI data cards, automated data capture and actionable thresholds, so that your organisation's quality is managed on evidence, not on perception.