Why do your dashboards lie while your pipelines tell the truth?
You check the dashboard and everything looks fine, yet the business feels wrong. The dashboard is not broken; it is being gamed, and Goodhart's law explains why.
The law that breaks your KPIs
Wikipedia's entry on Goodhart's law states the adage plainly: "When a measure becomes a target, it ceases to be a good measure." Charles Goodhart expressed the core idea in a 1975 article on UK monetary policy. His original wording is sharper: "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." A dashboard metric you manage by is a measure under pressure. That is exactly the condition where it starts to fail.
- Related ideas predate Goodhart; Campbell's law formulations date to 1969
- Jerome Ravetz made a similar point in his 1971 book on scientific knowledge
Dashboards add an incentive layer
A dashboard is a graphical interface that summarizes information for viewing, per Wikipedia's article on business dashboards. That summary is the problem. The moment a number becomes visible and tied to accountability, teams optimize the number. The raw data flowing through your pipeline has no audience and no incentive. It just records what happened, which is why it keeps telling the truth after the dashboard has stopped.
Data quality is judged by fitness, not purity
A gamed metric can look consistent and complete while no longer representing the real-world construct it refers to. Wikipedia's data quality article defines quality as accuracy, completeness, consistency, reliability, and fitness for intended purpose. Data is high quality if it is "fit for [its] intended uses in operations, decision making and planning". Note the trap: a gamed metric can look consistent and complete while no longer representing the real-world construct it refers to. The same article warns that people's views on data quality often disagree, even for the same data used for the same purpose.
- Quality is relative to use, not absolute
- Disagreement about quality is normal; standards exist partly for this
Read the pipeline before the dashboard
The practical fix is to compare the two. Your pipeline is the unpressured record; your dashboard is the pressured summary of it. When they diverge, the divergence itself is the finding. Per the Goodhart's law entry, the regularity collapses specifically when pressure is applied for control purposes, so look hardest at the metrics tied to bonuses, reviews, and OKRs.
What this does not solve
Applying it to your SaaS dashboard is inference, not tested result. This framing does not tell you which metrics to use instead, nor how often to re-baseline them. Wikipedia's sources here are encyclopedic and definitional, not empirical studies of metric gaming in companies. Goodhart's law also predates modern analytics by decades; applying it to your SaaS dashboard is inference, not tested result.
Which metric on your main dashboard is tied to someone's bonus, and when did you last compare it to raw pipeline data?
If your dashboard and pipeline disagreed this week, who in your org would actually notice?
Pick your most-reviewed dashboard metric and pull the underlying raw numbers for the same period. Compare them today.
Sources
Recommended for you
- Data
Why does GDPR see personal data where you see none?
Your AI product's logs and inputs probably count as personal data under GDPR, because the law's definition is far broader than the everyday idea of PII.
- Data
Why does data quality belong to the business, not the data team?
Your data team can check schemas, but only the business knows what the data is for. Quality is fitness for purpose, and purpose lives outside engineering.
- Data
When should you store raw data instead of cleaning it first?
Your data choice locks in who can use it. Here is what the definitions alone tell you about warehouse versus lake.
- Data
Why does hiring one AI engineer still leave your data broken?
You hired for AI and still got outputs that contradict your dashboards. The problem is the role boundary nobody drew between data work and AI work.
Enjoyed this article?
Subscribe for new articles. No spam. Unsubscribe anytime.