Communication · Executive Insight Brief · 395
The agreement table matters as much as kappa
A chance-adjusted agreement statistic becomes decision-useful only when executives inspect the marginals, disagreement pattern, rater design, and consequences behind the number.
Premise
Cohen’s kappa (κ) answers a narrower question than many dashboards imply. For two raters classifying the same units into categories, it summarizes agreement after subtracting the agreement implied by the raters’ marginal category frequencies. It does not determine whether either rater is correct, whether the codebook is valid, or whether the remaining disagreements matter operationally. The executive use is therefore quality control over a rating process—not certification by a single coefficient.
What the model is: Cohen’s Kappa in Knowledge →
Continue in the series
- The essential model
- Why this matters
- Risks and limits
- Executive takeaway
- Key questions
- Selected sources
The full brief is in its volume, with the other works of the series.
Need the full brief now? Request it under NDA →
New works are announced in The Debrief. Subscribe →
- FamilyTool
- CategoryMeasurement & Assessment Instruments
- DomainEvidence & Evaluation
What these mean
Editorial orientation · Family
Tool
In this series, Tool names the editorial family primarily used to act on, examine, measure or organize communication. It includes instruments, methods, practices, protocols, codes and institutional arrangements. Membership does not certify effectiveness, portability or safe use in every setting.
Editorial orientation · Category
Measurement & Assessment Instruments
Produces measures, scores, profiles or assessments through defined items, observations or measurement rules. The entry identifies what is measured and the conditions governing interpretation.
- KnowledgeCohen’s Kappa