SearchPublisher and Institutes
You are here: Home :: Area NEM :: Medical science :: Human medicine
D. Neveu, P. Aubas, F. Seguret, A. Kramar, P. Dujols
Measuring Agreement for Ordered Ratings in 3 3 Tables
Keywords: Weighted kappa, agreement, concordance, order relationship, maximal and partial disagreement
OBJECTIVES: When two raters consider a qualitative variable ordered according to three categories, the qualitative agreement is commonly assessed with a symmetrically weighted kappa statistic. However, these statistics can present paradoxes, since they may be insensitive to variations of either complete agreements or disagreements. METHODS: Agreement may be summarized by the relative amounts of complete agreements, partial and maximal disagreements beyond chance. Fixing the marginal totals and the trace, we computed symmetrically weighted kappa statistics and we developed a new statistic for qualitative agreements. Data sets from the literature were used to illustrate the methods. RESULTS: We show that agreement may be better assessed with the unweighted kappa index, c, and a new statistic , which assesses the excess of maximal disagreements with respect to the partial ones, and does not depend on a particular weighting system. When is equal to zero, maximal and partial disagreements beyond chance are equal. With its estimated large sample variance, we compared the values of two contingency tables. CONCLUSIONS: The (c, ) pair is sensitive to variations in agreements and/or disagreements and enables locating the difference between two qualitative agreements. The qualitative agreement is better with increasing values of c and .
Methods of Information in Medicine, Schattauer
Print ISSN: 0026-1270
Volume: 45, 01/2006
Pages: 541 - 547