We can look at the data in Table III with kappa (remember that No. 100): Kappa only addresses its maximum theoretical value of 1 if the two observers distribute codes in the same way, i.e. if the corresponding amounts are the same. Everything else is less than a perfect match. Nevertheless, the maximum value Kappa could achieve helps, as uneven distributions help interpret the actual value received from Kappa. The equation for maximum is: Kappa`s statistics are often used to test the reliability of interroteurs. The importance of the reliability of reference values lies in the fact that it represents the extent to which the data collected in the study are correct representations of the measured variables. The measurement of the extent to which data collectors assign the same score to the same variables is called the reliability of the interrater. Although there were many methods for measuring the reliability of Interraters, they were traditionally measured as a percentage of agreement, calculated as the number of chord results divided by the total number of points. In 1960, Jacob Cohen criticized the use of the agreement as a percentage because of its inability to take random agreement into account.
He introduced the Cohen-Kappa, which was designed to take into account the possibility that the spleens, due to uncertainty, guessed at least a few variables. Like most correlation statistics, the kappa can be between 1 and 1. While the Kappa is one of the most used statistics to test the reliability of interramas, it has limitations. Judgments about the level of Kappa that should be acceptable for health research are questioned. Cohen`s proposed interpretation may be too lenient for health-related studies, as it implies that a value of up to 0.41 might be acceptable. Kappa and approval percentage are compared, and levels for Kappa and percentage approval that should be requested in health studies. [-kappa-`frac`Pr[X-Y]-Pr[X-Y| X “Text” and “Y” “independent text,” |] X -Text and text “Y” -Independent]Percentage match calculation (fictitious data). An example of Kappa`s statistics calculated in Figure 3 is available. Note that the agreement percentage is 0.94, while the Kappa is 0.85 – a significant reduction in the level of congruence. The greater the expected random chord, the lower the resulting value of the Kappa.
The higher the prevalence, the lower the overall level of compliance. There is a tendency for the level of support to be lower when prevalence increases. At the .90 observer`s accuracy level, there are 33, 32 and 29 perfect matches for equipacable, variable and highly variable. To address this problem, most clinical trials now express the Interobserver agreement using Kappa`s statistics, which normally have values between 0 and 1. (The appendix at the end of this chapter shows how statistics are calculated.) A value of 0 indicates that the match observed is accurate with that expected at random, and a value of 1 indicates a perfect match. According to the agreement, a value of 0 to 0.2 indicates a slight agreement; 0.2 to 0.4 fair agreement; 0.4 to 0.6 moderate agreement; 0.6 to 0.8 essential agreement; and from 0.8 to 1.0 almost perfect match.† Rarely physical characters have values below 0 (theoretically as low as -1), suggesting that the observed chord was worse than the random chord. Cohen kappa`s statistics are a method of evaluating agreements (rather than associations) between advisors. Kappa is defined as follows: This is a simple procedure when the values are zero and one and the number of data collectors is two.
If there are more data collectors, the procedure is a little more complex (Table 2). However, as long as the values are limited to only two values, the calculation remains simple.