iaa.kappa {UCS} R Documentation

## Inter-Annotator Agreement: Cohen's Kappa (iaa)

### Description

Compute the kappa statistic (Cohen, 1960) as a measure of intercoder agreement on a binary variable between two annotators, as well as a confidence interval according to Fleiss, Cohen & Everitt (1969). The data can either be given in the form of a 2-by-2 contingency table or as two parallel annotation vectors.

### Usage

```iaa.kappa(x, y=NULL, conf.level=0.95)
```

### Arguments

 `x` either a 2-by-2 contingency table in matrix form, or a vector of logicals `y` a vector of logicals; ignored if `x` is a matrix `conf.level` confidence level of the returned confidence interval (default: 0.95, corresponding to 95% confidence)

### Value

A data frame with a single row and the following variables:
 `kappa` sample estimate for the kappa statistic `sd` sample estimate for the standard deviation of the kappa statistic `kappa.min, kappa.max` two-sided asymptotic confidence interval for the “true” kappa, based on normal approximation with estimated variance
The single-row data frame was chosen as a return structure because it `print`s nicely, and results from different comparisons can easily be combined with `rbind`.

### References

Cohen, Jacob (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.

Fleiss, Joseph L.; Cohen, Jacob; Everitt, B. S. (1969). Large sample standard errors of kappa and weighted kappa. Psychological Bulletin, 72(5), 323–327.

`iaa.pta`

### Examples

```## kappa should be close to zero for random codings
p <- 0.1			# proportion of true positives
x <- runif(1000) < p		# 1000 candidates annotated randomly
y <- runif(1000) < p
iaa.kappa(x, y)
```

[Package UCS version 0.5 Index]