Comparing Two Populations: Paired Difference Experiment

McNemar's test is a normal approximation used on nominal data. It is applied to $2 \times 2$ contingency tables with a dichotomous trait, with matched pairs of subjects, to determine whether the row and column marginal frequencies are equal ("marginal homogeneity").

A contingency table used in McNemar's test tabulates the outcomes of two tests on a sample of $n$ subjects, as follows:

$2 \times 2$ Contingency Table

A contingency table used in McNemar's test tabulates the outcomes of two tests on a sample of $n$ subjects.

The null hypothesis of marginal homogeneity states that the two marginal probabilities for each outcome are the same, i.e. $p_a + p_b = p_a + p_c$ and $p_c + p_d = p_b + p_d$. Thus, the null and alternative hypotheses are:

${ H }_{ 0 }:{ p }_{ b }={ p }_{ c }$

${ H }_{ 1 }:{ p }_{ b }\neq { p }_{ c }$

Here $p_a$, etc., denote the theoretical probability of occurrences in cells with the corresponding label. The McNemar test statistic is:

$\displaystyle{{ \chi }^{ 2 }=\frac { { \left( b-c \right) }^{ 2 } }{ b+c }}$

Under the null hypothesis, with a sufficiently large number of discordants, ${ \chi }^{ 2 }$ has a chi-squared distribution with $1$ degree of freedom. If either $b$ or $c$ is small ($b+c<25$) then ${ \chi }^{ 2 }$ is not well-approximated by the chi-squared distribution. The binomial distribution can be used to obtain the exact distribution for an equivalent to the uncorrected form of McNemar's test statistic. In this formulation, $b$ is compared to a binomial distribution with size parameter equal to $b+c$ and "probability of success" of $\frac{1}{2}$, which is essentially the same as the binomial sign test. For $b+c<25$, the binomial calculation should be performed. Indeed, most software packages simply perform the binomial calculation in all cases, since the result then is an exact test in all cases. When comparing the resulting ${ \chi }^{ 2 }$ statistic to the right tail of the chi-squared distribution, the $p$-value that is found is two-sided, whereas to achieve a two-sided $p$-value in the case of the exact binomial test, the $p$-value of the extreme tail should be multiplied by $2$.

If the ${ \chi }^{ 2 }$ result is significant, this provides sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis that $p_b \neq p_c$, which would mean that the marginal proportions are significantly different from each other.