4.2.3 Likelihood

In both Fisher’s approach and the Neyman-Pearson framework, the p-value represents the probability of getting the observed test statistic, or anything more extreme. In the the Neyman-Pearson framework, the ’more extreme’ aspect of this definition is determined by $H_{a}$. Thus, $p = P (data \mid Hypothesis)$ (or, $p = P (data \mid H_{o})$ for Neyman-Pearson). Both of these definitions of $p$ are based on the frequentist notion of probability and assume that the parameter being estimated is fixed. In our example, $\mu $ never changes, we just get different values of $\bar y$ every time we sample.

While working on ways to estimate parameter values, Fisher developed methods that began to alter this assumption. What if we assume that the data (e.g., our $\bar y$) is fixed, and we let the parameter value be the variable? Conceptually, this may sound like a pointless mental exercise, but the mathematical implications are significant. For example, here, instead of having point estimates of a parameter, we deal with the entire probability distribution of the parameter of interest. Although this new concept of probability, sometimes called ’inverse probability’, proved useful, Fisher was uncomfortable even calling it probability. Instead, he used the term likelihood. Given a hypothesis, the likelihood of that hypothesis can be represented as

\begin{equation} L(Hypothesis \mid data) = aP(data \mid Hypothesis). \end{equation}

This can roughly be translated as, the likelihood of our hypothesis, given the data we have collected, is equal to the probability of the data, assuming that our hypothesis is true, multiplied by a constant $a$. The actual calculation of the likelihood is a bit more complicated than the relative frequencies that have been used up to now. For this discussion, we will ignore the details of how to calculate the likelihood and instead focus on its application at a conceptual level. In our example, we are focusing on the likelihood that $\mu =5$, given that we have $\bar y = 6.6$.

This concept can be used to estimate parameters and we will see its application in Chapter. It can also be used in a simple fashion to compare hypotheses. For example, let $L(H_{1})$ be the likelihood of some hypothesis, which we will label as 1, and $L(H_{2})$ be the likelihood of hypothesis 2. Note that here we do not explicitly focus on a null and alternate hypothesis, rather, we are simply interested in comparing the support of one hypothesis over the other, given observed data. The likelihood ratio (LR) is defined as

[ LR = \frac{L(H_{1})}{L(H_{2})}. ]

If $LR > 1$, then the data support $H_{1}$, if $LR < 1$, then the data support $H_{2}$. Note the use of the LR to support one hypothesis over another. This is a break from the classical falsificationist approach.

However, the LR also can be used in a classical null hypothesis testing approach. To do so, we define $LR = L(H_{a})/L(H_{o})$. It can be shown that, under certain circumstances, this LR ratio will follow a $\chi ^{2}$ distribution. The $\chi ^{2}$ distribution will be introduced in Chapter 5. Here, you can simply think of it as a theoretical sampling distribution to which the observed LR can be compared using the logic of the Neyman-Pearson framework. Thus, it is the theoretical sampling distribution of the LR assuming that $H_{0}$ is true. Once the observed LR is calculated, a p-value can be calculated based on its location in the $\chi ^{2}$ distribution.