Hypothesis Testing Formulas

Cheatsheet Content

1. ANOVA (Analysis of Variance) Overview ANOVA is a statistical test used to compare the means of three or more groups to determine if at least one group mean is significantly different from the others. It uses the F-distribution to compare the variance between groups to the variance within groups. One-Way ANOVA: Compares means of groups based on one categorical independent variable (factor). Two-Way ANOVA: Compares means based on two categorical independent variables. 2. One-Way ANOVA Hypothesis Test Procedure Step 1: State the Hypotheses Null Hypothesis ($H_0$): All group means are equal. $H_0: \mu_1 = \mu_2 = \dots = \mu_k$ (where $\mu_i$ is the mean of the $i$-th group, and $k$ is the number of groups) Alternative Hypothesis ($H_1$ or $H_a$): At least one group mean is different from the others. $H_1:$ Not all $\mu_i$ are equal. Step 2: Set the Significance Level ($\alpha$) Choose a significance level (e.g., $\alpha = 0.05$ or $\alpha = 0.01$). This is the probability of making a Type I error (rejecting $H_0$ when it is true). Step 3: Check Assumptions Independence: The samples are randomly and independently drawn from each population. Normality: The populations from which the samples are drawn are normally distributed. (Can be relaxed for large sample sizes per group, $n_i \ge 30$, or if the central limit theorem applies). Homoscedasticity (Equality of Variances): The variances of the populations are equal. (Can be checked using Levene's test or Bartlett's test). If violated, consider Welch's ANOVA or transformations. Step 4: Calculate the Test Statistic (F-statistic) The F-statistic is calculated as the ratio of the Mean Square Between Groups (MSB) to the Mean Square Within Groups (MSW). Sum of Squares Total (SST): Total variation in the data. $SST = \sum_{i=1}^{k} \sum_{j=1}^{n_i} (x_{ij} - \bar{\bar{x}})^2$ where $x_{ij}$ is the $j$-th observation in the $i$-th group, $n_i$ is the sample size of the $i$-th group, and $\bar{\bar{x}}$ is the grand mean. Sum of Squares Between Groups (SSB) / Sum of Squares Factor (SSF): Variation between group means. $SSB = \sum_{i=1}^{k} n_i (\bar{x}_i - \bar{\bar{x}})^2$ where $\bar{x}_i$ is the mean of the $i$-th group. Sum of Squares Within Groups (SSW) / Sum of Squares Error (SSE): Variation within groups (error). $SSW = \sum_{i=1}^{k} \sum_{j=1}^{n_i} (x_{ij} - \bar{x}_i)^2$ Note: $SST = SSB + SSW$ Degrees of Freedom: $df_{between} = k-1$ $df_{within} = N-k$ (where $N = \sum n_i$ is the total sample size) $df_{total} = N-1$ Mean Square Between Groups (MSB): $MSB = \frac{SSB}{df_{between}} = \frac{SSB}{k-1}$ Mean Square Within Groups (MSW): $MSW = \frac{SSW}{df_{within}} = \frac{SSW}{N-k}$ F-statistic: $F = \frac{MSB}{MSW}$ Often presented in an ANOVA table: Source of Variation Sum of Squares (SS) Degrees of Freedom (df) Mean Square (MS) F Between Groups (Factor) $SSB$ $k-1$ $MSB = \frac{SSB}{k-1}$ $F = \frac{MSB}{MSW}$ Within Groups (Error) $SSW$ $N-k$ $MSW = \frac{SSW}{N-k}$ Total $SST$ $N-1$ Step 5: Determine the Critical Region (or P-value) Critical Value Approach: Find the critical F-value ($F_{\alpha, df_{between}, df_{within}}$) from the F-distribution table for the given $\alpha$, $df_{between}$, and $df_{within}$. The critical region is $F > F_{\alpha}$. P-value Approach: Calculate the P-value associated with the calculated F-statistic and its degrees of freedom. This is the probability of observing an F-statistic as extreme as, or more extreme than, the one calculated, assuming $H_0$ is true. Step 6: Make a Decision Critical Value Approach: If $F_{calculated} > F_{critical}$, reject $H_0$. If $F_{calculated} \le F_{critical}$, fail to reject $H_0$. P-value Approach: If P-value $\le \alpha$, reject $H_0$. If P-value $> \alpha$, fail to reject $H_0$. Step 7: State the Conclusion If $H_0$ is rejected, conclude that there is statistically significant evidence that at least one group mean is different from the others. (This does not tell which specific means differ; post-hoc tests are needed for that). If $H_0$ is not rejected, conclude that there is not enough evidence to suggest that the group means are different. 3. Post-Hoc Tests (if $H_0$ is rejected) If the ANOVA F-test is significant, it indicates that at least one group mean is different, but not which ones. Post-hoc tests (e.g., Tukey's HSD, Bonferroni, Scheffé, Dunnett's) are used for pairwise comparisons to identify specific differences while controlling the family-wise error rate. Tukey's Honestly Significant Difference (HSD): Commonly used when comparing all possible pairs of means.