Hypothesis Tests and Errors
In this module, we discussed hypothesis testing. When testing hypotheses, there is always the chance that researchers make errors. We learned about the two types of errors in our readings. When we set up hypothesis tests or interpret results from others’ hypothesis tests, it is important to understand the nature of these errors.
Read through the section on Type 1 and Type 2 errors. Particularly, see pages 143-144 of the Fox textbook to help you. Create a fictional research study and respond to the following:
- For the fictional research study you created, write a null hypothesis and a research hypothesis.
- Describe in detail what the two errors would look like in your fictional test.
- Make some judgments about which type of error is worse for your fictional research study. Which error type would lead you to draw conclusions and make decisions that diverge more from reality?
- How would you set the significance level based on which error is worse?
Hypothesis testing is used to test our hypothesis about populations, and also to examine if there are differences between groups. In the first case, we establish a hypothesis about the population, collect sample data, and see how likely the sample results are given our hypothesis about the population. If the sample results are plausible under the hypothesis, we retain the hypothesis. On the other hand, if the sample results are unlikely, we then reject the hypothesis.
More frequently, when we are interested in testing theories about criminal justice phenomena, we are interested in finding out whether there are differences between groups, where the two groups are defined by the presence or absence of an important independent variable. Here, an observed difference between the groups in the dependent variable, or effect, may be attributable to the independent variable.
For instance, if you want to test your theory that regular meetings with a counselor will keep parolees from recidivating, you would collect data on parolees who meet with a counselor and those who do not (groups defined by the presence or absence of the independent variable). You would then want to know whether the length of time before they offend again (the dependent variable) is different for the two groups. If there is no difference, meeting the counselor has no effect on recidivism.
Essentially, in hypothesis testing, you are asking two questions: “Are these two samples different?” “Will these differences be present in the population?”A hypothesis test requires the following components:
- The null hypothesis: This always states that there is no difference between the two groups and serves as the starting point of the test.
- The research hypothesis: The research hypothesis states that there is a difference between the groups. Research hypotheses can be directional or non-directional. Directional research hypotheses state that the inequality between groups is in a particular direction (i.e., bigger or smaller), whereas non-directional hypotheses are not concerned with whether one group is bigger than the other, just that they are different. In the example of recidivism, the researcher expects that the higher the frequency of meetings, the lower the rate of reoffending.
- The test statistic: Recall that in the last module we talked about the sampling distribution of the mean and how it could be used to estimate the population mean. When we are trying to decide if there are differences between two groups, we are testing whether the means of the two groups are different. This difference between the group means also has a sampling distribution (i.e., the difference between the means will be different for each sample). We know the shape of the sampling distribution, and just as we used Z-scores to get the probability of a certain value occurring, we use the test statistic to test the probability of a difference between groups occurring. You do not need to know how the test statistic is calculated; you just need to know which test statistic to use and how to interpret the results.
- Significance level of the test: Once you have calculated the test statistic, you rely on the properties of its sampling distribution to determine if the difference is statistically significant. Significance levels tell us the probability that the differences we find between groups is due to chance or some other unknown error. When this probability is low (less than .05 or .01), any differences we find are not due to chance and will be found in the population.
Once we understand the basic components of a hypothesis test, we can learn how to set them up and interpret the results. The PowerPoint presentation will explain how to do this.
- Fox, Levin, & Forde, Elementary Statistics in Criminal Justice Research (4th ed.)
- Chapter 7: Testing Differences Between Means
Important: In the previous modules you have used the Salkind textbook for your labs and have been asked to work in Excel as you read along. For this module, do not worry about actually running the tests in Excel. Focus on the explanations of the tests, when to use them and how to interpret them.
- Salkind, Statistics for People Who (Think They) Hate Statistics: Using Microsoft Excel 2016 (4th ed.)
- Chapter 7: Hypotheticals and You: Testing Your Questions
- Chapter 9: Significantly Significant: What It Means for You and Me
- Chapter 10: Only the Lonely: The One-Sample Z test
- Chapter 11: t(ea) for Two: Tests Between the Means of Different Groups
- Chapter 12: t(ea) for Two (Again): Tests Between the Means of Related Groups
- Module notes
/* Style Definitions */
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
font-family:”Times New Roman”,serif;