Learning objectives
Name the four stages of a statistical investigation — formulate a question, collect data, analyze the data, interpret the results — and place a described activity in the right stage
Identify the population, the sample, the observational units and the variables in a study, using N and n for the two sizes, and explain why a sample is studied
Write an investigative question that names the population, the variable or variables and the summary of interest, and tell it apart from a question that has one fixed answer
State the three components of an investigative question — the data, the analysis and the conclusion — and explain why the question must be feasible and settled before the data are examined
Key definitions
Statistical investigation
A way of answering a question with data in four stages: formulate a question, collect data, analyze the data and interpret the results in context.
Investigative question
A question about a population that can only be answered by collecting data, because the answers vary from one observational unit to the next. It names the population, the variable or variables and what is to be summarized or compared.
Population and sample
The population is every individual or item the question is about; its size is written N. The sample is the part of the population from which data are actually obtained; its size is written n.
Observational unit
One individual or item on which data are recorded, such as one student, one household or one battery.
Variable
A characteristic recorded for every observational unit that can take different values from one unit to another.
Datum and data set
A datum is one entry in the records: the value noted for one variable on one observational unit. The data set is everything the study recorded, taken together.
Explanation
The four stages. Every statistical investigation moves through the same four stages, and the exam is built around them: formulate a question, collect data, analyze the data and interpret the results. Formulating means deciding exactly what you want to know, and about whom. Collecting means choosing a census, a sample survey, an observational study or an experiment, and carrying it out. Analyzing means graphs, summary statistics and, later in the course, confidence intervals and significance tests. Interpreting means answering the original question in context and saying how far the answer can be trusted. The order matters because the question drives every later choice: it decides which variables are recorded, which procedure is used and what kind of conclusion is possible. You may be asked to place an activity in its stage: choosing 120 names at random is collecting, drawing a histogram of the replies is analyzing, and writing "about a third of the students at this school sleep for at least eight hours" is interpreting.
The parts of a study. A school has 1840 students, and a researcher asks 120 of them, chosen at random, how long they slept last night. The population is the whole group the question is about — all 1840 students — and its size is written N, so N = 1840. The sample is the part of the population from which data are actually obtained — the 120 students asked — and its size is written n, so n = 120. Each student is an observational unit, the individual on which data are recorded, and hours of sleep is the variable, a characteristic that differs from one unit to another. One recorded value, such as 7.5 hours, is a datum; the 120 values together form the data set. A sample is studied because measuring every member of a population is usually impossible, too slow or too expensive, and a well-chosen sample still lets us draw a conclusion about the whole population. Every answer in this course must be given in context: write "the hours of sleep of students at this school", never just "the data".
What makes a question investigative. An investigative question is one that can only be answered with data, because the answer varies from one observational unit to the next. "How long did Minh sleep last night?" has one fixed answer and needs no statistics. "What is the mean number of hours that students at this school sleep on a school night?" is investigative: sleep times vary, so data from many students and a summary of those data are needed. A good investigative question has a clear purpose, names the population, names the variable or variables, and says what is to be summarized or compared — a proportion, a mean, a difference between two groups or a relationship between two variables. Keep it separate from the question put to each person when the data are gathered. "How many hours did you sleep last night?" is answered by one student and produces one datum; the investigative question is about the whole group.
The three components. A well-written investigative question tells you three things. First, the data: which variables must be recorded, and on which observational units. Second, the analysis: either an estimate of a population value, which later in the course becomes a confidence interval, or a decision about a claim, which becomes a significance test — and for a test the wording fixes the direction: greater than, less than or different from. Third, the conclusion: the population to which the answer will apply and the kind of statement that will be possible, including whether cause and effect can be claimed. Compare two questions. "What proportion of students at this school sleep for at least eight hours on a school night?" needs one categorical variable, calls for an estimate, and its answer applies to this school. "Do students at this school who stop using screens an hour before bed fall asleep faster, on average, than those who do not?" needs a quantitative response (minutes taken to fall asleep) and a grouping variable, calls for a test with a direction (faster means a smaller mean), and can support a cause-and-effect conclusion only if the screen rule is assigned to students at random in an experiment.
From question to method. Because the question fixes the variables and the summary, it already points to the procedure that will answer it. One categorical variable in one group leads to a single proportion; the same variable compared across two groups leads to a difference of two proportions. One quantitative variable in one group leads to a single mean; two measurements on the same units, such as before and after, lead to a mean difference from paired data; and one quantitative variable compared across two separate groups leads to a difference of two means. Two categorical variables lead to a chi-square test, and two quantitative variables lead to a scatterplot, correlation and regression. In an inference question the hypotheses are read straight from the investigative question: "Is the proportion greater than 0.30?" becomes H₀: p = 0.30 against Hₐ: p greater than 0.30.
Settle the question first, and make it answerable. The investigative question is fixed before the data are examined. If you look through the data and then choose the question the data happen to support — reporting whichever comparison came out largest, or switching from "different" to "greater" once you have seen the direction — the pattern you report may be nothing more than chance, and the analysis no longer means what it claims. The question must also be feasible: the data it calls for can really be collected and analyzed with the access, time and measurements available. "Is the new timetable better?" fails on every count: better at what, for whom, and measured how? A workable version is "Is the proportion of students at this school who are late at least once a week lower under the new timetable than it was under the old one?" The population, the variable, the summary and the direction are now stated, and the attendance register can supply the data.
Formulae
N = number of individuals in the population; n = number of individuals in the sample
Stages: formulate a question → collect data → analyze the data → interpret the results
Components of an investigative question: the data (variables) → the analysis (an estimate, or a test with its direction) → the conclusion (population, and association or cause)
Worked examples
q: A local newspaper claims that teenagers in Riverton who have a part-time job get less sleep. Riverton has 5200 residents aged 13 to 19, and a researcher can survey 200 of them, chosen at random. (a) Write an investigative question for the claim. (b) Identify the population, the sample, the observational units and the variables, giving N and n. (c) State the three components of the question. (d) Say what conclusion the study could support. (e) After the survey the researcher notices that sleep hardly differs by job but differs by year group, and wants to report that instead. Comment.
steps: (a) "Do Riverton teenagers who have a part-time job sleep for fewer hours on a school night, on average, than Riverton teenagers who do not?" The question names the population, the two variables and the summary to compare, with a direction.
(b) The population is all 5200 Riverton residents aged 13 to 19, so N = 5200. The sample is the 200 teenagers surveyed, so n = 200. Each teenager is an observational unit. The variables are job status (has a part-time job or not), which is categorical, and hours of sleep on a school night, which is quantitative.
(c) Data: job status and hours of sleep for every teenager in the sample. Analysis: a comparison of two means by a significance test whose direction is "less than" for the group with jobs. Conclusion: a statement about all Riverton teenagers, because the 200 were chosen at random.
(d) Nobody is assigned a job at random; the teenagers decide for themselves. The study can therefore show that having a job is associated with less sleep among Riverton teenagers, but not that the job causes the lost sleep. Older teenagers, for example, may be both more likely to work and more likely to sleep less.
(e) The year-group comparison was chosen after looking at the data, so it must not be reported as the answer to the investigation: among many possible comparisons, one will often look striking by chance. It can be written down as a new question and tested with fresh data.
In context
Leading medical journals will publish a clinical trial only if its main question and outcome measure were registered before the first patient was recruited, so that the result reported at the end answers the question that was asked, not one chosen after seeing the data.
A census office spends years settling the purpose and wording of every item before a form is sent out, because a question that cannot be answered consistently produces data that cannot be analyzed.
Common misconceptions
Students often treat any question as a statistical one. In fact a question with one fixed answer, such as the height of one named student, involves no variability and needs no data analysis.
Students often give the question asked of each person as the investigative question. In fact "How many hours did you sleep?" produces one datum, while the investigative question is about the whole population and names a summary such as a mean or a proportion.
Students often think the question can be adjusted once the data are in. In fact changing it after the analysis means the reported result may only reflect a chance pattern in that particular data set.
Students often call the people who answered "the population". In fact those people are the sample, of size n; the population, of size N, is the whole group the question is about.
Study tips
The first free-response question of the exam is set on formulating questions and collecting data. When you write an investigative question, check three things: the population is named, the variable is named, and the summary or comparison (a mean, a proportion, a difference, a relationship) is stated.
If the question involves a claim, state its direction — greater, less or different — because that direction becomes the alternative hypothesis later.
Give N and n with their numbers and name the observational unit; "students" alone does not say whether you mean the 120 sampled or all 1840.
For the conclusion component, say which population the answer applies to and whether a causal statement is possible; cause and effect needs random assignment, which only an experiment has.
Reasoning
Once the question is fixed, the next decision is how to collect the data: the choice of sampling method or experimental design follows from it.
Summary
A statistical investigation has four stages: formulate a question, collect data, analyze the data, interpret the results.
The population (size N) is everyone the question is about; the sample (size n) is the part actually measured.
An investigative question needs data because answers vary; it names the population, the variable and the summary.
Its three components point to the data, the analysis and the conclusion.
Fix the question before looking at the data, and check that the data can really be collected.
Check your understanding
Question 1: Which of these is an investigative question?
1. How many students are enrolled at this school?
2. What is the mean daily screen time of students at this school?
3. How many minutes did Lan spend on her phone yesterday?
Question 2: A researcher surveys 120 of the 2400 members of a gym. Which is correct?
1. N = 120 and n = 2400
2. N = 2400 and n = 120
3. N = 120 and the sample is the 2400 members
Question 3: Drawing a boxplot of the reaction times that were collected belongs to which stage?
1. Formulating a question
2. Analyzing the data
3. Collecting data