"statistically significant" means and ONLY means that the difference between the experimental and control group are probably not be explained by chance. "statistically significant" is one of those phrases that is both deceptive and mysterious.

It is deceptive because a study could be statistically significant but not substantively or meaningfully significant. For example, you might see an experiment on a diet aid where the claim is made, "After using it for 6 months the dieters lost a statistically significant amount of weight." You might think evidence means the diet aid is really good. Not necessarily; you might look into it and it turns out the 320 pound dieters lost an average of 8 pounds and the control group stayed the same. This does not seem significant in the everyday meaning of the term but the important qualifier is statistically. To say the result is statistically significant means that chance does not explain this 8 pound difference. One way to think about it is if a bunch of 320 pound people went on this diet we could predict with some confidence they would lose, on the average, 8 pounds after six months. If it had come out statistically non-significant they would probably not lose an average of 8 pounds.   

It is mysterious because many people have a difficult time understanding the probability theory that justifies the judgment that a difference between groups exists  or, alternatively, is a chance difference. A finding can LOOK like there is a difference but that appearance may be an illusion. Statistics are used to put an exact value on how probable it is the results are real =  that is would really exist in the population, versus deceptive = appear to exist but are simply accidental chance. Specifically, if the result is reported as p < .05 that means the probability that chance explains the results is less than 5 out of 100 and psychologists are then willing to say the finding exists.

Return to Home Page   

(With the uncertainty inherent in using statistics, as we must do in many social science studies, you can see why induction can't be used to "prove" a hypothesis. But it can be used to confirm or disconfirm a hypothesis which is very helpful in scientific progress.) 
Return to There are a Variety of Ways We Know