P-Value
Definition: A p-value is a measure of statistical significance which tells us the probability of an event occurring due to chance alone. The higher the p-value, the higher the probability that the event you are observing can be explained by chance. P-value results range from 0.0 to 1.0. In general, p-values of either 0.05 or 0.01 are used as a cutoff value, although this value is arbitrary. Results larger than the cutoff are considered likely to attribute the event to chance, while results smaller than the cutoff value are likely to have occurred because of a real explanation.
When analyzing data, you start with a null hypothesis. The null hypothesis acts like a straw man; it is assumed to be true in order to knock it down as false with a statistical test. When the data are analyzed, such tests determine the p-value, the probability of obtaining the study results by chance if the null hypothesis is true. The null hypothesis is rejected in favor of its alternative if the p-value is less than ‘alpha,’ the predetermined level of statistical significance (often 0.5 or 0.01).
"Non-significant" results, those with p-values greater than 'alpha,' do not imply that there is no association in the population. They only imply that the association observed in the sample is small compared with what could have occurred by chance alone.
Example: An investigator might find that men with hypertension were twice more likely to develop complications due to a smallpox vaccination than those with normal blood pressure, but with a p-value of 0.09. This means that even if hypertension and smallpox vaccine complications were not associated in the population, there is a 9% chance of finding such an association due to random error in the sample (chance). If the investigator had set the significance level at 0.05, he/she would have to conclude that the association in the sample was "not statistically significant." It might be tempting to change the level of statistical significance, but a better choice would be to report that "the results, although suggestive of an association, did not achieve statistical significance (p-value = 0.09).” This solution acknowledges that statistical significance is not an "all or none" situation. (Designing Clinical Research, Hulley and Cummings, 1988)
