P-Value

A p-value is a statistic from hypothesis testing. It gives the probability of obtaining data at least as extreme as what you observed, assuming the null hypothesis (usually "no effect" or "no difference") is true. A small p-value means the data would be surprising under that assumption.

Why it matters

P-values are used to judge whether an observed difference, such as a higher conversion rate in an A/B test, is likely to be more than random noise. A threshold such as 0.05 is a widely used convention, not a universal rule. Statistical significance does not mean the effect is large or important, and running many tests inflates false positives unless you adjust for multiple comparisons.

Example

Variant B converts 5.4% vs 5.0% for A. Test result: p = 0.03.
Meaning: if there were truly no difference, a gap this large
would appear about 3% of the time by chance.

Common misconceptions

• "p = 0.03 means a 97% chance the effect is real." It does not.
• "p > 0.05 proves there is no effect." It may just mean too little data.
• Stopping a test early the moment p dips below 0.05 distorts results.

Related

• Standard deviation
• A/B testing
• Correlation vs causation
• Sampling

This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.