What a one-sample t test asks

A one-sample t test evaluates whether a population mean differs from a prespecified null value when the population standard deviation is unknown and estimated by the sample SD. The statistic measures the observed mean difference in estimated standard-error units.

The null hypothesis is μ = μ₀. A two-sided alternative asks whether μ differs in either direction. Right- and left-sided alternatives ask a directional question that should be chosen before seeing the estimate.

Worked summary-data example

The default sample has mean 52, SD 10, and n = 25, compared with μ₀ = 50. The mean difference is 2. The standard error is 10/√25 = 2, so t = 2/2 = 1 with 24 degrees of freedom.

Under a two-sided t distribution, |t| at least 1 has p≈0.3273. At alpha 0.05, this does not cross the rejection threshold. The result is not proof that μ equals 50; it says the observed statistic is not unusual enough under the null to reject with this procedure.

The 95% critical t is about 2.0639. The interval is 52 ± 2.0639×2, or approximately 47.8722 through 56.1278. It includes 50, agreeing with the two-sided test. Cohen’s d = (52−50)/10 = 0.2.

Choose the alternative before analysis

Choosing one side because the observed mean points that way invalidates the nominal error rate. If direction was not fixed in advance, use the two-sided result. A directional p-value does not make the opposite direction impossible.

Student t versus normal z

The t distribution accounts for uncertainty from estimating σ with s. It has heavier tails than the standard normal, especially with small df. As n grows, t approaches normal and critical values converge.

If the population σ were genuinely known from an external source under a justified model, a z procedure could be appropriate. Merely having a sample SD does not make σ known. Most ordinary one-sample mean analyses therefore use t.

Assumptions and robustness

Observations should be independent or the analysis must model dependence. The outcome should be quantitative, and the sampling process should support inference to the target population. Strong selection bias is not repaired by a t distribution.

For small samples, severe skewness and outliers can distort the mean and t inference. With larger samples, the sampling mean often becomes approximately normal, but heavy tails and dependence can slow that protection. Inspect raw data whenever possible; summary inputs cannot reveal shape.

A one-sample t test of paired differences is valid when each pair produces one difference and those differences are independent across pairs. Enter the mean and SD of differences, not the two groups’ separate SDs.

Confidence intervals and test agreement

A two-sided alpha 0.05 test and a 95% two-sided confidence interval give matching decisions under the same t method: reject μ₀ exactly when the interval excludes it. A 90% interval matches alpha 0.10, and a 99% interval matches alpha 0.01.

A one-sided test corresponds to a one-sided confidence bound, not the two-sided interval displayed here. The calculator labels its interval as two-sided even when a directional p-value is selected. Keep those inferential objects distinct.

Confidence intervals show compatible mean values and estimation precision. A wide interval can include effects that are both practically important and negligible, revealing that “not significant” may mean inconclusive.

Effect size and practical importance

Cohen’s d divides the mean difference by the sample SD. It is signed: positive means x̄ exceeds μ₀, negative means it is below. Conventional magnitude labels are rough context, not universal decision thresholds.

Standardized and raw effects answer different questions. A two-unit change may be trivial on one outcome and consequential on another. Report the raw difference, units, interval, and subject-matter threshold along with d.

The sample d is biased upward in magnitude in very small samples. Hedges-type correction can be used when a bias-adjusted standardized effect is required. Statistical significance also depends on n, so a tiny effect can be significant in a large sample and a meaningful effect imprecise in a small one.

P-values and decisions

A p-value is the probability, assuming the null model and test assumptions, of a statistic at least as incompatible with the null in the selected tail definition. It is not the probability that H₀ is true, the chance the result occurred “by accident,” or the probability a replication will agree.

Alpha is a prespecified long-run false-positive threshold under the null, not a measure of effect importance. The dashboard uses reject/do-not-reject language. “Accept H₀” would overstate what a nonsignificant result establishes.

Multiple outcomes, repeated looks, and selective reporting inflate false-positive opportunity. Apply the planned multiplicity or sequential method rather than treating every unadjusted p below 0.05 as confirmatory.

High-stakes boundary: Medical, legal, regulatory, and safety decisions require the complete dataset, design-aware analysis, uncertainty, and professional review—not a summary calculator alone.

Power and the observed-effect trap

A priori power uses a meaningful alternative effect before data collection. “Observed power” computed by inserting the observed effect is largely a transformation of the p-value and adds little information after the test. Use the confidence interval to discuss what effects remain compatible with the data.

If planning a future sample, specify the smallest mean difference worth detecting, expected SD, alpha, desired power, and tail. The standardized planning effect is difference/SD. Inflate for unusable cases separately and investigate whether larger n is feasible and ethically justified.

A nonsignificant pilot should not automatically supply a zero effect for planning, nor should its noisy observed d be treated as truth. Use prior evidence and sensitivity scenarios.

Equivalence and noninferiority questions

Failure to reject μ=μ₀ does not establish equivalence. An equivalence design specifies a lower and upper margin of practically negligible differences and uses two one-sided tests or a corresponding confidence-interval rule. The interval must fit within the equivalence region.

Noninferiority uses a one-sided margin and requires careful direction, population, adherence, and bias considerations. Choosing a margin after observing results undermines the claim. A conventional one-sample t p-value against zero is not a substitute.

If the scientific goal is to show that any difference is small, define “small” numerically before data collection and power the equivalence procedure.

Outliers and alternative methods

The mean and SD are sensitive to extreme observations. A single value can shift the mean, inflate s, or do both. Verify data provenance and report sensitivity without deleting a valid observation solely to obtain significance.

A one-sample Wilcoxon signed-rank test addresses a distributional location question under symmetry assumptions and is not simply a median t test. A sign test uses only directions and makes fewer shape assumptions but can be less powerful. Bootstrap intervals can be useful when designed and interpreted carefully.

Transformations change the estimand. Testing log outcomes concerns a geometric-scale relationship after back-transformation, not the original arithmetic mean difference. Choose the method for the scientific quantity, not as a reaction to one p-value.

Data-quality and reporting checklist

1. Define the population, outcome, units, and sampling method.
2. State why μ₀ was selected independently of the observed data.
3. Report n, mean, sample SD, mean difference, t, df, exact p, and confidence interval.
4. Name the alternative and alpha, including any multiplicity adjustment.
5. Describe outlier, missing-data, and assumption checks.
6. Separate statistical evidence from practical or clinical thresholds.

Rounding should occur after calculation. A p-value close to alpha can move across the printed boundary if summaries are heavily rounded. Preserve sufficient digits in the mean and SD and report p with an inequality only when it is beyond reliable numerical resolution.

Frequently asked questions

References

NIST/SEMATECH e-Handbook — one-sample t test

NIST/SEMATECH e-Handbook — t distribution

American Statistical Association — statement on statistical significance and p-values