Bench Tools Free browser-based calculators for the lab and for bioinformatics

Tools›Statistics

How to Choose a Statistical Test: By Data Type, Number of Groups, and Paired vs. Independent Design

Answer three questions to identify the right test: comparing values or proportions, how many groups, paired or independent. Each row links to the corresponding tool.

Choosing a test requires answering just three questions: are you comparing numeric values or proportions? how many groups? and is the data paired? The table below is organized along these three dimensions; each row links directly to the corresponding tool.

Comparing “Numeric Values” — mean or distribution location

Groups Design Data distribution Test Tool
1 group vs. known value — Approximately normal One-sample t-test t-test
2 groups Independent Approximately normal Welch t-test (default) t-test
2 groups Independent Skewed / outliers / ordinal Mann-Whitney U Nonparametric
2 groups Paired Approximately normal Paired t-test t-test
2 groups Paired Skewed / ordinal Wilcoxon signed-rank Nonparametric
3+ groups Independent Normal, equal variances One-way ANOVA ANOVA
3+ groups Independent Normal, unequal variances Welch ANOVA ANOVA
3+ groups Independent Skewed / ordinal Kruskal-Wallis ANOVA
3+ groups Repeated measures — Repeated-measures ANOVA / mixed-effects model Not available on this site

Comparing “Proportions” — categorical data and contingency tables

Table Design Condition Test Tool
2×2 Independent All expected counts ≥ 5 Pearson chi-square Chi-square & Fisher
2×2 Independent Any expected count < 5 Fisher’s exact test Chi-square & Fisher
R×C Independent All expected counts ≥ 5 Pearson chi-square Chi-square & Fisher
2×2 Paired — McNemar’s test Chi-square & Fisher

Measuring “Association” — between two variables

Variable types Relationship Test Tool
Two continuous Linear Pearson r Correlation
Two continuous Monotone but nonlinear Spearman ρ / Kendall τ Correlation
One continuous vs. one continuous Predicting a value Linear regression Standard curve

After Testing

Situation Method Tool
Many tests performed at once BH / BY / Bonferroni correction Multiple testing correction
ANOVA is significant; need to identify which pairs differ Pairwise comparisons + multiple testing correction ANOVA
Need to determine sample size before the experiment Sample size estimation Sample size
Is the overlap between gene lists significant? Hypergeometric / Fisher Enrichment significance

The Three Questions That Determine Everything

1. Numeric values or proportions? Body weight, expression level, and OD values are numeric — use the t-test family. Counts of “responders/non-responders” or “positive/negative” are proportions — use the chi-square family. Treating proportions as numeric values (e.g., running a t-test on 0/1 data) is the most common mismatch.

2. How many groups? Two groups: use a t-test. Three or more: use ANOVA. Running t-tests on all pairs of three groups is wrong — the probability of at least one false positive across three comparisons is ~14%, and ~26% across six, far above the 5% you intended.

3. Paired or independent? Two measurements on the same subject (before and after), or matched case-control pairs, are paired data. A paired test operates on the differences, removing between-subject variability, and is usually more powerful than an independent test. Analyzing paired data as independent loses power; doing the reverse inflates the apparent significance.

Assessing Normality

Strictly speaking, you should inspect the data distribution — but two practical rules cover most situations:

With fewer than 5–6 observations, normality is untestable. At that sample size, normality tests (e.g., Shapiro-Wilk) have almost no power, and running one tells you nothing useful. When you are uncertain about the distribution, a rank-based test is safer — but check the minimum achievable p-value for your sample size first: for a paired signed-rank test with n=5, the minimum two-tailed p is 0.0625, regardless of how large the effect is.

When data are clearly skewed, contain outliers, or are inherently ordinal (e.g., Likert ratings), go straight to a rank-based test. There is no need to run a normality test first and then decide — that approach raises the overall false-positive rate.

What This Site Does Not Cover

In the interest of honesty: repeated-measures ANOVA and mixed-effects models (multiple measurements per subject, or designs with random effects) are not available here. These models require specifying a covariance structure, which makes them unsuitable for a fill-in-the-numbers web tool. Use R (lme4/nlme) or SPSS instead.

Survival analysis (Kaplan-Meier, Cox regression) is also outside the scope of this site.

FAQ

Why not run a normality test first and then decide which method to use?

Because that raises the overall false-positive rate: you use the same data both to select the method and to run the test, so the selection itself is data-dependent. Moreover, normality tests have almost no power at small sample sizes — they are least useful precisely when you need them most. In practice, decide in advance based on data type and field conventions: use rank-based tests for obviously skewed or ordinal data, parametric tests otherwise, and always report effect sizes.

What exactly is wrong with running t-tests on all pairs of three groups?

Each test's α = 0.05 is a per-test false-positive rate. With three groups and three pairwise comparisons, the probability of at least one false positive is ~14%; with six groups and fifteen comparisons, it is ~54%. ANOVA answers "is there any difference among these groups?" in a single test, keeping the overall false-positive rate at α. When pairwise comparisons are genuinely needed, run them after ANOVA and apply a multiple-testing correction to the p-values.

What happens if paired data are analyzed as independent?

You lose power. The value of a paired design is that it removes between-subject variability — two measurements on the same person are not affected by that person's baseline level. Treating them as two independent groups puts between-subject variability into the error term, masking the true effect. The reverse error — forcing independent data into a paired framework — inflates apparent significance.

Why doesn't this site include repeated-measures ANOVA?

It requires choosing a covariance structure (whether the sphericity assumption holds, and if not, which correction to apply), and different choices give different results. Wrapping that into a "fill in a few numbers and get an answer" web tool is likely to mislead users. The same applies to mixed-effects models. Use R (lme4/nlme) or SPSS for these analyses.

What else should I check after choosing a test?

Two things. First, multiple testing: if you ran many tests in one study, correct the p-values. Second, effect size and confidence intervals: a p-value only tells you whether the difference looks like random variation; with a large enough sample, a trivially small difference can reach statistical significance. Report both.

Related tools

Multiple Testing Correction Calculator: BH (FDR), BY, and Bonferroni

Paste a column of p-values and get BH q-values, BY, and Bonferroni corrections side by side in one step.

qPCR 2^−ΔΔCt Relative Quantification Online Calculator

Paste your Ct table and get ΔCt, ΔΔCt, fold-change, and within-group dispersion calculated automatically.

Gene List Intersection, Union & Difference — Online Venn Set Calculator

Paste 2–3 gene/ID lists and instantly get intersections, unions, and unique elements, with deduplication and case normalization.
No ads, no tracking, no sign-up — and every formula here is checked against a known answer. Keeping it that way takes ongoing work. If it saved you time, buy me a coffee.
Buy me a coffee