Skip to main content

RESEARCH STATISTICS GUIDE

Which statistical test should I use?

The right test depends on three things: what you want to find out, what type of outcome variable you have, and whether your groups contain different people or the same people measured more than once.Explore research statistics & data analysis →

FORMULA OR CORE IDEA

Question type + outcome type + number of groups + paired or independent → test

WORKED EXAMPLE

A student compares exam scores (numeric, roughly normal) between two separate classes. That is two independent groups with a numeric outcome, so the test is an independent-samples t-test. If the scores were heavily skewed, the Mann-Whitney U test would be used instead.

QUICK REFERENCE

Statistical test decision table

Parametric test first; the non-parametric alternative is used when the outcome is ordinal or not normally distributed.
Research situationParametric testNon-parametric / categorical alternative
Two independent groupsIndependent t-test (Welch)Mann-Whitney U
Same people measured twicePaired t-testWilcoxon signed-rank
Three or more independent groupsOne-way ANOVAKruskal-Wallis H
Same people, three or more timesRepeated-measures ANOVAFriedman test
Relationship between two variablesPearson correlationSpearman's rank correlation
Two categorical variables—Chi-square test of independence (Fisher's exact if small)
Yes/no measured twice—McNemar's test
One sample vs a known valueOne-sample t-testOne-sample Wilcoxon signed-rank

STEP BY STEP

01

Write your research question as a comparison or a relationship

"Do online and in-person students score differently?" is a comparison of groups. "Is study time related to exam score?" is a relationship between two variables. "Is gender associated with preferred learning mode?" is an association between two categorical variables.

02

Identify the type of your outcome variable

Numeric and roughly normal (test scores, heights, summed scale totals); ordinal or skewed (single Likert items, ranks, income); or categorical (yes/no, chosen option). This decides between parametric tests, non-parametric tests, and chi-square tests.

03

Count the groups or time points

One group compared with a reference value, two groups, or three or more. Two groups lead to t-tests or Mann-Whitney; three or more lead to ANOVA or Kruskal-Wallis.

04

Decide whether the measurements are paired

If the same participants are measured twice (pre-test and post-test), the data are paired and need a paired t-test, Wilcoxon signed-rank test, or McNemar's test. Different people in each group are independent.

05

Check the assumptions before you run it

Look at a histogram or run a Shapiro-Wilk test for normality, check for outliers, and confirm expected counts of at least 5 for chi-square. If an assumption fails, switch to the non-parametric alternative.

06

Report the test, statistic, degrees of freedom, p value, and effect size

For example: t(48) = 2.31, p = .025, d = 0.65. APA style also expects means and standard deviations (or medians for non-parametric tests).

COMMON QUESTIONS

Frequently asked questions

How do I decide which statistical test to use?

Answer four questions in order: (1) Are you comparing groups, looking for a relationship, or testing an association between categories? (2) Is the outcome numeric, ordinal, or categorical? (3) How many groups or time points are there? (4) Are the measurements independent or paired? The decision table above maps each combination to a test, and the Statistical Test Chooser walks through the same questions interactively.

Which test should I use for a pre-test and post-test?

If the same people take both tests and the score differences are roughly normal, use a paired t-test. If the differences are skewed or the outcome is ordinal, use the Wilcoxon signed-rank test. For a yes/no outcome measured before and after, use McNemar's test. With three or more time points, use repeated-measures ANOVA or the Friedman test.

Should I use correlation or regression?

Use correlation to describe how strongly two variables move together (Pearson r or Spearman rho). Use regression when you want to predict an outcome from one or more predictors and report how much it changes per unit of each predictor (B, β and R²).

What is the difference between parametric and non-parametric tests?

Parametric tests (t-test, ANOVA, Pearson) compare means and assume roughly normal data with similar spread. Non-parametric tests (Mann-Whitney, Wilcoxon, Kruskal-Wallis, Spearman) compare ranks, so they suit ordinal data, skewed data and small samples, at the cost of slightly less power when the data really are normal.

Should I use an ANOVA or a t-test?

Use a t-test for exactly two groups and a one-way ANOVA for three or more. Running several t-tests instead of one ANOVA raises the chance of a false positive: with three groups, three t-tests at α = .05 give about a 14% familywise error rate.

Should I use ANOVA or chi-square?

Use ANOVA when the outcome is a number, such as a score or time. Use chi-square when the outcome is a category, such as pass/fail or which option was chosen.

Which statistical test is used for Likert scale data?

For single Likert items, use non-parametric tests: Mann-Whitney U for two groups or Kruskal-Wallis for three or more. For scale scores made by summing or averaging several items, t-tests and ANOVA are widely accepted if the scores are roughly normal.

What are the 5 basic statistical analyses?

Most theses use descriptive statistics (mean, SD, frequencies), comparison of groups (t-test or ANOVA), relationships (correlation), association between categories (chi-square), and prediction (regression).

What if my data are not normally distributed?

Use the non-parametric alternative in the table above. With large samples (roughly 30+ per group), t-tests and ANOVA are fairly robust to mild non-normality, but strong skew or outliers still call for a non-parametric test.

FROM RESEARCH QUESTION TO TEST

Thesis research questions mapped to the right test, plus the mistakes panels catch

Typical thesis research questions and the test for each

  • What is the profile of the respondents (sex, age, year level)? → frequency and percentage.
  • What is the level of the respondents' perception or satisfaction? → weighted mean with a Likert interpretation scale (add the SD).
  • Is there a significant difference in perception by sex (2 groups)? → independent t-test, or Mann-Whitney U for ordinal or skewed scores.
  • Is there a significant difference by year level or age group (3+ groups)? → one-way ANOVA, or Kruskal-Wallis H.
  • Is there a significant relationship between two scale scores (e.g. study habits and grades)? → Pearson r, or Spearman rho.
  • Is a category (e.g. strand) associated with another category (e.g. preferred course)? → chi-square test of independence (Fisher's exact for small 2×2 tables).
  • Did scores change from pre-test to post-test? → paired t-test, or Wilcoxon signed-rank.
  • Which factors predict the outcome? → multiple linear regression (logistic regression for a yes/no outcome).

Five mistakes thesis panels often catch

  • Running several t-tests instead of one ANOVA for 3+ groups, which inflates the chance of a false positive (three t-tests at α = .05 give about a 14% chance of at least one false result).
  • Using chi-square on means or on Likert averages. Chi-square needs counts in categories.
  • Treating pre-test and post-test scores of the same students as two independent groups. The design is paired.
  • Running Pearson r on a single Likert item. Use Spearman rho for single ordinal items; Pearson is defensible for composite scale scores.
  • Reporting only p. Panels increasingly expect an effect size (d, η², r or Cramér's V) and the descriptive statistics behind it.

How many respondents each test needs

These are the minimum totals for 80% power at α = .05 (two-tailed) for a medium effect, the same numbers G*Power gives. Smaller expected effects need far larger samples; run your own numbers in the power analysis calculator.

  • Independent t-test (d = 0.5): 64 per group, 128 in total.
  • One-way ANOVA with 3 groups (f = 0.25): about 53 per group, 159 in total.
  • Pearson correlation (r = .30): 84 participants.
  • Chi-square test, 2 × 2 table (w = 0.30): 88 participants.

Where to find each test in SPSS

  • Independent and paired t-tests: Analyze → Compare Means → Independent-Samples / Paired-Samples T Test.
  • One-way ANOVA: Analyze → Compare Means → One-Way ANOVA (tick Post Hoc → Tukey).
  • Pearson and Spearman: Analyze → Correlate → Bivariate.
  • Chi-square: Analyze → Descriptive Statistics → Crosstabs → Statistics → Chi-square.
  • Mann-Whitney U and Kruskal-Wallis: Analyze → Nonparametric Tests → Legacy Dialogs → 2 / K Independent Samples.