What statistical test should I use? (2023)

What statistical test should I use? It is often said that the design of a study is more important than the analysis. A badly designed study can never be retrieved, whereas a poorly analyzed study can usually be re-analyzed.

Updated: March 2021

In terms of selecting a statistical test, the most important question is "what is the main study hypothesis?". For example, nQuery has a vast list of statistical procedures to calculate sample size, in fact over 1000 sample size scenarios are covered. However, it is important that these are paired with a correctly designed trial.

The correct statistical test to use not only depends on your study design, but also the characteristics of your data. This will be a result of your research questions/hypotheses you are trying to answer.

The graph and table below can be used as a guide for which statistical test or descriptive statistic to use in your research. In order to use it, you must be able to identify all the variables in the data set and tellwhat kind of variablesthey are.

(Video) Statistical Tests: Choosing which statistical test to use

What statistical test should I use? (1)

As you know and can see there's a wide range of statistical tests to choose from.

The decision of which statistical test to use depends on:

  1. The research design
  2. The distribution of the data
  3. The type of variable
When choosing the correct test, ask yourself the following questions: What kind of data have you collected? What is your goal?
GoalMeasurement
(from Gaussian Population)
Rank, Score, or Measurement
(from Non- Gaussian Population)
Binomial
(Two Possible Outcomes)
Survival Time
Describe one groupMean, SDMedian, interquartile rangeProportionKaplan Meier survival curve
Compare one group to a hypothetical valueOne-samplet-testWilcoxon testChi-square
or Binomial test **
Compare two unpairedgroupsUnpairedttestMann-Whitney testFisher's test
(chi-square for large samples)
Log-rank test or Mantel-Haenszel*
Compare two paired groupsPairedttestWilcoxon testMcNemar's testConditional proportional hazards regression*
Compare three or more unmatched groupsOne-way ANOVAKruskal-Wallis testChi-square testCox proportional hazard regression**
Compare three or more matched groupsRepeated-measures ANOVAFriedman testCochrane Q**Conditional proportional hazards regression**
Quantify association between two variablesPearson correlationSpearman correlationContingency coefficients**
Predict value from another measured variableSimple linear regression
or Nonlinear regression
Nonparametric regression**Simple logistic regression*Cox proportional hazard regression*
Predict value from several measured or binomial variablesMultiple linear regression* or Multiple nonlinear regression**Multiple logistic regression*Cox proportional hazard

RECOMMENDED: Answers to the most common
sample size and power analysis questions

(Video) What statistical tests to use and when
1. Sample Size and Power Analysis
2. Hypothesis Testing
3. Statistical Power
4. Effect Size
5. Confidence Intervals
6. Bayesian Statistics
7. Adaptive Clinical Trials
8. Further Sample Size Topics

Read Now

(Video) Choosing a Statistical Test for Your IB Biology IA

What type of statistical test to use?

Below is an extract from theHandbook of Biological Statistics by Prof John H. McDonald.
This can be used as a further guide
to decide what statistical test to use in your research.

TestNominal VariablesMeasurement VariablesRanked VariablesPurposeNotesExample
Exact test for goodness-of-fit1test fit of observed frequencies to expected frequenciesuse for small sample sizes (less than 1000)count the number of red, pink and white flowers in a genetic cross, test fit to expected 1:2:1 ratio, total sample <1000
Chi-square test of goodness-of-fit1test fit of observed frequencies to expected frequenciesuse for large sample sizes (greater than 1000)count the number of red, pink and white flowers in a genetic cross, test fit to expected 1:2:1 ratio, total sample >1000
G–test of goodness-of-fit1test fit of observed frequencies to expected frequenciesused for large sample sizes (greater than 1000)count the number of red, pink and white flowers in a genetic cross, test fit to expected 1:2:1 ratio, total sample >1000
RepeatedG–tests of goodness-of-fit2test fit of observed frequencies to expected frequencies in multiple experiments-count the number of red, pink and white flowers in a genetic cross, test fit to expected 1:2:1 ratio, do multiple crosses
TestNominal VariablesMeasurement VariablesRanked VariablesPurposeNotesExample
Fisher's exact test2test hypothesis that proportions are the same in different groupsuse for small sample sizes (less than 1000)count the number of live and dead patients after treatment with drug or placebo, test the hypothesis that the proportion of live and dead is the same in the two treatments, total sample <1000
Chi-square test of independence2test hypothesis that proportions are the same in different groupsuse for large sample sizes (greater than 1000)count the number of live and dead patients after treatment with drug or placebo, test the hypothesis that the proportion of live and dead is the same in the two treatments, total sample >1000
G–test of independence2test hypothesis that proportions are the same in different groupslarge sample sizes (greater than 1000)count the number of live and dead patients after treatment with drug or placebo, test the hypothesis that the proportion of live and dead is the same in the two treatments, total sample >1000
Cochran-Mantel-Haenszel test3test hypothesis that proportions are the same in repeated pairings of two groupsalternate hypothesis is a consistent direction of differencecount the number of live and dead patients after treatment with drug or placebo, test the hypothesis that the proportion of live and dead is the same in the two treatments, repeat this experiment at different hospitals
TestNominal VariablesMeasurement VariablesRanked VariablesPurposeNotesExample
Arithmetic mean1description of central tendency of data--
Median1description of central tendency of datamore useful than mean for very skewed datamedian height of trees in forest, if most trees are short seedlings and the mean would be skewed by a few very tall trees
Range1description of dispersion of dataused more in everyday life than in scientific statistics-
Variance1description of dispersion of dataforms the basis of many statistical tests; in squared units, so not very understandable-
Standard deviation1description of dispersion of datain same units as original data, so more understandable than variance-
Standard error of the mean1description of accuracy of an estimate of a mean--
Confidence interval1description of accuracy of an estimate of a mean--
TestNominal VariablesMeasurement VariablesRanked VariablesPurposeNotesExample
One-samplet–test1test the hypothesis that the mean value of the measurement variable equals a theoretical expectation-blindfold people, ask them to hold arm at 45° angle, see if mean angle is equal to 45°
Two-sample t–test11test the hypothesis that the mean values of the measurement variable are the same in two groupsjust another name for one-way anova when there are only two groupscompare mean heavy metal content in mussels from Nova Scotia and New Jersey
One-way anova11test the hypothesis that the mean values of the measurement variable are the same in different groups-compare mean heavy metal content in mussels from Nova Scotia, Maine, Massachusetts, Connecticut, New York and New Jersey
Tukey-Kramer test11after a significant one-way anova, test for significant differences between all pairs of groups-compare mean heavy metal content in mussels from Nova Scotia vs. Maine, Nova Scotia vs. Massachusetts, Maine vs. Massachusetts, etc.
Bartlett's test11test the hypothesis that the standard deviation of a measurement variable is the same in different groupsusually used to see whether data fit one of the assumptions of an anova

compare standard deviation of heavy metal content in mussels from Nova Scotia, Maine, Massachusetts, Connecticut, New York and New Jersey

TestNominal VariablesMeasurement VariablesRanked VariablesPurposeNotesExample
Nested anova2+1test hypothesis that the mean values of the measurement variable are the same in different groups, when each group is divided into subgroupssubgroups must be arbitrary (model II)compare mean heavy metal content in mussels from Nova Scotia, Maine, Massachusetts, Connecticut, New York and New Jersey; several mussels from each location, with several metal measurements from each mussel
Two-way anova21test the hypothesis that different groups, classified two ways, have the same means of the measurement variable-compare cholesterol levels in blood of male vegetarians, female vegetarians, male carnivores, and female carnivores
Pairedt–test21test the hypothesis that the means of the continuous variable are the same in paired datajust another name for two-way anova when one nominal variable represents pairs of observationscompare the cholesterol level in blood of people before vs. after switching to a vegetarian diet
Wilcoxon signed-rank test21test the hypothesis that the means of the measurement variable are the same in paired dataused when the differences of pairs are severely non-normalcompare the cholesterol level in blood of people before vs. after switching to a vegetarian diet, when differences are non-normal
TestNominal VariablesMeasurement VariablesRanked VariablesPurposeNotesExample
Linear regression2see whether variation in an independent variable causes some of the variation in a dependent variable; estimate the value of one unmeasured variable corresponding to a measured variable-measure chirping speed in crickets at different temperatures, test whether variation in temperature causes variation in chirping speed; or use the estimated relationship to estimate temperature from chirping speed when no thermometer is available
Correlation2see whether two variables covary-measure salt intake and fat intake in different people's diets, to see if people who eat a lot of fat also eat a lot of salt
Polynomial regression2test the hypothesis that an equation with X2, X3, etc. fits the Y variable significantly better than a linear regression--
Analysis of covariance (ancova)12test the hypothesis that different groups have the same regression linesfirst test the homogeneity of slopes; if they are not significantly different, test the homogeneity of the Y-interceptsmeasure chirping speed vs. temperature in four species of crickets, see if there is significant variation among the species in the slope or Y-intercept of the relationships
TestNominal VariablesMeasurement VariablesRanked VariablesPurposeNotesExample
Multiple regression3+fit an equation relating several X variables to a single Y variable-measure air temperature, humidity, body mass, leg length, see how they relate to chirping speed in crickets
Simple logistic regression11fit an equation relating an independent measurement variable to the probability of a value of a dependent nominal variable-give different doses of a drug (the measurement variable), record who lives or dies in the next year (the nominal variable)
Multiple logistic regression12+fit an equation relating more than one independent measurement variable to the probability of a value of a dependent nominal variable-record height, weight, blood pressure, age of multiple people, see who lives or dies in the next year
TestNominal VariablesMeasurement VariablesRanked VariablesPurposeNotesExample
Sign test21test randomness of direction of difference in paired data-compare the cholesterol level in blood of people before vs. after switching to a vegetarian diet, only record whether it is higher or lower after the switch
Kruskal–Wallis test11test the hypothesis that rankings are the same in different groupsoften used as a non-parametric alternative to one-way anova40 ears of corn (8 from each of 5 varieties) are ranked for tastiness, and the mean rank is compared among varieties
Spearman rank correlation2see whether the ranks of two variables covaryoften used as a non-parametric alternative to regression or correlation40 ears of corn are ranked for tastiness and prettiness, see whether prettier corn is also tastier


References
Chapter 37+45 of the second edition of Intuitive Biostatistics Harvey Motulsky.
McDonald, J.H. 2014. Handbook of Biological Statistics (3rd ed.). Sparky House Publishing, Baltimore, Maryland.
Campbell MJ, Machin D. In: Medical Statistics: A Common-sense Approach , 2nd edn. Chichester: Wiley, 1993:2.
Stats Flow Chart

(Video) T-test, ANOVA and Chi Squared test made easy.

Videos

1. How To Know Which Statistical Test To Use For Hypothesis Testing
(Amour Learning)
2. RAN planning and optimisation in the cloud-native era (on-demand)
(TelecomTV)
3. How to choose an appropriate statistical test
(TileStats)
4. Choosing a Statistical Test
(Erich Goldstein)
5. Deep Work Rules for focused success in a distracted world | full audiobook
(THUMASAS)
6. What statistical test should I use? Part 1
(Freeman Chabala PhD.)
Top Articles
Latest Posts
Article information

Author: Saturnina Altenwerth DVM

Last Updated: 02/20/2023

Views: 5829

Rating: 4.3 / 5 (64 voted)

Reviews: 87% of readers found this page helpful

Author information

Name: Saturnina Altenwerth DVM

Birthday: 1992-08-21

Address: Apt. 237 662 Haag Mills, East Verenaport, MO 57071-5493

Phone: +331850833384

Job: District Real-Estate Architect

Hobby: Skateboarding, Taxidermy, Air sports, Painting, Knife making, Letterboxing, Inline skating

Introduction: My name is Saturnina Altenwerth DVM, I am a witty, perfect, combative, beautiful, determined, fancy, determined person who loves writing and wants to share my knowledge and understanding with you.