GloNeuro

Learn

Research design

Research design is the plan by which a study answers its question. The design determines which conclusions a study can support.

Research questions

A research question states what is to be found out, about whom, and what would count as an answer. Its type determines which designs can answer it.

Four kinds of research question. The drawings are illustrations, not data.
Describing
How common or how large is something? For example, how common is migraine among university students? The sample must represent the population of interest.
Associating
Do two things tend to occur together? For example, do students who sleep less score lower on a memory test?
Predicting
Can information available now indicate what will happen, such as who will have another seizure within a year? A good predictor need not be a cause.
Explaining causes
Would changing one thing change another? For example, does a night without sleep impair memory? Causal questions place the greatest demands on the design.

Study types

Designs differ most in who decides which people, animals or cells are exposed. In an observational study the researcher records exposures as they occur; in an experimental study the researcher assigns them, ideally by chance. In a natural experiment an outside event does the assigning.

  • Case reports and case series

    A case report describes a single patient in detail; a case series describes a small group with a feature in common. Scoville and Milner described severe loss of recent memory after surgical removal of the hippocampus on both sides of the brain (Scoville and Milner, 1957). Case reports record something new and raise questions, but without a comparison group they cannot show how common a finding is or what caused it.

  • Ecological studies

    The units are groups, such as regions or countries, and the study compares group averages or rates: for example, head-injury admissions in each region against the proportion of cyclists there who wear helmets. Ecological studies are quick and inexpensive, but a relationship between group averages need not hold for individuals (the ecological fallacy) (Robinson, 1950).

  • Cross-sectional studies

    A sample is measured once, so exposures and outcomes are recorded together: for example, physical activity and hippocampal volume at a single research visit. Such studies describe how common a condition is and generate hypotheses, but they usually cannot show which came first.

  • Case–control studies

    People with a condition (cases) are compared with similar people without it (controls), and the earlier exposures of both groups are established, as in studies of past smoking in people with and without Parkinson’s disease (Hernán et al., 2002). The design suits rare conditions. Controls are difficult to choose, cases may recall the past differently (recall bias), and the measure of association is the odds ratio.

  • Retrospective cohort studies

    A group defined by an exposure is followed forward in time using existing records, for example the hospital records of people who had a concussion ten years earlier. Follow-up is long and inexpensive, but the records were collected for other purposes and may lack important variables.

  • Prospective cohort studies

    People are recruited, their exposures are measured and they are followed to record who develops which outcomes. Doll and Hill followed British doctors whose smoking habits had been recorded (Doll and Hill, 1954). Because exposure is recorded before the outcome, the order of events is clear. Cohort studies are large, slow and expensive, and the exposed and unexposed may differ in other ways (confounding).

  • Natural experiments

    An event outside the researchers’ control, such as a new law or an arbitrary rule, exposes some people and not others in a way that is close to random (Craig et al., 2017). For example, a school-entry date allows children with a year more or less of schooling to be compared. The argument depends on showing that the groups are genuinely comparable.

  • Randomised controlled trials

    The researchers allocate treatment by chance, so the groups are expected to be alike in every other respect, measured or not (Schulz et al., 2010). In one trial, people with early Parkinson’s disease were randomly assigned to nicotine or placebo patches for a year (Oertel et al., 2023). The randomised trial is the strongest single design for testing whether a treatment works, but it can be expensive, impossible or unethical.

  • Crossover trials

    Each participant receives every treatment, in random order, often separated by a washout period, and so acts as his or her own control. Fewer participants are needed. The design suits stable, long-lasting conditions and fails when a treatment has lasting effects (carry-over).

  • Within-subject laboratory experiments

    Each participant performs every condition of a task, such as visual search with few or many distractors, and the order of conditions is varied across participants (counterbalancing). Mental processes are measured precisely in modest samples, but practice and fatigue can influence the results.

  • Animal experiments

    The researcher changes one element of the nervous system, such as a gene, a group of neurons or a drug, and measures the effect, with randomisation and blinding. Each study needs prior approval from an institutional animal ethics committee: in India, an Institutional Animal Ethics Committee working under the CCSEA, the statutory committee constituted under the Prevention of Cruelty to Animals Act 1960 (CCSEA, n.d.), with equivalent frameworks in the European Union, under Directive 2010/63/EU (European Parliament and Council, 2010), the United States, under the Animal Welfare Act and the PHS Policy (OLAW, 2015), and the United Kingdom (Animals (Scientific Procedures) Act 1986). The ARRIVE guidelines, developed by the NC3Rs in the United Kingdom and endorsed by journals internationally, set out what should be reported (Percie du Sert et al., 2020). Rats with hippocampal damage, for example, learned poorly to find a hidden platform (Morris et al., 1982). Such experiments give strong evidence about mechanisms, but results may not translate to people.

Study designs arranged by the researcher’s control over exposure. Positions are approximate and are not a ranking. Selecting a design shows its description.

Key concepts

Population and sample

The population is everyone the question concerns; the sample is the part actually studied. A random sample represents the population on average. A convenience sample, made up of whoever is easiest to reach, often does not.

Sample

Simulated population of 240 people, in which older people (filled circles) live closer to the research centre (top left). A random sample of 24 reflects the population; a convenience sample of the 24 people nearest the centre does not.

Variables

A variable is any characteristic that can differ between the units of a study. Its role depends on the question.

Show

The roles of variables in a study of sleep and memory. Arrows show assumed causal effects.
Independent variable
The variable that the researcher changes or compares, such as hours of sleep. In observational studies it is called the exposure.
Dependent variable
The outcome that is measured, such as the number of words recalled.
Confounding variable
A third variable that influences both exposure and outcome, such as age. Ignored, it can create an association or conceal one.
Mediating variable
A step on the path from cause to effect, such as memory consolidation between sleep and recall.
Moderating variable
A variable that changes the size or direction of an effect, such as a gene variant that makes sleep loss more harmful.

Bias

Bias is systematic error. Unlike chance error, it does not diminish as a study grows. Selection bias arises when the people studied differ systematically from the population, for example when volunteers are unusually healthy. Measurement bias arises when measurements are wrong in a consistent way, for example when an assessor who knows the treatment scores favourably; blinding prevents this. Recall bias arises when people with an outcome report past exposures differently from people without it. Survivorship bias arises when only those who have survived a process are observed.

Validity

Internal validity is the correctness of a result within the study: whether a difference is due to the treatment rather than to confounding, bias or chance. External validity is the extent to which the result applies to other people, settings or species. Construct validity is the extent to which a measure captures what it is intended to measure.

Replication and pre-registration

Replication is the repetition of a study to establish whether its result holds. When 100 psychology studies were repeated, 36% of the replications gave statistically significant results, compared with 97% of the originals, and the effects were on average about half as large (Open Science Collaboration, 2015). Pre-registration records the hypotheses and analysis plan before the data are examined, which separates planned tests from exploratory findings (Nosek et al., 2018).

Statistical power

Power is the probability that a study detects an effect of a given size if the effect is real. It increases with the size of the effect and the number of units. The average power of neuroscience studies is very low, so many significant results overstate the size of the effect (Button et al., 2013).

Power for a comparison of two groups (two-sided test, 5% significance level). The grey curve shows the results expected with no effect, the coloured curve those expected with the chosen effect; the shaded area beyond the threshold is the power. Values use the normal approximation.

The unit of analysis

A statistical test must count the independent units that were randomised or sampled. With three treated and three control mice, the experiment has three units per group, however many neurons are recorded. Treating each neuron as independent, known as pseudoreplication, makes chance differences between animals appear to be treatment effects (Lazic, 2010).

Simulated experiment with no true treatment effect: 3 mice per group, 8 neurons per mouse. Welch’s t-test is applied to the 24 neurons per group as if they were independent, and to the mean of each mouse. The count shows how often each analysis gives p < 0.05.

The hierarchy of evidence

The hierarchy of evidence is usually drawn as a pyramid, from case reports at the base through observational studies and randomised trials to systematic reviews at the apex. For questions about treatments, higher designs protect better against confounding and bias.

The traditional hierarchy (pyramid) and a more cautious view (bars): study quality varies within every level, so the ranges overlap. The bars are an illustration, not measurements.

The pyramid ranks designs, not individual studies: well-designed observational studies have given average results similar to those of trials on the same clinical questions (Concato et al., 2000). It also concerns treatments only; questions about prevalence or risk factors are often best answered by observational designs. The GRADE approach therefore rates the certainty of the whole body of evidence on a question, downgrading trials with a risk of bias and upgrading observational evidence of very large effects (Guyatt et al., 2008).

Association, correlation and causation

Two variables are associated when knowing one provides information about the other. A correlation is an association between two ordered variables that rise and fall together, or in opposite directions. Causation is a claim about what would happen if one variable were changed. Every correlation is an association, but not every association is a correlation, and neither demonstrates causation on its own.

Relations between the three concepts. Correlations form a subset of associations. Causal effects usually produce associations, but an association alone cannot show which effects are causal.

References

  1. Animals (Scientific Procedures) Act 1986, c. 14. United Kingdom. https://www.legislation.gov.uk/ukpga/1986/14/contents
  2. Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365–376. https://doi.org/10.1038/nrn3475
  3. Committee for Control and Supervision of Experiments on Animals (CCSEA), Department of Animal Husbandry and Dairying, Government of India (n.d.). About CCSEA. Accessed October 2026. https://ccsea.gov.in/
  4. Concato, J., Shah, N., & Horwitz, R. I. (2000). Randomized, controlled trials, observational studies, and the hierarchy of research designs. New England Journal of Medicine, 342(25), 1887–1892. https://doi.org/10.1056/NEJM200006223422507
  5. Craig, P., Katikireddi, S. V., Leyland, A., & Popham, F. (2017). Natural experiments: an overview of methods, approaches, and contributions to public health intervention research. Annual Review of Public Health, 38, 39–56. https://doi.org/10.1146/annurev-publhealth-031816-044327
  6. Doll, R., & Hill, A. B. (1954). The mortality of doctors in relation to their smoking habits. BMJ, 1(4877), 1451–1455. https://doi.org/10.1136/bmj.1.4877.1451
  7. European Parliament and Council of the European Union (2010). Directive 2010/63/EU of 22 September 2010 on the protection of animals used for scientific purposes. Official Journal of the European Union, L 276, 33–79. https://eur-lex.europa.eu/eli/dir/2010/63/oj
  8. Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008). GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 336(7650), 924–926. https://doi.org/10.1136/bmj.39489.470347.AD
  9. Hernán, M. A., Takkouche, B., Caamaño-Isorna, F., & Gestal-Otero, J. J. (2002). A meta-analysis of coffee drinking, cigarette smoking, and the risk of Parkinson’s disease. Annals of Neurology, 52(3), 276–284. https://doi.org/10.1002/ana.10277
  10. Lazic, S. E. (2010). The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis? BMC Neuroscience, 11, 5. https://doi.org/10.1186/1471-2202-11-5
  11. Morris, R. G., Garrud, P., Rawlins, J. N., & O’Keefe, J. (1982). Place navigation impaired in rats with hippocampal lesions. Nature, 297(5868), 681–683. https://doi.org/10.1038/297681a0
  12. Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114
  13. Oertel, W. H., Müller, H.-H., Unger, M. M., et al. (2023). Transdermal nicotine treatment and progression of early Parkinson’s disease. NEJM Evidence, 2(9), EVIDoa2200311. https://doi.org/10.1056/EVIDoa2200311
  14. Office of Laboratory Animal Welfare, National Institutes of Health (2015). Public Health Service Policy on Humane Care and Use of Laboratory Animals (revised 2015). Bethesda, MD: NIH. https://olaw.nih.gov/policies-laws/phs-policy.htm
  15. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716
  16. Percie du Sert, N., Hurst, V., Ahluwalia, A., et al. (2020). The ARRIVE guidelines 2.0: updated guidelines for reporting animal research. PLoS Biology, 18(7), e3000410. https://doi.org/10.1371/journal.pbio.3000410
  17. Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15(3), 351. https://doi.org/10.2307/2087176
  18. Schulz, K. F., Altman, D. G., & Moher, D., for the CONSORT Group (2010). CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ, 340, c332. https://doi.org/10.1136/bmj.c332
  19. Scoville, W. B., & Milner, B. (1957). Loss of recent memory after bilateral hippocampal lesions. Journal of Neurology, Neurosurgery, and Psychiatry, 20(1), 11–21. https://doi.org/10.1136/jnnp.20.1.11

Last updated