• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site

Improving your statistical inferences

2019/2020
Academic Year
ENG
Instruction in English
3
ECTS credits
Course type:
Elective course
When:
3 year, 1 module

Course Syllabus

Abstract

This course aims to help you to draw better statistical inferences from empirical research. First, we will discuss how to correctly interpret p-values, effect sizes, confidence intervals, Bayes Factors, and likelihood ratios, and how these statistics answer different questions you might be interested in. Then, you will learn how to design experiments where the false positive rate is controlled, and how to decide upon the sample size for your study, for example in order to achieve high statistical power. Subsequently, you will learn how to interpret evidence in the scientific literature given widespread publication bias, for example by learning about p-curve analysis. Finally, we will talk about how to do philosophy of science, theory construction, and cumulative science, including how to perform replication studies, why and how to pre-register your experiment, and how to share your results following Open Science principles. In practical, hands on assignments, you will learn how to simulate t-tests to learn which p-values you can expect, calculate likelihood ratios and get an introduction the binomial Bayesian statistics, and learn about the positive predictive value which expresses the probability published research findings are true. We will experience the problems with optional stopping and learn how to prevent these problems by using sequential analyses. You will calculate effect sizes, see how confidence intervals work through simulations, and practice doing a-priori power analyses. Finally, you will learn how to examine whether the null hypothesis is true using equivalence testing and Bayesian statistics, and how to pre-register a study, and share your data on the Open Science Framework.
Learning Objectives

Learning Objectives

  • To learn how to improve the way you interpret the scientific literature, design experiments, and share empirical results
  • To learn how to implement better research practices in your own research
Expected Learning Outcomes

Expected Learning Outcomes

  • Interpretation of p-values
  • Likelihood Function skills
  • Bayesian Statistics skills
  • Statistical power estimation
  • Calculation of effect sizes
  • Interpretation of confidence intervals
  • The use of p-curve analysis
  • Understanding of the construction of theory
  • Hypothesis formulation skills
  • Understanding of open science, pre-registration, and the importance of data sharing
Course Contents

Course Contents

  • Introduction + Frequentist Statistics
    Interpret p-values correctly. Examine the distribution of p-values as a function of the statistical power of the test.
  • Multiple Comparisons, Statistical Power, Pre-Registration.
    The difference between type 1 and type 2 errors. Find out which results are most likely in your research. Recognize the effects of optional shutdown.
  • Likelihoods & Bayesian Statistics.
    The differences between likelihood and Bayesian approaches. The strengths and weaknesses of likelihood and Bayesian approaches to inferences. The benefits of Bayesian thinking when drawing statistical inferences.
  • Effect Sizes.
    Comparison of standard and non-standard effect values. Calculate effect sizes from summary data or test statistics. Interpretation of effect sizes.
  • Confidence Intervals, Sample Size Justification, P-Curve analysis.
    Correct interpretation of confidence intervals. The difference between frequent confidence intervals and Bayesian confidence intervals. The use of p-curve analysis to assess evidence of value in study sets.
  • Philosophy of Science & Theory.
    Different points of view on the philosophy of science. Various ways to facilitate the construction of a theory. Is the null hypothesis a true prediction?
  • Open Science.
    Pre-registration of your experiment. Sharing data and analysis scenarios with your research report.
Assessment Elements

Assessment Elements

  • non-blocking Practice exam
  • non-blocking Final exam
Interim Assessment

Interim Assessment

  • Interim assessment (1 module)
    0.5 * Final exam + 0.5 * Practice exam
Bibliography

Bibliography

Recommended Core Bibliography

  • Agresti, A., & Finlay, B. (2014). Statistical Methods for the Social Sciences: Pearson New International Edition (Vol. Pearson new international ed., 4. ed). Harlow England: Pearson. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=nlebk&AN=1418314
  • Lee, P. M. (2012). Bayesian Statistics : An Introduction (Vol. 4th ed). Chichester, West Sussex: Wiley. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=463079
  • Statistics and Causality : Methods for Applied Empirical Research, edited by Wolfgang Wiedermann, and Eye, Alexander von, John Wiley & Sons, Incorporated, 2016. ProQuest Ebook Central, https://ebookcentral.proquest.com/lib/hselibrary-ebooks/detail.action?docID=4530803.

Recommended Additional Bibliography

  • Bolstad, W. M. (2017). Introduction to Bayesian Statistics (Vol. Third edition). Hoboken, N.J.: Wiley. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1342637
  • Donovan, T. M., & Mickey, R. M. (2019). Bayesian Statistics for Beginners : A Step-by-step Approach. Oxford: OUP Oxford. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=2139683
  • Improving the conduct and reporting of statistical analysis in psychology. (2016). Psychometrika, 81(1), 33–38. https://doi.org/10.1007/s11336-015-9444-2
  • Reinhart, A. (2015). Statistics Done Wrong : The Woefully Complete Guide. San Francisco: No Starch Press. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=984483
  • Seemon Thomas. (2014). Basic Statistics. [N.p.]: Alpha Science Internation Limited. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1663598