Welcome
Welcome to Statistical Inference! This book covers the mathematical theory and computational practice of classical statistical inference, the formal discipline for drawing uncertainty-aware and quantified conclusions about a population from a sample of it.
Statistical inference lies at the foundation of data analysis. It supplies the framework for estimating unknown parameters, quantifying the uncertainty inherent in working with random samples, and testing scientific hypotheses against empirical evidence in a formal way. Mastering its foundations equips the reader with knowledge that is broadly applicable across scientific disciplines and data analysis tasks.
The book builds its subject from the ground up, aiming to keep a consistent balance between mathematical rigor and practical implementation. It is designed to be self-contained, including the necessary probabilistic tools for statistical inference. Each chapter develops the theoretical foundations, including some proofs of key results, and illustrates them with numerous figures, detailed examples, and self-contained R code.
A collection of more than 300 exercises and examples, with at least 25 exercises per chapter, is included to help the reader test their understanding and apply the methods. These exercises vary in difficulty and type: from straightforward applications of the methods to more challenging problems. The reader should close each chapter with a clear understanding of why and how the methods work, together with the practical ability to deploy them.
Scope of the book
The central focus of the book is classical parametric inference for one and two populations, with special emphasis on normal populations as the natural starting point for exact, distribution-based results. A recurring and unifying theme is large-sample theory: the central limit theorem, the asymptotic properties of the maximum likelihood estimator, asymptotic confidence intervals, and asymptotic tests form a coherent progression that extends the reach of the methods to non-normal and more general settings.
Beyond the more “basic syllabus”, the book also treats topics of particular theoretical and practical relevance: transformations of random vectors as the mathematical backbone of statistical inference; the Cramér–Rao lower bound as the fundamental limit on estimator precision; the Rao–Blackwell theorem as a constructive path to variance reduction; the multiparameter case in maximum likelihood estimation with its asymptotic theory and estimation of the Fisher information matrix; bootstrap confidence intervals as a modern, computationally-driven alternative to analytical intervals; the Neyman–Pearson lemma as the theoretical foundation for optimal hypothesis testing; and the likelihood ratio test as a powerful framework for test construction in arbitrary parametric models. Topics such as Bayesian inference, nonparametric methods, and regression analysis lie outside the scope of this text, but the foundations developed here provide essential preparation for them.
The book begins with Chapter 1, Preliminaries, which reviews the probabilistic tools needed throughout: probability fundamentals, random variables and their types, expectation and variance, the moment-generating function, random vectors together with their marginal and conditional distributions, and transformations of random vectors. These tools, especially the latter, form the mathematical backbone of statistical inference and all subsequent chapters.
Chapter 2, Introduction, sets up the inferential framework. Starting from the concept of a simple random sample and a statistic, it derives the key sampling distributions associated with normal populations (chi-square, Student’s \(t\), and Snedecor’s \(\mathcal{F}\)). The chapter closes with the central limit theorem and its proof and applications, bridging the exact results of normal-population theory with the asymptotic methods developed in later chapters.
Chapter 3, Point estimation, gives a treatment of the properties that characterize a good estimator: bias, invariance, consistency, sufficiency and minimal sufficiency, efficiency, and robustness. These criteria provide a principled framework for evaluating and comparing estimators. Key concepts such as the likelihood, Fisher information, and Cramér–Rao lower bound are introduced in this chapter. The chapter also presents the Rao–Blackwell theorem, which shows how sufficient statistics can be leveraged to improve unbiased estimators.
Chapter 4, Estimation methods, covers the two most prominent strategies for constructing estimators: the method of moments, which matches population and sample moments to obtain simple estimators, and maximum likelihood estimation, which derives estimators by maximizing the likelihood function and enjoys a set of desirable properties. The asymptotic normality of the maximum likelihood estimator is presented for the uniparameter and multiparameter cases, including the situation where the Fisher information matrix is estimated.
Chapter 5, Confidence intervals, presents the pivotal-quantity method as the general approach for confidence interval construction. It applies it in two main ways. First, through sampling distributions in normal populations, to obtain exact confidence intervals for means and variances. Second, through asymptotic normality, to obtain asymptotic confidence intervals for the mean, based on the central limit theorem, and for general parameters, based on large-sample maximum likelihood theory. A different route is taken with bootstrap-based confidence intervals, which provide general interval construction without the need for analytical formulae.
Finally, Chapter 6, Hypothesis tests, develops the formal framework for hypothesis testing: null and alternative hypotheses, test function, decision rule, error types, \(p\)-value, and power. A similar agenda to that of confidence intervals is presented: classical tests for means and variances in normal populations are obtained through exact sampling distributions, while asymptotic distributions are used to obtain asymptotically-valid tests. The chapter concludes with the two main theoretical pillars in hypothesis testing: the Neyman–Pearson lemma, which characterizes the most powerful test for simple hypotheses, and the likelihood ratio test, which extends optimal testing to composite hypotheses across a broad class of parametric models.
Software & code
The book contains a substantial amount of code, with snippets that are intended to be self-contained within the chapter in which they appear. This helps illustrate how the methods and theory translate to practice. The software employed throughout this book is the statistical language R. Only very basic knowledge of R is assumed.
Several packages that are not included within R by default are used throughout the book. All of the packages can be installed with the following commands:
# Installation of required packages
packages <- c("boot", "knitr", "latex2exp", "MASS", "mvtnorm", "numDeriv",
"viridisLite")
install.packages(packages)The book makes explicit mention of the package to which a function belongs by using the operator ::, except when the use of the functions of a package is very repetitive and that package is loaded. You can load all the packages by running:
# Load packages
lapply(packages, library, character.only = TRUE)The snippets of R code of the book are conveniently collected in the following scripts:
- Chapter 1:
01-preliminaries.R. - Chapter 2:
02-intro-inference.R. - Chapter 3:
03-point-estimation.R. - Chapter 4:
04-estimation-methods.R. - Chapter 5:
05-confidence-intervals.R. - Chapter 6:
06-hypothesis-tests.R.
To download them, use your browser’s Save link as… option.
Citation
You may use the following \(\text{B}{\scriptstyle\text{IB}}\mkern-2mu \text{T}\mkern-3mu\lower.5ex\hbox{E}\mkern-2mu\text{X}\) entry when citing this book:
@book{StatisticalInference2026,
title = {Statistical Inference},
author = {Garc\'ia-Portugu\'es, E. and Molina, I.},
year = {2026},
note = {Version 3.0.0. ISBN 978-84-09-29680-4},
url = {https://egarpor.github.io/SI-UC3M/}
}You may also want to use the following template:
García-Portugués, E. and Molina, I. (2026). Statistical Inference. Version 3.0.0. ISBN 978-84-09-29680-4. Available at https://egarpor.github.io/SI-UC3M/.
A previous version of the book at the now-discontinued bookdown.org hosting service was known as A First Course on Statistical Inference.
License
All the material in this book is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License (CC BY-NC-ND 4.0). You may not use this material except in compliance with the aforementioned license. The human-readable summary of the license states that:
-
You are free to:
- Share – Copy and redistribute the material in any medium or format.
-
Under the following terms:
- Attribution – You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- NonCommercial – You may not use the material for commercial purposes.
- NoDerivatives – If you remix, transform, or build upon the material, you may not distribute the modified material.
Contributions
Contributions, reporting of typos, and feedback on the book are very welcome. Send an email to edgarcia@est-econ.uc3m.es and I will gladly add your name to the list of contributors.
Credits
More general and complete monographs on statistical inference include the books by Casella and Berger (2002), Silvey (1975), Lehmann and Casella (1998), Lehmann and Romano (2005), and Shao (2003).
This book was made possible thanks to the excellent open-source software Xie (2016), Xie (2020), Allaire et al. (2020), Xie and Allaire (2020), and R Core Team (2020). In addition, some layout improvements build on the outstanding work of Úcar (2018).
Last but not least, the book has benefited from contributions from the following people (in alphabetical order):
- Arconada Manteca, Lúa (fixed one typo).
- Askey, Jonathan (fixed one typo).
- Blanco Blanco, Germán (fixed one typo).
- Díaz Pérez, Álvaro (fixed one typo).
- Diego Velarde, Alba (fixed one typo).
- Fernández de Marcos Giménez de los Galanes, Alberto (fixed three typos).
- Fernández Martín, Manuel (fixed one typo).
- Fernández Morín, María (fixed two typos).
- Gianelli Nan, Sofía (fixed three typos).
- Goñi Artieda, Javier (fixed one typo).
- Hackett, Nicole (fixed one typo).
- Levenfeld Sabau, Gabriela Hortensia (fixed two typos).
- Lodato Olano, Paolo Salvatore (fixed two typos).
- Montero de Espinosa Reina, Juan Antonio (fixed four typos).
- Morilla Ortega, Francisco Jesús (fixed two typos).
- Novillo Correas, Álvaro (fixed one typo).
- Pérez Gómez, Eloy (fixed one typo).
- Pérez Peralta, Arturo (fixed one typo).
- Pérez Piedra, Pilar (fixed one typo).
- Qian, Jingye (fixed seven typos).
- Rab, Ismael Ahmad (fixed three typos).
- Sánchez Polo, Jorge (fixed one typo).
- Serrano Ortega, Diego (fixed four typos).
- Smith, Kendal Raymond William (performed a thorough proofreading fixing eighty typos and issues).
- Velázquez Jamart, Rolando (fixed four typos).
- Vidal Pereira, Tomás (fixed one typo).