[PDF] Mathematical Statistics Statistics is about the mathematical

View - Download Mathematical Statistics

Previous PDF

Next PDF

Mathematical Statistics

Sara van de Geer

September 2010

2

Contents

1 Introduction 7

1.1 Some notation and model assumptions . . . . . . . . . . . . . . . 7

1.2 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.3 Comparison of estimators: risk functions . . . . . . . . . . . . . . 12

1.4 Comparison of estimators: sensitivity . . . . . . . . . . . . . . . . 12

1.5 Confidence intervals . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.5.1 Equivalence confidence sets and tests . . . . . . . . . . . . 13

1.6 Intermezzo: quantile functions . . . . . . . . . . . . . . . . . . . 14

1.7 How to construct tests and confidence sets . . . . . . . . . . . . . 14

1.8 An illustration: the two-sample problem . . . . . . . . . . . . . . 16

1.8.1 Assuming normality . . . . . . . . . . . . . . . . . . . . . 17

1.8.2 A nonparametric test . . . . . . . . . . . . . . . . . . . . 18

1.8.3 Comparison of Student"s test and Wilcoxon"s test . . . . . 20

1.9 How to construct estimators . . . . . . . . . . . . . . . . . . . . . 21

1.9.1 Plug-in estimators . . . . . . . . . . . . . . . . . . . . . . 21

1.9.2 The method of moments . . . . . . . . . . . . . . . . . . . 22

1.9.3 Likelihood methods . . . . . . . . . . . . . . . . . . . . . 23

2 Decision theory 29

2.1 Decisions and their risk . . . . . . . . . . . . . . . . . . . . . . . 29

2.2 Admissibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.3 Minimaxity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.4 Bayes decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.5 Intermezzo: conditional distributions . . . . . . . . . . . . . . . . 35

2.6 Bayes methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.7 Discussion of Bayesian approach (to be written) . . . . . . . . . . 39

2.8 Integrating parameters out (to be written) . . . . . . . . . . . . . 39

2.9 Intermezzo: some distribution theory . . . . . . . . . . . . . . . . 39

2.9.1 The multinomial distribution . . . . . . . . . . . . . . . . 39

2.9.2 The Poisson distribution . . . . . . . . . . . . . . . . . . . 41

2.9.3 The distribution of the maximum of two random variables 42

2.10 Sufficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.10.1 Rao-Blackwell . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.10.2 Factorization Theorem of Neyman . . . . . . . . . . . . . 45

2.10.3 Exponential families . . . . . . . . . . . . . . . . . . . . . 47

2.10.4 Canonical form of an exponential family . . . . . . . . . . 48

3

4CONTENTS

2.10.5 Minimal sufficiency . . . . . . . . . . . . . . . . . . . . . . 53

3 Unbiased estimators 55

3.1 What is an unbiased estimator? . . . . . . . . . . . . . . . . . . . 55

3.2 UMVU estimators . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3.2.1 Complete statistics . . . . . . . . . . . . . . . . . . . . . . 59

3.3 The Cramer-Rao lower bound . . . . . . . . . . . . . . . . . . . . 62

3.4 Higher-dimensional extensions . . . . . . . . . . . . . . . . . . . . 66

3.5 Uniformly most powerful tests . . . . . . . . . . . . . . . . . . . . 68

3.5.1 An example . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.5.2 UMP tests and exponential families . . . . . . . . . . . . 71

3.5.3 Unbiased tests . . . . . . . . . . . . . . . . . . . . . . . . 74

3.5.4 Conditional tests . . . . . . . . . . . . . . . . . . . . . . . 77

4 Equivariant statistics 81

4.1 Equivariance in the location model . . . . . . . . . . . . . . . . . 81

4.2 Equivariance in the location-scale model (to be written) . . . . . 86

5 Proving admissibility and minimaxity 87

5.1 Minimaxity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

5.2 Admissibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

5.3 Inadmissibility in higher-dimensional settings (to be written) . . . 95

6 Asymptotic theory 97

6.1 Types of convergence . . . . . . . . . . . . . . . . . . . . . . . . . 97

6.1.1 Stochastic order symbols . . . . . . . . . . . . . . . . . . . 99

6.1.2 Some implications of convergence . . . . . . . . . . . . . . 99

6.2 Consistency and asymptotic normality . . . . . . . . . . . . . . . 101

6.2.1 Asymptotic linearity . . . . . . . . . . . . . . . . . . . . . 101

6.2.2 Theδ-technique . . . . . . . . . . . . . . . . . . . . . . . 102

6.3 M-estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

6.3.1 Consistency of M-estimators . . . . . . . . . . . . . . . . . 106

6.3.2 Asymptotic normality of M-estimators . . . . . . . . . . . 109

6.4 Plug-in estimators . . . . . . . . . . . . . . . . . . . . . . . . . . 114

6.4.1 Consistency of plug-in estimators . . . . . . . . . . . . . . 117

6.4.2 Asymptotic normality of plug-in estimators . . . . . . . . 118

6.5 Asymptotic relative efficiency . . . . . . . . . . . . . . . . . . . . 121

6.6 Asymptotic Cramer Rao lower bound . . . . . . . . . . . . . . . 123

6.6.1 Le Cam"s 3

rdLemma . . . . . . . . . . . . . . . . . . . . . 126

6.7 Asymptotic confidence intervals and tests . . . . . . . . . . . . . 129

6.7.1 Maximum likelihood . . . . . . . . . . . . . . . . . . . . . 131

6.7.2 Likelihood ratio tests . . . . . . . . . . . . . . . . . . . . . 135

6.8 Complexity regularization(to be written). . . . . . . . . . . . . 139

7 Literature 141

CONTENTS5

These notes in English will closely followMathematische Statistik, by H.R. K¨unsch (2005), but are as yet incomplete.Mathematische Statistikcan be used as supplementary reading material in German. Mathematical rigor and clarity often bite each other. At some places, not all subtleties are fully presented. A snake will indicate this.

6CONTENTS

Chapter 1

Introduction

Statistics is about the mathematical modeling of observable phenomena, using stochastic models, and about analyzing data: estimating parameters of the model and testing hypotheses. In these notes, we study various estimation and testing procedures. We consider their theoretical properties and we investigate various notions of optimality.

1.1 Some notation and model assumptions

The data consist of measurements (observations)x1,...,xn, which are regarded as realizations of random variablesX1,...,Xn. In most of the notes, theXi are real-valued:Xi?R(fori= 1,...,n), although we will also consider some extensions to vector-valued observations. Example 1.1.1Fizeau and Foucault developed methods for estimating the speed of light (1849, 1850), which were later improved by Newcomb and Michel- son. The main idea is to pass light from a rapidly rotating mirror to a fixed mirror and back to the rotating mirror. An estimate of the velocity of light is obtained, taking into account the speed of the rotating mirror, the distance travelled, and the displacement of the light as it returns to the rotating mirror.Fig. 1 The data are Newcomb"s measurements of the passage time it took light to travel from his lab, to a mirror on the Washington Monument, and back to his lab. 7

8CHAPTER 1. INTRODUCTION

distance: 7.44373 km.

66 measurements on 3 consecutive days

first measurement: 0.000024828 seconds= 24828 nanoseconds The dataset has the deviations from 24800 nanoseconds.

The measurements on 3 different days:l

l l l l l l l l l l lquotesdbs_dbs7.pdfusesText_5

[PDF] Maths Suite géométrique terminale

[PDF] Maths super urgent avec grosse récompense (voir le devoir )

[PDF] maths sur les fonction

[PDF] Maths sur les fonctions

[PDF] Maths sur les probabilités exercices

[PDF] maths sur puissances

[PDF] Maths sur Thalès pour demain

[PDF] maths svp

[PDF] maths table carrée , nappe ronde

[PDF] Maths Tableau

[PDF] maths tableur troisième

[PDF] Maths tarif

[PDF] maths taux de variation

[PDF] maths terminale es fonction exponentielle

[PDF] Maths Terminale S