Table of Contents
Preface vii
Part 1 Critical review and outline of the Bayesian alternative 1
1 Uncertainty in physics and the usual methods of handling it 3
1.1 Uncertainty in physics 3
1.2 True value, error and uncertainty 5
1.3 Sources of measurement uncertainty 6
1.4 Usual handling of measurement uncertainties 7
1.5 Probability of observables versus probability of 'true values' 9
1.6 Probability of the causes 11
1.7 Unsuitability of frequentistic confidence intervals 11
1.8 Misunderstandings caused by the standard paradigm of hypothesis tests 15
1.9 Statistical significance versus probability of hypotheses 19
2 A probabilistic theory of measurement uncertainty 25
2.1 Where to restart from? 25
2.2 Concepts of probability 27
2.3 Subjective probability 29
2.4 Learning from observations: the 'problem of induction' 32
2.5 Beyond Popper's falsification scheme 34
2.6 From the probability of the effects to the probability of the causes 34
2.7 Bayes' theorem for uncertain quantities 36
2.8 Afraid of 'prejudices'? Logical necessity versus frequent practical irrelevance of priors 37
2.9 Recovering standard methods and short-cuts to Bayesian reasoning 39
2.10 Evaluation of measurement uncertainty: general scheme 41
2.10.1 Direct measurement in the absence of systematic errors 41
2.10.2 Indirect measurements 42
2.10.3 Systematic errors 43
2.10.4 Approximate methods 46
Part 2 A Bayesian primer 49
3 Subjective probability and Bayes1 theorem 51
3.1 What is probability? 51
3.2 Subjective definition of probability 52
3.3 Rules of probability 55
3.4 Subjective probability and 'objective' description of the physical world 58
3.5 Conditional probability and Bayes' theorem 60
3.5.1 Dependence of the probability on the state of information 60
3.5.2 Conditional probability 61
3.5.3 Bayes' theorem 63
3.5.4 'Conventional' use of Bayes' theorem 66
3.6 Bayesian statistics: learning by experience 68
3.7 Hypothesis 'test' (discrete case) 71
3.7.1 Variations over a problem to Newton 72
3.8 Falsificationism and Bayesian statistics 76
3.9 Probability versus decision 76
3.10 Probability of hypotheses versus probability of observations 77
3.11 Choice of the initial probabilities (discrete case) 78
3.11.1 General criteria 78
3.11.2 Insufficient reason and Maximum Entropy 81
3.12 Solution to some problems 82
3.12.1 AIDS test 82
3.12.2 Gold/silver ring problem 83
3.12.3 Regular or double-head coin? 84
3.12.4 Which random generator is responsible for the observed number? 84
3.13 Some further examples showing the crucial role of background knowledge 85
4 Probability distributions (a concise reminder) 89
4.1 Discrete variables 89
4.2 Continuous variables: probability and probability density function 92
4.3 Distribution of several random variables 98
4.4 Propagation of uncertainty 104
4.5 Central limit theorem 108
4.5.1 Terms and role 108
4.5.2 Distribution of a sample average 111
4.5.3 Normal approximation of the binomial and of the Poisson distribution 111
4.5.4 Normal distribution of measurement errors 112
4.5.5 Caution 112
4.6 Laws of large numbers 113
5 Bayesian inference of continuous quantities 115
5.1 Measurement error and measurement uncertainty 115
5.1.1 General form of Bayesian inference 116
5.2 Bayesian inference and maximum likelihood 118
5.3 The dog, the hunter and the biased Bayesian estimators 119
5.4 Choice of the initial probability density function 120
5.4.1 Difference with respect to the discrete case 120
5.4.2 Bertrand paradox and angels' sex 121
6 Gaussian likelihood 123
6.1 Normally distributed observables 123
6.2 Final distribution, prevision and credibility intervals of the true value 124
6.3 Combination of several measurements - Role of priors 125
6.3.1 Update of estimates in terms of Kalman filter 126
6.4 Conjugate priors 126
6.5 Improper priors - never take models literally! 127
6.6 Predictive distribution 127
6.7 Measurements close to the edge of the physical region 128
6.8 Uncertainty of the instrument scale offset 131
6.9 Correction for known systematic errors 133
6.10 Measuring two quantities with the same instrument having an uncertainty of the scale offset 133
6.11 Indirect calibration 136
6.12 The Gauss derivation of the Gaussian 137
7 Counting experiments 141
7.1 Binomially distributed observables 141
7.1.1 Observing 0% or 100% 145
7.1.2 Combination of independent measurements 146
7.1.3 Conjugate prior and many data limit 146
7.2 The Bayes problem 148
7.3 Predicting relative frequencies - Terms and interpretation of Bernoulli's theorem 148
7.4 Poisson distributed observables 152
7.4.1 Observation of zero counts 154
7.5 Conjugate prior of the Poisson likelihood 155
7.6 Predicting future counts 155
7.7 A deeper look to the Poissonian case 156
7.7.1 Dependence on priors - practical examples 156
7.7.2 Combination of results from similar experiments 158
7.7.3 Combination of results: general case 160
7.7.4 Including systematic effects 162
7.7.5 Counting measurements in the presence of background 165
8 Bypassing Bayes' theorem for routine applications 169
8.1 Maximum likelihood and least squares as particular cases of Bayesian inference 169
8.2 Linear fit 172
8.3 Linear fit with errors on both axes 175
8.4 More complex cases 176
8.5 Systematic errors and integrated likelihood1 177
8.6 Linearization of the effects of influence quantities and approximate formulae 178
8.7 BIPM and ISO recommendations 181
8.8 Evaluation of type B uncertainties 183
8.9 Examples of type B uncertainties 184
8.10 Comments on the use of type B uncertainties 186
8.11 Caveat concerning the blind use of approximate methods 189
8.12 Propagation of uncertainty 191
8.13 Co variance matrix of experimental results - more details 192
8.13.1 Building the covariance matrix of experimental data 192
8.13.1.1 Onset uncertainty 193
8.13.1.2 Normalization uncertainty 195
8.13.1.3 General case 196
8.14 Use and misuse of the covariance matrix to fit correlated data 197
8.14.1 Best estimate of the true value from two correlated values 197
8.14.2 Offset uncertainty 198
8.14.3 Normalization uncertainty 198
8.14.4 Peelle's Pertinent Puzzle 202
9 Bayesian unfolding 203
9.1 Problem and typical solutions 203
9.2 Bayes' theorem stated in terms of causes and effects 204
9.3 Unfolding an experimental distribution 205
Part 3 Further comments, examples and applications 209
10 Miscellanea on general issues in probability and inference 211
10.1 Unifying role of subjective approach 211
10.2 Prequentists and combinatorial evaluation of probability 213
10.3 Interpretation of conditional probability 215
10.4 Are the beliefs in contradiction to the perceived objectivity of physics? 216
10.5 Prequentists and Bayesian 'sects' 220
10.5.1 Bayesian versus frequentistic methods 221
10.5.2 Subjective or objective Bayesian theory? 222
10.5.3 Bayes' theorem is not everything 226
10.6 Biased Bayesian estimators and Monte Carlo checks of Bayesian procedures 226
10.7 Frequentistic coverage 229
10.7.1 Orthodox teacher versus sharp student - a dialogue by George Gabor 232
10.8 Why do frequentistic hypothesis tests 'often work'? 233
10.9 Comparing 'complex' hypotheses - automatic Ockham' Razor 239
10.10 Bayesian networks 241
10.10.1 Networks of beliefs - conceptual and practical applications 241
10.10.2 The gold/silver ring problem in terms of Bayesian networks 242
11 Combination of experimental results: a closer look 247
11.1 Use and misuse of the standard combination rule 247
11.2 'Apparently incompatible' experimental results 249
11.3 Sceptical combination of experimental results 252
11.3.1 Application to ε′/ ε 259
11.3.2 Posterior evaluation of σi 262
12 Asymmetric uncertainties and nonlinear propagation 267
12.1 Usual combination of 'statistic and systematic errors' 267
12.2 Sources of asymmetric uncertainties in standard statistical procedures 269
12.2.1 Asymmetric χ2 and 'Δχ2 = 1 rule' 269
12.2.2 Systematic effects 272
12.2.2.1 Asymmetric beliefs on systematic effects 273
12.2.2.2 Nonlinear propagation of uncertainties 273
12.3 General solution of the problem 273
12.4 Approximate solution 275
12.4.1 Linear expansion around E(X) 276
12.4.2 Small deviations from linearity 278
12.5 Numerical examples 280
12.6 The non-monotonic case 282
13 Which priors for frontier physics? 285
13.1 Frontier physics measurements at the limit to the detector sensitivity 285
13.2 Desiderata for an optimal report of search results 286
13.3 Master example: Inferring the intensity of a Poisson process in the presence of background 287
13.4 Modelling the inferential process 288
13.5 Choice of priors 288
13.5.1 Uniform prior 289
13.5.2 Jeffreys' prior 290
13.5.3 Role of priors 292
13.5.4 Priors reflecting the positive attitude of researchers 292
13.6 Prior-free presentation of the experimental evidence 295
13.7 Some examples of II function based on real data 298
13.8 Sensitivity bound versus probabilistic bound 299
13.9 Open versus closed likelihood 302
Part 4 Conclusion 305
14 Conclusions and bibliography 307
14.1 About subjective probability and Bayesian inference 307
14.2 Conservative or realistic uncertainty evaluation? 308
14.3 Assessment of uncertainty is not a mathematical game 310
14.4 Bibliographic note 310
Bibliography 313
Index 325
