Read an Excerpt
CHAPTER 1
ELEMENTARY THEORY OF PROBABILITY
We define as elementary theory of probability that part of the theory in which we have to deal with probabilities of only a finite number of events. The theorems which we derive here can be applied also to the problems connected with an infinite number of random events. However, when the latter are studied, essentially new principles are used. Therefore the only axiom of the mathematical theory of probability which deals particularly with the case of an infinite number of random events is not introduced until the beginning of Chapter II (Axiom VI).
The theory of probability, as a mathematical discipline, can and should be developed from axioms in exactly the same way as Geometry and Algebra. This means that after we have defined the elements to be studied and their basic relations, and have stated the axioms by which these relations are to be governed, all further exposition must be based exclusively on these axioms, independent of the usual concrete meaning of these elements and their relations.
In accordance with the above, in § 1 the concept of a field of probabilities is defined as a system of sets which satisfies certain conditions. What the elements of this set represent is of no importance in the purely mathematical development of the theory of probability (cf. the introduction of basic geometric concepts in the Foundations of Geometry by Hilbert, or the definitions of groups, rings and fields in abstract algebra).
Every axiomatic (abstract) theory admits, as is well known, of an unlimited number of concrete interpretations besides those from which it was derived. Thus we find applications in fields of science which have no relation to the concepts of random event and of probability in the precise meaning of these words.
The postulational basis of the theory of probability can be established by different methods in respect to the selection of axioms as well as in the selection of basic concepts and relations. However, if our aim is to achieve the utmost simplicity both in the system of axioms and in the further development of the theory, then the postulational concepts of a random event and its probability seem the most suitable. There are other postulational systems of the theory of probability, particularly those in which the concept of probability is not treated as one of the basic concepts, but is itself expressed by means of other concepts. However, in that case, the aim is different, namely, to tie up as closely as possible the mathematical theory with the empirical development of the theory of probability.
§ 1. Axioms
Let E be a collection of elements [xi], η, ζ, ..., which we shall call elementary events, and F a set of subsets of E; the elements of the set F will be called random events.
I. F is a field of sets.
II. F contains the set E.
III. To each set A in F is assigned a non-negative real number P(A). This number P(A) is called the probability of the event A.
IV. P(E) equals 1.
V. If A and B have no element in common, then
P(A + B)=P(A)+P(B)
A system of sets, F, together with a definite assignment of numbers P(A), satisfying Axioms I-V, is called a field of probability.
Our system of Axioms I-V is consistent. This is proved by the following example. Let E consist of the single element [xi] and let F consist of E and the null set 0. P(E) is then set equal to 1 and P(0) equals 0.
Our system of axioms is not, however, complete, for in various problems in the theory of probability different fields of probability have to be examined.
The Construction of Fields of Probability. The simplest fields of probability are constructed as follows. We take an arbitrary finite set E = {[xi]1, [xi]2, ..., [xi]k} and an arbitrary set {p1, p2, ..., pk} of non-negative numbers with the sum p1 + p2 + ... + pk = 1. F is taken as the set of all subsets in E, and we put
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]
such cases, p1, p2, ..., pk are called the probabilities of the elementary events [xi]1, [xi]2, ..., [xi]k or simply elementary probabilities. In this way are derived all possible finite fields of probability in which F consists of the set of all subsets of E. (The field of probability is called finite if the set E is finite.) For further examples see Chap. II, § 3.
§ 2. The Relation to Experimental Data
We apply the theory of probability to the actual world of experiments in the following manner:
1) There is assumed a complex of conditions, G, which allows of any number of repetitions.
2) We study a definite set of events which could take place as a result of the establishment of the conditions G. In individual cases where the conditions are realized, the events occur, generally, in different ways. Let E be the set of all possible variants [xi]1, [xi]2, ... of the outcome of the given events. Some of these variants might in general not occur. We include in set E all the variants which we regard a priori as possible.
3) If the variant of the events which has actually occurred upon realization of conditions G belongs to the set A (defined in any way), then we say that the event A has taken place.
Example: Let the complex G of conditions be the tossing of a coin two times. The set of events mentioned in Paragraph 2) consists of the fact that at each toss either a head or tail may come up. From this it follows that only four different variants (elementary events) are possible, namely: HH, HT, TH, TT. If the "event A" connotes the occurrence of a repetition, then it will consist of a happening of either of the first or fourth of the four elementary events. In this manner, every event may be regarded as a set of elementary events.
4) Under certain conditions, which we shall not discuss here, we may assume that to an event A which may or may not occur under conditions G, is assigned a real number P(A) which has the following characteristics:
(a) One can be practically certain that if the complex of conditions G is repeated a large number of times, n, then if m be the number of occurrences of event A, the ratio m/n will differ very slightly from P(A).
(b) If P(A) is very small, one can be practically certain that when conditions G are realized only once, the event A would not occur at all.
The Empirical Deduction of the Axioms. In general, one may assume that the system F of the observed events A, B, C, ... to which are assigned definite probabilities, form a field containing as an element the set E (Axioms I, II, and the first part of III, postulating the existence of probabilities). It is clear that 0 ≤ m/n ≤ 1 so that the second part of Axiom III is quite natural. For the event E, m is always equal to n, so that it is natural to postulate P(E) = 1 (Axiom IV). If, finally, A and B are nonintersecting (incompatible), then m = m1 + m2 where m, m1, m2 are respectively the number of experiments in which the events A + B, A, and B occur. From this it follows that
m/n m1/n + m2/n.
It therefore seems appropriate to postulate that P(A + B) = P(A) + P(B) (Axiom V).
Remark 1. If two separate statements are each practically reliable, then we may say that simultaneously they are both reliable, although the degree of reliability is somewhat lowered in the process. If, however, the number of such statements is very large, then from the practical reliability of each, one cannot deduce anything about the simultaneous correctness of all of them. Therefore from the principle stated in (a) it does not follow that in a very large number of series of n tests each, in each the ratio m/n will differ only slightly from P(A).
Remark 2. To an impossible event (an empty set) corresponds, in accordance with our axioms, the probability P(0) = 05, but the converse is not true: P(A) =0 does not imply the impossibility of A. When P(A) = 0, from principle (b) all we can assert is that when the conditions G are realized but once, event A is practically impossible. It does not at all assert, however, that in a sufficiently long series of tests the event A will not occur. On the other hand, one can deduce from the principle (a) merely that when P(A) = 0 and n is very large, the ratio m/n will be very small (it might, for example, be equal to 1/n).
§ 3. Notes on Terminology
We have defined the objects of our future study, random events, as sets. However, in the theory of probability many set-theoretic concepts are designated by other terms. We shall give here a brief list of such concepts.
Theory of Sets
1. A and B do not intersect, i.e., AB = 0.
2. AB ... N = 0.
3. AB ... N = X.
4. A + B + ... + N = X.
5. The complementary set [bar.A].
6. A = 0.
7. A = E.
8. The system U of the sets A1, A2, ..., An forms a decomposition of the set E if A1 + A2 + ... + An = E.
(This assumes that the sets Ai do not intersect, in pairs.)
9. B is a subset of A: B [subset] A.
Random Events
1. Events A and B are incompatible.
2. Events A, B, ..., N are incompatible.
3. Event X is defined as the simultaneous occurrence of events A, B, ..., N.
4. Event X is defined as the occurrence of at least one of the events A, B, ..., N.
5. The opposite event [bar.A] consisting of the non-occurence of event A.
6. Event A is impossible.
7. Event A must occur.
8. Experiment U consists of determining which of the events Al, A2, ..., An occurs. We therefore call A1, A2, ..., An the possible results of experiment U.
9. From the occurrence of event B follows the inevitable occurrence of A.
§ 4. Immediate Corollaries of the Axioms; Conditional Probabilities; Theorem of Bayes
From A + [bar.A] = E and the Axioms IV and V it follows that
P(A)+P([bar.A]) = 1 (1)
P([bar.A]) = 1 - P(A) . (2)
Since [bar.E] = 0, then, in particular,
P(0) = 0. (3)
If A, B, ..., N are incompatible, then from Axiom V follows the formula (the Addition Theorem)
P(A + B + ... + N) = P(A) + P(B) + ... + P(N) . (4)
If P(A) > 0, then the quotient
PA (B) = P(AB)/P(A) (5)
is defined to be the conditional probability of the event B under the condition A.
From (5) it follows immediately that
P(AB)=P(A) PA(B). (6)
And by induction we obtain the general formula (the Multiplication Theorem)
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (7)
The following theorems follow easily:
PA(B) ≥ 0, (8)
PA(E) = 1, (9)
PA(B + C) = PA(B) + PA(C). (10)
Comparing formulae (8) — (10) with axioms III — V, we find that the system F of sets together with the set function PA(B) (provided A is a fixed set), form a field of probability and therefore, all the above general theorems concerning P (B) hold true for the conditional probability PA(B) (provided the event A is fixed).
It is also easy to see that
PA(A) = 1. (11)
From (6) and the analogous formula
P(AB) = P(B) PB(A)
we obtain the important formula:
PB(A) = P(A) PA(B)/P(B), (12)
which contains, in essence, the Theorem of Bayes.
(Continues…)
Excerpted from "Foundations of the Theory of Probability"
by .
Copyright © 2018 Dover Publications, Inc..
Excerpted by permission of Dover Publications, Inc..
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
