Bernoulli distribution

It happens very often in real life that an event may have only two outcomes that matter. For example, either you pass an exam or you do not pass an exam, either you get the job you applied for or you do not get the job, either your flight is delayed or it departs on time, etc. The probability theory abstraction of all such situations is a Bernoulli trial. Bernoulli trial is an experiment with only two possible outcomes that have positive probabilities p and q such that p + q = 1. The outcomes are said to be "success" and "failure", and are commonly denoted as "S" and "F" or, say, 1 and 0. For example, when rolling a die, we may be only interested whether 1 shows up, in which case,naturally, P(S) = 1/6 and P(F) = 5/6. If, when rolling two dice, we are only interested whether thesum on two dice is 11, P(S) = 1/18, P(F) = 17/18. The Bernoulli process is a succession of independent Bernoulli trials with the same probability of success. One important question about a succession of n Bernoulli trials is the probability of k success. Since the individual trials are independent, we are talking of the product of probabilities of successes and failures. Such a product is independent of the order in individual successes and failures come about. For example, P(SSFSF) = P(SFFSS) = P(FFSSS) = p3q2. In general, the probability of k successes in n trials is denoted b(k; n, p) and is equal to b(k; n, p) = C(n, k)pkqn - k, where C(n, k) is the binomial coefficient n choose k. Observe that, by the binomial formula, b(k; n, p) over k from 0 to n is exactly 1: b(k; n, p) = C(n, k)pkqn - k = (p + q)n = 1. As a function of k, b(k; n, p) is known as the binomial distribution and plays an important role the theory of probabilities.

Binomial Distribution
Binomial distribution is the distribution of a total number of successes in a given number of Bernoulli trials. The common notation is b(k; n, p), where k is the number of successes, n is the number of trials, p is the probability of success. We know that b(k; n, p) = C(n, k) pk(1 - p)n - k. For a fixed number of trials n, the binomial distribution always behaves in the same way: as a function of k, it is monotone increasing up to a certain point m after which (perhaps with an exception of the next point) it is monotone decreasing. Indeed, b(k; n, p) / b(k-1; n, p) = C(n, k) pk(1 - p)n - k / C(n, k-1) pk-1(1 - p)n - k + 1 (1) = (n - k + 1) p / k(1 - p) = 1 + ((n + 1) p - k) / k(1 - p). It is clear now that the right hand side in (1) is greater than 1 whenever k > (n + 1) p and it is less than 1 when k < (n + 1) p. It may happen, of course, that m = (n + 1) p is an integer in which case b(m; n, p) = b(m - 1; n, p). In any event, there is only one integer m that satisfies (2) (n + 1) p - 1 < m (n + 1) p. Summing up, as a function of k, the expression b(k; n, p) is monotone increasing for k < m and monotone decreasing for k > m, with the exception of one case where (n + 1) p is an integer. In this case, there are two maximum values for m = (n + 1) p and m - 1.

The number m that satisfies (2) is known as the most probable (most likely) number of successes in n Bernoulli trials. As a matter of fact, b(m; n, p) is quite small for a large n, even for a reasonable value of p. It is also always different from the average number of successes. The latter could be found the following way. Letting q = 1 - p, we get k b(k; n, p) = k C(n, k) pkqn - k = pqn-1 k C(n, k) (p/q)k - 1 = pqn-1 n (1 + p/q)n - 1 = pqn-1 n (q + p)n - 1 / qn - 1 = p n, where we used the identity n (1 + x)n - 1 = k C(n, k) xk - 1. The result for the expected value np = k b(k; n, p) might have been anticipated given the interpretation of the probability as a relative frequency. Note that the expected value np is always different from the most likely value (n + 1) p, provided of course p 0.