Math 361: Reading Guide to the Pitman text

Section 1.1: Equally likely outcomes

• p.2 - 5: Here the basic mathematical model (consisting of a set Omega, the "sample space", with elements corresponding to the possible "outcomes") for a probabilistic (or random) experiment in which all outcomes are equally likely is introduced. Examples 1,2, 3 are very simple, but instructive.
• p.6 - end of section: This deals with the concept of "odds" which is commonly used in betting situations and is directly related to the "probability" concept: Saying that the odds are 1 : 3 for an event A means that A has probability 1/(1+3). All you need to know from this part is this relation between "odds" and "probability".
• Birthday problem: An important type of problem is the birthday problem, which was discussed in class, and which came up again in HW 1. This problem fits well within this framework (probability spaces with equally likely outcomes); it does show up in the text, but only in a later section.
• Exercises 1.1
  • 3 (Sampling with/without replacement) This is an important example, that was discussed in class.
  • 7: A simple exercise in working with an explicit outcome space.
  • 8: A slightly more complicated version of the previous problem; this problem was included in HW 1.

Section 1.2: Interpretations

Section 1.3: Distributions

• p.19, table: This is a "translation table" between "event language" and set-theoretic language; you should familiarize yourself with that table.
• p.21: Rules of Probability: These three rules are the famous "Kolmogorov axioms" which form the basis of modern probability theory. All other rules can be derived from these three fundamental rules. Important examples of such additional rules are given on p.21 - 23: the complement rule, the difference rule, and the inclusion-exclusion rule (formula for P(A union B)).
• Example 2: This is a simple example of a general probability distribution, defined by a two-row table.
• p.24 - end of section: Skip this part. (Named distributions will come up later anyway.)
• Exercises 1.3
  • 8 - 10: These are fairly routine exercises in applying probability rules. Several of these problems were part of HW 2. In doing these exercises, it is helpful to draw a picture.
  • 11: Here an analog of the inclusion-exclusion formula for 3 events is to be derived. Not overly difficult, but requires considerably more work than the case of 2 events.

Section 1.4: Conditional Probability and Independence

• p. 33 - 35: You can skip this part (or skim through it). It is intended to motivate the concept of a conditional probability and deals with the special case of equally likely probability.
• p. 36, General formula for conditional probability. This is the definition of a cond. prob. that you should work with.
• p.36 (bottom) - 40 (top), Tree diagrams. Skip this section.
• p.40 - 41, Average Rule: This is an important section. You should know the formula on p. 41, understand its proof via the addition and multiplication rules (you will have to do similar proofs in homework and exam problems), and study Example 7.
• p.42 - 45, Independence: The approach taken in the book is different from (but equivalent to) the one taken in class. In class, we defined independence by the multiplication rule given at the bottom of p.42., and then derived the conditional probability formula (P(A|B) = P(A) from that definition. The book does it the other way around. The top of p.43 gives a good example of deriving further properties from the definition of independence, using appropriate probability rules.
• Exercises 1.4
  • 2. This is a fairly standard exercise in using conditional probabilities. There are four relevant events. Introduce notations for these events, express the given probabilities and the prob. asked in the problem as probabilities involving these events, and try to derive the latter from the former by applying appropriate rules.
  • 3,4: These are easy exercises in using the definitions of conditional probabilities and independence, and probability rules. For 4, call the two events A and B, then express both the given data and the probabilities asked for in terms of these two events. For example, the prob. that "at least one event occurs" is 1- P(none of the event occurs) = 1 - P(A^cB^c). In the latter formula, apply first the multiplication rule (which is applicable here since whenever A and B are independent, then so are the complements of these two sets - see top of p.43), then the complement rule.
  • 7: This is a somewhat unconventional (but not hard) problem. The key is to use the formula for P(A union B) in terms of P(A), P(B), and P(AB), plug in P(A), then determine P(B) so that P(AB)=0 (part (a)), or P(AB)=P(A)P(B) (part (b)).
  • 12: An instructive problem. Use the definition of conditional probabilities, and then apply appropriate probability rules to eliminate all probabilities involving complements and to express everything in terms of P(F), P(G), and P(FG).

Section 1.5: Bayes' Rule

• Example 3 (false positives) Of the examples given in this section, this one is the most instructive. There is a similar problem in the current HW assignment.
• Bayes' Rule for Odds (bottom of p. 51 through end of section): You can skip that section.
• Exercises 1.5:
  • 4 (Transmission errors): This problem, part of HW 2, is a rather routine Bayes' rule problem.
  • 5 (False diagnosis): Another illustration of Bayes' rule in a real-world scenario, similar to Example 3.

Section 1.6: Sequences of Events

Chapter 1 Summary (pp. 72 - 73)

Section 2.1: The Binomial Distribution

• p.80: You can skip the tree diagram derivation of the binomial distribution (which we didn't cover in class). You should, however, be familiar with the approach taken in class, where the binomial distribution was introduced within the general success/failure framework.
• p. 81, Formula for binomial distribution: The boxed formula given here is probably the most important formula you'll encounter in this class, so be sure to memorize this formula. You should also be familiar with the basic properties of binomial coefficients, and the binomial theorem (or binomial expansion) given in the same box.
• p. 84 - 85, Consecutive Odds Ratios: You can skip this section, through the boxed formula on p.85 (but don't skip Example 2 on the bottom of p. 85, which is quite instructive).
• p. 86, Most likely number of successes: Skip this page.
• p. 87 - 89: Graphs of the binomial distribution: These bar graphs ("histograms") of the binomial distribution for various values of p and n are interesting in that they show how changes in p affect these distributions, and that for large n these distributions approximate a bell-shaped curve. This approximation, the "normal approximation", is extremely important, and will be studied in detail in 2.2.
• p.90, Expected number of successes: Skip this page. (We will cover this topic later in the course.)

• Exercises 2.1:
  • 1: A quickie.
  • 2:This is essentially the 4-child example worked out in class.
  • 3:Simple, but instructive, applications of the binomial formula to various types of probabilities. Highly recommended! (The answers are in the back of the book.)
  • 4, 5:Problem 4 is part of HW 3; Problem 5 is of the same type. Use the definition of conditional probabilities. For Problem 5(b) split the 20 tosses (= trials) into two separate, independent S/F trial sequences, one consisting of the first 2 tosses, the other consisting of the remaining 18 tosses. (The other parts can be dealt with similarly.)
  • 7: A variation of this problem is part of HW 3. It's a two-stage problem, similar to Example (2) (on 4-child families) worked out in class.
  • 10: After using the definition of conditional probabilities and the binomial distribution formula, his becomes a rather simple exercise in manipulating binomial coefficients.
  • 12:This is part of HW 3.

Section 2.2: The normal approximation

Graphs: The graphs on p. 96 - 98 illustrate nicely the normal distribution and how this distribution approximates the binomial distribution.

Example 1: This example was (with different numbers) done in class using the first formula on the normal approximation handout; the method given in the text is much more complicated, so stick to the method presented in class.

Square root law, p. 100: This is more a rule of thumb than a well-defined mathematical "law". It may give you an idea of what to expect, but you cannot use this "law" when doing problems; for that, you will need to apply the normal approximation formula. The square root "law" will not be on the exam.

Confidence intervals, p. 101-102. You should not memorize specific probabilities for confidence intervals, but you should know how to derive confidence intervals from general formulas for normal approximation. (The derivation is quite easy and was done in class.)

Examples 2 - 4: These are instructive examples in applying confidence intervals. Examples 2 and 4 deal with the situation in which p is unknown, as in the case of opinion polls. (See the class of 2/21 for setting up a S/F model for opinion polls.)

Skew-normal approximation, p. 103 - end of section: You can skip this part.

Exercises 2.2:

• 1,3: This is a quickie.
• 6: Part of HW 4. A routine application of normal approximation.
• 8:A quickie.
• 9 (airline overbooking): This was essentially done in class.
• 10: Part of HW 4.
• 11 (batting averages): An instructive, but not too difficult problem.
• 12: An instructive but not very hard problem, related to the derivation of confidence intervals.
• 13: A problem on opinion polls and confidence intervals.
• 14: Part of HW 4. Part (a) is completely routine; part (b) is (somewhat) similar to the airline overbooking problem.

Section 2.3: Derivation of the normal approximation

Section 2.4: The Poisson approximation

• p. 117: The three graphs of binomial distributions with different values of n and p (related by p=1/n in all three cases) illustrate the Poisson approximation. The graphs are nearly identical, and if the graph of the Poisson distribution with parameter mu = 1 had been plotted as well, it would be also nearly indistinguishable from these three graphs. This confirms the fact that (under appropriate conditions, which are clearly satisfied here), a binomial distribution is well approximated by the Poisson distribution with parameter mu = np (which is 1 in this case).
• p.118: Here some theoretical background behind the Poisson approximation is presented. You are not expected to know this material and you can therefore skip that page, but you might want to skim through the page if you are curious as to why the Poisson approximation works.
• Example 4: This is a simple, but instructive example which you should study. We did a similar example in class.
• Formulas: You need to memorize the boxed formulas for the Poisson distribution on p. 201, and for the Poisson approximation on p.119. (The latter is simply the formula for the Poisson distribution with a particular choice of mu, namely mu = n p).
• Graphs: The bar graphs ("histograms") of the binomial and Poisson distribution on p. 117 and p. 120 give you an idea of what these distributions look like with various choices of the parameters involved (n, p, mu).
• Exercises 2.4 Problems 2 - 5 are easy quickies, similar to Example 4.

Section 2.5: Random sampling

Chapter 2 Summary (p. 130 - 131)

• Consecutive ratios, and mode of binomial distribution (bottom of p. 130 and top of p. 131)
• In the formula for P(a to b) on p. 131 you can ignore the "1/2 term" (use the simplified version given on the Normal Distribution handout).
• Formulas for P(mu - sigma to mu + sigma), etc. (middle of p. 131): You should know how to derive these formulas via the general normal approximation formula, but you should not memorize the numerical values given here.
• Square root law: Again, do not memorize the given formula, but know how to derive the formula.
• Random sampling (bottom of p.131): This will be discussed later, in connection with combinatorial probabilities. You do not need to know these formulas for now.

3.1 Random variables: Introduction

• p.139 - 141 (Introduction and Distribution of X): Read this part. It's an easy read, and has some simple but instructive examples.
• p.141, bottom - 143: You can skip most of this part, to the extent that it involves a function g(W). However, the examples on p. 143 are simple illustrations of "Type II" problems (as defined in the lecture of 3/22).
• p.144 - 150, top (joint distributions): Study this part, and in particular Examples 2 - 4. The boxed formulas on top of p. 147 are illustrations of type II probability computations.
• p.150, middle - end of section (Conditional distributions, Several random variables, Symmetry): Skip these sections except for the definitions of independence of random variables (see the bottom of p. 151 for the case of two r.v.'s and middle of p.154 for more than two r.v.'s)
• Exercises 3.1
  • 1. Quickie
  • 2. An instructive problem that was (essentially) done in class.
  • 3. An easy problem, but (b) is somewhat tedious (unless you spot the pattern).
  • 4. Part of HW 6.
  • 6. Part of HW 6.
  • 7. This boils down to a familiar exercise in set-theoretic probability.
  • 9. Part of HW 6.
  • 13(a). Part of HW 6.
  • 14. Part of HW 6. An interesting and instructive problem, and one of the problems included in the Problem Sampler handed out at the beginning of class.
  • 22. Somewhat unconventional, but easier than it looks. Just use the definitions of independence.

3.2 Expectation

The presentation of expectation in the text is excellent, and you should read all of Section 3.2, except for the following parts:

• p. 171, bottom - 174: Skip the tail sum formula on p. 171, Boole's inequality on p. 173, and Markov's inequality on p. 174. However, the first part of Example 9, and the "Discussion" following that example, is instructive as it illustrates the minimum trick.
• p. 178 - 179: Skip the part on "Expectation and Prediction."
• Exercises 3.2
  • 1. Quickie
  • 3. Quickie
  • 5. A practical application of the "fair price" interpretation of an expectation. The calculations are not difficult, but somewhat tedious, as they break down into many cases.
  • 7. Use the indicator method.
  • 8. Part of HW 7.
  • 9. Routine. Just use the properties of the expectation.
  • 10. This is similar to, though somewhat simpler than, 3.3.8, which is part of HW 7. Not particularly hard - just use the definition of an indicator random variable. The key is to understand what the product of two indicator r.v.'s, I_A I_B, is.
  • 13. Part of HW 7.
  • 14. Another case for the indicator method. Of the same type as the birthday problem done in class.

Be sure to study the formula summary on p. 181 and memorize those formulas; this summary complements the end-of-chapter summary on p. 248 - 249.

3.3 Variance, Standard Deviation, Normal Approximation

• p. 185 - 188 This part contains the basic definitions of variance and SD, and examples for computing these quantities. The variance of a r.v. is defined in terms of the expectation, and the standard deviation (SD) is simply the square root of the variance. To compute the variance, use the first of the two formulas given on p. 186. To find the SD, one almost always has to first compute the variance, which in turn requires computing expectations.
• p. 190, Example 6: This is an example in which a r.v. is given to have (approximately) normal distribution (as opposed to subsequent examples in which the random variables given in the problem have other distributions (e.g., uniform on {1,2,...,6}), and the normal distribution arises indirectly via normal approximation.
• p.191 - 193, middle: Skip the parts on "Tail probabilities" and "Chebyshev's inequality".
• p. 193 - 195: The addition rule for variances (valid if the random variables are independent) is important, and the square root law at the bottom of p. 194 is a simple application of the addition rule. The "Law of Averages" on p. 195 is primarily of theoretical interest; for solving concrete problems, however, it is not of much use. The normal approximation formula serves that purpose.
• p. 196 - 201: Normal approximation is the most important part of this section, and one of the most important topics in this class. You need to memorize the boxed formula on p. 196 (and also memorize the special case in which the lower bound (a <=...) and the Phi(a) term are omitted, since this is the form that occurs most often in problems. The graphs of distributions on p. 199 - 201 represent convincing illustrations of normal approximation at work.
  Skip: Skewness (p. 198).
• Exercises 3.3.
  • 3, 5. These problems are simple exercises in using the properties of expectation and variance.
  • 13. Part of HW 8.
  • 15(a). Another simple example in using properties of the variance.
  • 19. Part of HW 8.
  • 20. Part of HW 8.
  • 21(a). An interesting, and not very difficult, application of normal approximation.
  • 22. Part of HW 8.
  • 23. Part of HW 8.
  • 29(c). An instructive exercise, in which n, the number of terms in the sum (or average), is the unknown. Similar problems came up earlier in the semester (e.g., airline overbooking, polling).

3.4/3.5 Discrete Distributions

• Distributions to memorize: The relevant formulas for named distributions are given on the distribution summary on p. 476. From that table you need to know (i.e., memorize) the following distributions.
  • uniform on {a,a+1,...,b}: definition (i.e., formula for P(k) and range)
  • binomial(n,p): definition, mean (=expectation), variance
  • Poisson(mu): definition, mean
  • geometric(p) on {1,2,...}: definition, mean
  You do not need to memorize formulas for the other distributions in this table, or formulas for the variance of the Poisson and geometric distributions.
• Tools: Two important tools in working with geometric and Poisson distributions are:
  • Geometric series formula
  • Exponential series
  Both formulas are given in the box on p. 518.
• Examples: Among the examples given in the book, you should study the following:
  • Example 1, 3.4: Application of geometric series formula, example of an infinite distribution. A similar example was given in class (3/21).
  • Example 2, 3.4: Another application of the geometric series formula.
  • Example 4, 3.4 (Problem 1): This is related to the baseball world series problem.
• Skip: In 3.5, you can skip the skew normal approximation (p. 225) and p. 228 through the end of the section.
• Exercises 3.4
  • 1. An easy review exercise. Only requires the binomial distribution.
  • 5. (a) and (b) are quickies, (c) can be derived from (b); (c) is slightly trickier.
  • 7. (a) and (b) are variations of an example from class (3/21); (c) and (d) are a somewhat harder.
  • 11. The special case when p_A = p_B was done in class (3/19).
  • 12. A very instructive exercise in using the geometric distribution and the geometric series formulas (both the finite and infinite versions). (a) is the easiest of the three parts and will be done in class; (b) is more difficult, and involves more computations. (d) is not particularly hard using the idea behind the minimum/maximum trick.
• Exercises 3.5.
  • 1. A quickie.
  • 9. (a) is a quickie, and so is (b) with the complement trick. (c) requires a bit more work, but is otherwise routine.
  • 10. (a) and (b) are routine applications of the rules for expectation and variance (for the latter see the box on p. 193). (c) was done in class (3/19).
  • 11. (a) and (c) are simple examples of Type II problems. For (b) use the rules for expectation, the independence of X and Y, and the formula for the variance of a Poisson variable (see p. 476).
  • 15(a)(c). These are instructive, and not particularly difficult, two stage problems, with the first stage being the computation of the probability that a single page is "good" in the sense that it has no mistakes, and the second stage consisting of modeling the "good" pages as successes in a S/F sequence. [You can omit (b) which is slightly trickier.]

3.6 Symmetry

Chapter Summary, p. 248 - 249

Exercises 4.1

• 1: Not quite as trivial as it looks since the normal table (as given in the book) is not precise enough to compute the probabilities. The correct approach is to use the interpret the probabilities as areas under the normal curve, and to estimate those areas by the width of the "bar", times the height.
• 3: A routine exercise, This was done in class.
• 4: Similar to Problem 3. Part of HW 9.
• 5: (a) through (c) are routine, if you know the integral of 1/(1+x^2).
• 7: Part of HW 9.

4.2 Exponential and Gamma Distributions

The second topic is the Poisson Process defined on p. 284 and illustrated in Example 3. You can skip the material on the Gamma distribution (p.286 - 292).

Exercises 4.2

• 1: (a) and (b) are easy problems involving the exponential distribution. (A similar example was done in class.) (c) and (d) are trickier. Let t denote the time to be computed. and consider each of the 1024 atoms as a S/F trial with S meaning that the atom has not decayed by time t. The success probability, p, can be computed as in (a) or (b) (it will involve the unknown t). Further, given p, the expected number of surviving atoms at time t is 1024 p, by the formula for the expectation of the binomial distribution. For (c) set this expectation equal to 0.1 and solve for t. For (d) compute the probability that there are 0 successes, i.e., (1-p)^1024.
• 3: A routine application of the exponential distribution.
• 4: Part of HW 10.
• 5: A routine exercise in the Poisson process.

4.3 Hazard rates

4.4 Change of variables

Exercises 4.4.

• 3, 4, 5: All of these are simple exercises in changing variables. They can be done by the method illustrated in class, via the c.d.f.

4.5 Cumulative distribution functions

Exercises 4.5

• 5: Routine, but some care has to be taken because of the density involves the absolute value of x.
• 7(a): An exercise in changing variables. As illustrated in class (or in 5(b) of HW 9), do this by finding first the c.d.f. of Y, then differentiating to get the density.

4.6 Order Statistics

Chapter Summary, p. 332 - 333

• Hazard rates
• Expectation from the survival function
• Change of variables
• Percentiles
• Transformation by the inverse c.d.f.
• Order statistics

Exercises 4.R

• 3: This was done in class, using the maximum trick.
• 4: Part of HW 10. Parts (a) and (c) are fairly routine, but the integrations require some care because the integrand involves absolute values. To deal with the absolute value, consider separately the integral from -infinity to 0 and from 0 to +infinity. Computing the expectation and variance requires integration by parts which leads to a tedious and lengthy computation.
• 5: Part of HW 9.
• 14: An exercise in the Poisson process. Part of HW 10.
• 25: Part of HW 9.

5.1 Uniform Distributions

Exercises 5.1

• 1: An easy problem.
• 3: Also very simple.
• 4: An instructive example, which was done in class.
• 7: This deals with the minimum/maximum trick. A more general version (involving the minimum of n independent uniform r.v.'s) was worked out in class.

5.2 Densities

Integration techniques. Most problems involving joint densities require computing double integrals. A class handout, Double Integrals, gives some tips for doing such integrals and contains a number of practice problems. Solutions to these practice problems are available.

Examples: Among the examples given in this section, Examples 1 and 2 are very instructive and similar to problems worked out in class. You can skip Example 3.

Exercises 5.2

• 3: A routine exercise in working with joint density functions.
• 4: Part of HW 10.
• 5: An instructive exercise, which reduces to a (relatively simple) double integral computation. A similar problem was worked out in class.
• 11: Parts (a)-(c) are simple exercises in using the properties of expectation. Part (d) reduces (after writing the expectation as a product of expectations) to the computation of two single integrals.

5.3 Independent Normal Variables

Exercises 5.3

• 2: (a) is an easy exercise in using properties of expectation and variance; (b) is an application of the formula for the distribution of a sum (or, rather, a linear combination) of independent normal r.v.'s.
• 3: (a) and (b) are further illustrations of the same formula. (Part (c) requires the Chi-square distribution, which you are not expected to know.)
• 5: This was worked out in class.
• 6(a)(b): Another class example.

5.4 Operations