Variância Condicional

Prévia do material em texto

people who take their own hats.
Exercise 7.56 (Ross (2015; p.309)). These lecture notes have n typos. If they are read by k readers and each
reader finds each of the typos independently and with equal probability, p. Find the expected number of typos
that are found by none of the readers.
Exercise 7.57 (Ross (2015; p.379)). Ten hunters are waiting for ducks to fly by. When a flock of ducks flies
overhead, the hunters fire at the same time, but each chooses his target at random, independently of the others.
If each hunter independently hits his target with probability .6, compute the expected number of ducks that are
hit. Assume that the number of ducks in a flock is a Poisson random variable with mean 6.
Exercise 7.58. Let X, Y be random variables such that, Y ∼ N(0, 1) and X|Y = y ∼ N(y, 1).
(a) Find E[X|Y ].
(b) Find E[X].
(c) Find E[(X − E[X])2].
Exercise 7.59. Let N ∼ Poisson(1). Also, given N , Y = ∑Ni=1Xi, where X1, . . . , Xn are independent and
fXi|N (xi|n) = sin(xi)I(0 < xi < pi). Find E[Y ].
Exercise 7.60. Let (X,Y ) be a pair of random variables such that
f(x, y) =
xe−x(y+1) if x > 0 and y > 00 otherwise
(a) Determine fX|Y (x|y).
(b) Determine E[X|Y = y].
(c) Determine E[X].
(d) Calculate P(0 < X < 1|Y = 2).
(e) Calculate P(X < Y |X < 2Y ).
Exercise 7.61. Prove that if X is a discrete random variable, then, for every a ∈ R, E[X] = E[X|X ≤ a]P(X ≤
a) + E[X|X > a]P(X > a).
7.5 Conditional Variance
Definition 7.62. The conditional variance of X given Y = y, V[X|Y = y] is defined as
V[X|Y = y] = E[(X− E[X|Y = y])(X− E[X|Y = y])t]
In particular,
V[X] = E[(X− E[X])(X− E[X])t]
139
Note that V[X|Y = y] is a d× d matrix and that
V[X|Y = y]i,j = E[(Xi − E[Xi|Y = y])(Xj − E[Xj |Y = y])]
From the above, Cov[Xi, Xj |Y = y] is defined as V[X|Y = y]i,j .
Lemma 7.63. V[X|Y = y] = E[XXt|Y = y]− E[X|Y = y]E[X|Y = y]t.
Proof.
V[X|Y = y] =E[(X− E[X|Y = y])(X− E[X|Y = y])t] Definition 7.62
=E[XXt|Y = y]− E[XE[X|Y = y]t|Y = y]
− E[E[X|Y = y]Xt|Y = y] + E[X|Y = y]E[X|Y = y]t Lemma 7.49
=E[XXt|Y = y]− E[X|Y = y]E[X|Y = y]t
− E[X|Y = y]E[X|Y = y]t + E[X|Y = y]E[X|Y = y]t Lemma 7.49
=E[XXt|Y = y]− E[X|Y = y]E[X|Y = y]t
Lemma 7.64. Let k ∈ N∗ be arbitrary and A be a k × d matrix.
V[AX|Y = y] = AV[X|Y = y]At
In particular, if α ∈ Rd,
V[αtX|Y = y] = αtV[X|Y = y]α
Proof.
V[AX|Y = y] = E[AX(AX)t|Y = y]− E[AX|Y = y]E[AX|Y = y]t
= E[AXXtAt|Y = y]−AE[X|Y = y](AE[X|Y = y]t) Lemma 7.49
= AE[XXt|Y = y]At −AE[X|Y = y]E[X|Y = y]tAt Lemma 7.49
= A(E[XXt|Y = y]− E[X|Y = y]E[X|Y = y]t)At
= AV[X|Y = y]At Lemma 7.63
Lemma 7.65. For every X, V[X] is a positive semi-definite matrix.
Proof. Recall that V[X] is positive semi-definite if, for every α ∈ Rd, αTV[X]α ≥ 0. Note that αTX ∈ R.
V[αTX] ≥ 0 αTX ∈ R
αTV[X]α ≥ 0 Lemma 7.64 (19)
Conclude from eq. (19) that V[X] is positive semi-definite.
140
Lemma 7.66. Let b ∈ Rd,
V[X + b|Y = y] = V[X|Y = y]
Proof.
V[X + b|Y = y] =E[(X + b)(X + b)t|Y = y]− E[X + b|Y = y]E[X + b|Y = y]t Lemma 7.63
=E[XXt|Y = y] + bE[Xt|Y = y]t + E[Xt|Y = y]bt + bbt
− E[X|Y = y]E[X|Y = y]t − bE[Xt|Y = y]t − E[Xt|Y = y]bt − bbt Lemma 7.49
=E[XXt|Y = y]− E[X|Y = y]E[X|Y = y]t
=V[X|Y = y] Lemma 7.63
Lemma 7.67. If X1 and X2 are independent given Y, then
V[X1 + X2|Y = y] = V[X1|Y = y] + V[X2|Y = y]
Proof.
V[X1 + X2|Y = y] =E[(X1 + X2)(X1 + X2)t|Y = y]− E[X1 + X2|Y = y]E[X1 + X2|Y = y]t Lemma 7.63
=E[X1Xt1|Y = y] + E[X1Xt2|Y = y] + E[X2Xt1|Y = y] + E[X2Xt2|Y = y]
− E[X1|Y = y]E[X1|Y = y]t − E[X1|Y = y]E[X2|Y = y]t
− E[X2|Y = y]E[X1|Y = y]t − E[X2|Y = y]E[X2|Y = y]t Lemma 7.49
=E[X1Xt1|Y = y] + E[X1|Y = y]E[X2|Y = y]t
+ E[X2|Y = y]E[X1|Y = y]t + E[X2Xt2|Y = y]
− E[X1|Y = y]E[X1|Y = y]t − E[X1|Y = y]E[X2|Y = y]t
− E[X2|Y = y]E[X1|Y = y]t − E[X2|Y = y]E[X2|Y = y]t Lemma 7.50
=E[X1Xt1|Y = y]− E[X1|Y = y]E[X1|Y = y]t
+ E[X2Xt2|Y = y]− E[X2|Y = y]E[X2|Y = y]t
=V[X1|Y = y] + V[X2|Y = y] Lemma 7.63
Definition 7.68.
V[X|Y] = E[(X− E[X|Y])(X− E[X|Y])t]
Note that V[X|Y] is a function of Y and, therefore, is a random variable. Also, for every w ∈ Ω, V[X|Y](w) =
V[X|Y = Y(w)]. That is, V[X|Y] is a function of Y , g(Y ), such that g(Y(w)) = V[X|Y = Y(w)].
A useful result for computing variances is the following
141
Theorem 7.69 (Law of Total Variance).
V[X] = V[E[X|Y]] + E[V[X|Y]]
Proof. Note that,
V[E[X|Y]] = E[E[X|Y]E[X|Y]t]− E[E[X|Y]]E[E[X|Y]]t Lemma 7.63
= E[E[X|Y]E[X|Y]t]− E[X]E[X]t Theorem 7.52 (20)
Also,
E[V[X|Y]] = E[E[XXt|Y ]− E[X|Y]E[X|Y]t] Definition 7.62
= E[E[XXt|Y ]]− E[E[X|Y]E[X|Y]t] Lemma 7.49
= E[XXt]− E[E[X|Y]E[X|Y]t] Theorem 7.52 (21)
Conclude from eq. (20) and eq. (21) that
V[E[X|Y]] + E[V[X|Y]] = E[E[X|Y]E[X|Y]t]− E[X]E[X]t + E[XXt]− E[E[X|Y]E[X|Y]t]
= E[XXt]− E[X]E[X]t Theorem 7.52
= V[X] Lemma 7.63
Example 7.70. Recall Example 7.54.
V[T ] = V[E[T |N ]] + E[V[T |N ]] Theorem 7.69
= V
[
E
[
N∑
i=1
Xi
∣∣∣∣N
]]
+ E
[
V
[
N∑
i=1
Xi
∣∣∣∣N
]]
Example 7.54
= V
[
N∑
i=1
E [Xi|N ]
]
+ E
[
V
[
N∑
i=1
Xi
∣∣∣∣N
]]
Lemma 7.49
= V
[
N∑
i=1
E [Xi|N ]
]
+ E
[
N∑
i=1
V [Xi|N ]
]
Lemma 7.67
= V
[
N∑
i=1
nBpB
]
+ E
[
N∑
i=1
nBpB(1− pB)
]
Example 7.54
= V [NnBpB] + E [NnBpB(1− pB)]
= (nBpB)
2V[N ] + E [NnBpB(1− pB)] Lemma 7.64
= (nBpB)
2V[N ] + nBpB(1− pB)E[N ] Lemma 7.49
=
(nBpB)
2
pG
+ nBpB(1− pB)1− pG
p2G
Lemma 4.24
7.5.1 Exercises
Exercise 7.71. Let θ ∼ Beta(a, b) and X|θ = t ∼ Binomial(n, t). Find V[X].
142
Solution:
V[X] = V[E[X|θ]] + E[V[X|θ]] Theorem 7.69
= V[nθ] + E[nθ(1− θ)] Lemma 4.9
= n2V[θ] + nE[θ(1− θ)]
=
n2ab
(a+ b)2(a+ b+ 1)
+ n
Γ(a+ b)
Γ(a)Γ(b)
Γ(a+ 1)Γ(b+ 1)
Γ(a+ b+ 2)
Lemma 5.49
=
n2ab
(a+ b)2(a+ b+ 1)
+
nab
(a+ b+ 1)(a+ b)
=
n2ab+ nab(a+ b)
(a+ b)2(a+ b+ 1)
=
nab(a+ b+ n)
(a+ b)2(a+ b+ 1)
Exercise 7.72. Let (X,Y ) be a pair of random variables such that
f(x, y) =
12xy if 0 < y < x < 20 otherwise
(a) Determine fX and fY .
(b) Calculate E[X] and E[Y ].
(c) Calculate E[XY ]
(d) Calculate V[(X,Y )].
(e) Are X and Y independent?
7.6 *Some Multivariate Models
7.6.1 Multivariate Normal Distribution
Definition 7.73. Let Σ be a nonnegative definite d×dmatrix and µ ∈ Rd. We say that X ∼ Multivariate Normal(µ,Σ)
if, for every x ∈ Rd,
fX(x) =
1√
(2pi)d|Σ| exp
(
−1
2
(x− µ)Σ−1(x− µ)′
)
.
Lemma 7.74. If X ∼ Multivariate Normal(µ,Σ), then E[X] = µ and V ar[X] = Σ.
7.6.2 Multinomial Distribution
The multinomial distribution is a generalization of the binomial distribution. Say we throw a die with d faces
n times. If the trials are independent and each face has probability pi, i = 1, . . . , d, then (X1, . . . , Xd) ∼
Multinomial(n, (p1, . . . , pd)), where Xi is the number of times face i shows up.
Definition 7.75. Let p = (p1, . . . , pd) be such that pi > 0 for every i and
∑d
i=1 pi = 1 and n ∈ N. We say that
X = (X1, . . . , Xd) ∼ Multinomial(n,p) if
P(X1 = x1, . . . , Xn = xn) =
n!
x1! . . . xn!
d∏
i=1
pxii ,
143
for all xi ≥ 0 with
∑n
i=1 xi = n.
Lemma 7.76. If X = (X1, . . . , Xd) ∼ Multinomial(n,p), then E[X] = np and the component (i, j) of V ar[X] is
−npipj for i 6= j and npi(1− pi) otherwise.
Example 7.77. There are 3 candidates in an election, A, B and C. If 10% of the population vote for A, 40%
vote for B and 50% vote for C, then the probability that, in a sample of size n = 10 with replacement from this
population, we obtain 2 voters for A, 3 for B and 5 for C is given by
10!
2!3!5!
(0.1)2(0.4)3(0.5)5.
7.6.3 Exercises
Exercise 7.78. If X ∼ Multivariate Normal(µ,Σ) and a is a d dimensional vector, what is the expectation and
the variance of atX?
Exercise 7.79. Say X = (X1, X2) has a multivariate normaldistribution with mean µ and variance covariance
vector Σ, where (Σ)i,j = 0 for i 6= j. Prove that X1 is independent of X2 and find its distribution. This exercise
shows that, for multivariate normal distributions, covariance zero implies independence. We saw that this is
generally not true for other distributions.
Exercise 7.80. Say X = (X1, X2) has a multivariate normal distribution with mean µ and variance covariance
vector Σ. Compute the conditional distribution of X1|X2 = x2.
Exercise 7.81. Prove that if X ∼ Multinomial(n, (p1, . . . , pd)), then X1 ∼ Binomial(n, p1).
Exercise 7.82. Argue why the multinomial distribution models the number of times each face of a die shows up.
144