26 Expected Values of Linear Combinations of Random Variables

A linear rescaling of a random variable $X$ is of the form $aX+b$ where $a$ and $b$ are non-random constants. A linear combination of two random variables $X$ and $Y$ is of the form $aX + bY$ where $a$ and $b$ are non-random constants.

Properties of expected value

Expected value and linear rescaling: If $X$ is a random variable and $a, b$ are non-random constants then \[ \text{E}(aX + b) = a\text{E}(X) + b \]
Linearity of expected value: For any two random variables $X$ and $Y$, \[\begin{align*} \text{E}(X + Y) & = \text{E}(X) + \text{E}(Y) \end{align*}\]
That is, the expected value of the sum is the sum of expected values, regardless of how the random variables are related.
Therefore, you only need to know the marginal distributions of $X$ and $Y$ to find the expected value of their sum. (But keep in mind that the distribution of $X+Y$ will depend on the joint distribution of $X$ and $Y$.)
Combining properties of linear rescaling with linearity of expected value yields the expected value of a linear combination, which extends naturally to more than two random variables. \[ \text{E}(aX + bY + c) = a\text{E}(X)+b\text{E}(Y) + c \]

Properties of variance

Variance and linear rescaling: If $X$ is a random variable and $a, b$ are non-random constants then \[\begin{align*} \text{SD}(aX + b) & = |a|\text{SD}(X)\\ \text{Var}(aX + b) & = a^2\text{Var}(X) \end{align*}\]
Variance of sums and differences of random variables \[\begin{align*} \text{Var}(X + Y) & = \text{Var}(X) + \text{Var}(Y) + 2\text{Cov}(X, Y)\\ \text{Var}(X - Y) & = \text{Var}(X) + \text{Var}(Y) - 2\text{Cov}(X, Y) \end{align*}\]
The variance of the sum (or difference) is the sum of the variances if and only if $X$ and $Y$ are uncorrelated. \[\begin{align*} \text{Var}(X+Y) & = \text{Var}(X) + \text{Var}(Y)\qquad \text{if $X, Y$ are uncorrelated}\\ \text{Var}(X-Y) & = \text{Var}(X) + \text{Var}(Y)\qquad \text{if $X, Y$ are uncorrelated} \end{align*}\]
If $a, b, c$ are non-random constants and $X$ and $Y$ are random variables then \[ \text{Var}(aX + bY + c) = a^2\text{Var}(X) + b^2\text{Var}(Y) + 2ab\text{Cov}(X, Y) \]

Example 26.1 Recall the SAT score Application assignment.

Let $M$ be the Math score and $R$ be the Reading (and Writing) score of a randomly selected SAT taker. We’ll assume $(M, R)$ pairs follow a Bivariate Normal distribution.

Math scores ($M$) have mean 500 and standard deviation 140
Reading scores ($R$) have mean 520 and standard deviation 110

For each of the following scenarios, compute

$\text{E}(M + R)$
$\text{Var}(M + R)$
$\text{SD}(M + R)$

$\text{Corr}(M, R) = 0$
$\text{Corr}(M, R) = 0.6$
$\text{Corr}(M, R) = -0.9$
In which of these scenarios is $\text{SD}(M + R)$ largest? Smallest? Can you explain why intuitively?

Example 26.2 Recall the SAT score Application assignment.

Let $M$ be the Math score and $R$ be the Reading (and Writing) score of a randomly selected SAT taker. We’ll assume $(M, R)$ pairs follow a Bivariate Normal distribution.

Math scores ($M$) have mean 500 and standard deviation 140
Reading scores ($R$) have mean 520 and standard deviation 110

For each of the following scenarios, compute

$\text{E}(M - R)$
$\text{Var}(M - R)$
$\text{SD}(M - R)$

$\text{Corr}(M, R) = 0$
$\text{Corr}(M, R) = 0.6$
$\text{Corr}(M, R) = -0.9$
In which of these scenarios is $\text{SD}(M - R)$ largest? Smallest? Can you explain why intuitively?

Two continuous random variables $X$ and $Y$ have a Bivariate Normal distribution with parameters $\mu_X$, $\mu_Y$, $\sigma_X>0$, $\sigma_Y>0$, and $-1<\rho<1$ if the joint pdf is \[ {\scriptsize f_{X, Y}(x,y) = \frac{1}{2\pi\sigma_X\sigma_Y\sqrt{1-\rho^2}}\exp\left(-\frac{1}{2(1-\rho^2)}\left[\left(\frac{x-\mu_X}{\sigma_X}\right)^2+\left(\frac{y-\mu_Y}{\sigma_Y}\right)^2-2\rho\left(\frac{x-\mu_X}{\sigma_X}\right)\left(\frac{y-\mu_Y}{\sigma_Y}\right)\right]\right) } \]
It can be shown that if the pair $(X, Y)$ has a BivariateNormal($\mu_X$, $\mu_Y$, $\sigma_X$, $\sigma_Y$, $\rho$) distribution \[\begin{align*} \text{E}(X) & =\mu_X\\ \text{E}(Y) & =\mu_Y\\ \text{SD}(X) & = \sigma_X\\ \text{SD}(Y) & = \sigma_Y\\ \text{Corr}(X, Y) & = \rho \end{align*}\]
A Bivariate Normal Density has elliptical contours. For each height $c>0$ the set $\{(x,y): f_{X, Y}(x,y)=c\}$ is an ellipse. The density decreases as $(x, y)$ moves away from $(\mu_X, \mu_Y)$, most steeply along the minor axis of the ellipse, and least steeply along the major of the ellipse.
A scatterplot of $(x,y)$ pairs generated from a Bivariate Normal distribution will have a rough linear association and the cloud of points will resemble an ellipse.
If $X$ and $Y$ have a Bivariate Normal distribution, then the marginal distributions are also Normal: $X$ has a Normal$\left(\mu_X,\sigma_X\right)$ distribution and $Y$ has a Normal$\left(\mu_Y,\sigma_Y\right)$.
If $X$ and $Y$ have a Bivariate Normal distribution and $\text{Corr}(X, Y)=0$ then $X$ and $Y$ are independent. (Remember, in general it is possible to have situations where the correlation is 0 but the random variables are not independent.)
$X$ and $Y$ have a Bivariate Normal distribution if and only if every linear combination of $X$ and $Y$ has a Normal distribution. That is, $X$ and $Y$ have a Bivariate Normal distribution if and only if $aX+bY$ has a Normal distribution for all $a$, $b$.

Example 26.3 Let $M$ be the Math score and $R$ be the Reading (and Writing) score of a randomly selected SAT taker. We’ll assume $(M, R)$ pairs follow a Bivariate Normal distribution.

Math scores ($M$) have mean 500 and standard deviation 140
Reading scores ($R$) have mean 520 and standard deviation 110
The correlation is 0.6

Compute the probability that a student has a total score above 1400.
Compute the probability that a student has a higher Math than Reading score.

Properties of covariance

\[\begin{align*} \text{Cov}(X, X) &= \text{Var}(X)\qquad\qquad\\ \text{Cov}(X, Y) & = \text{Cov}(Y, X)\\ \text{Cov}(X, c) & = 0 \\ \text{Cov}(aX+b, cY+d) & = ac\text{Cov}(X,Y)\\ \text{Cov}(X+Y,\; U+V) & = \text{Cov}(X, U)+\text{Cov}(X, V) + \text{Cov}(Y, U) + \text{Cov}(Y, V) \end{align*}\]

Example 26.4 Let $X$ be the number of two-point field goals a basketball player makes in a game, $Y$ the number of three point field goals made, and $Z$ the number of free throws made (worth one point each). Assume $X$, $Y$, $Z$ have standard deviations of 2.5, 3.7, 1.8, respectively, and $\text{Corr}(X,Y) = 0.1$, $\text{Corr}(X, Z) = 0.3$, $\text{Corr}(Y,Z) = -0.5$.

Find the standard deviation of the number of fields goals in a game (not including free throws)
Find the standard deviation of total points scored on fields goals in a game (not including free throws)
Find the standard deviation of total points scored in a game.