14  Covariance and Correlation

14.1 Covariance

  • The covariance between two random variables \(X\) and \(Y\) is \[\begin{align*} \text{Cov}(X,Y) & = \text{E}\left[\left(X-\text{E}[X]\right)\left(Y-\text{E}[Y]\right)\right]\\ & = \text{E}(XY) - \text{E}(X)\text{E}(Y) \end{align*}\]
  • Covariance is the long run average of the products of the paired deviations from the mean
  • Covariance is the expected value of the product minus the product of expected values.
    • Remember: in general \(\text{E}(XY)\) is not necessarily equal to \(\text{E}(X)\text{E}(Y)\)

Example 14.1 In the meeting problem, assume the two arrival times \(X\) and \(Y\) each have a Uniform(0, 60) distribution, independently. Let \(T=\min(X, Y)\) be the time of the first arrival and let \(W=|X-Y|\) be the waiting time of the first person for the second to arrive.

  1. Describe in detail how you would use simulation to approximate \(\text{Cov}(T, W)\).




  2. Is \(\text{Cov}(T, W)\) positive, negative, or zero? Explain why.




  • \(\text{Cov}(X,Y)>0\) (positive association): above average values of \(X\) tend to be associated with above average values of \(Y\)
  • \(\text{Cov}(X,Y)<0\) (negative association): above average values of \(X\) tend to be associated with below average values of \(Y\)
  • \(\text{Cov}(X,Y)=0\) indicates that the random variables are uncorrelated: there is no overall positive or negative association.
    • \(X\) and \(Y\) are uncorrelated if and only if \(\text{E}(XY)= \text{E}(X)\text{E}(X)\)
    • But be careful: if \(X\) and \(Y\) are uncorrelated there can still be a relationship between \(X\) and \(Y\); there is just no overall positive or negative association.

Example 14.2 Roll a fair four-sided die twice. Let \(X\) be the sum of the two dice, and let \(Y\) be the larger of the two rolls (or the common value if both rolls are the same). The table below represents the joint distribution of \(X\) and \(Y\).

In previous exercises we computed \(\text{E}(X) = 5\) and \(\text{E}(Y) = 3.125\)

\(x\) \ \(y\) 1 2 3 4
2 1/16 0 0 0
3 0 2/16 0 0
4 0 1/16 2/16 0
5 0 0 2/16 2/16
6 0 0 1/16 2/16
7 0 0 0 2/16
8 0 0 0 1/16
  1. Compute \(\text{E}(XY)\).




  2. Compute \(\text{Cov}(X, Y)\).




Example 14.3 Consider the probability space corresponding to two rolls of a fair four-sided die. Let

  • \(X\) be the sum of the two rolls
  • \(Y\) the larger of the two rolls
  • \(W\) the number of rolls equal to 4
  • \(Z\) the number of rolls equal to 1
  • \(V = W + Z\)

Without doing any calculations, determine if the covariance between each of the following pairs of variables is positive, negative, or zero. Explain your reasoning conceptually.

  1. \(\text{Cov}(X,W)\)




  2. \(\text{Cov}(X,Z)\)




  3. \(\text{Cov}(X,V)\)




  4. \(\text{Cov}(W,Z)\)




  5. \(\text{Cov}(X,V) = 0\). Are \(X\) and \(V\) independent?




  • \(\text{Cov}(X,Y)=0\) indicates that the random variables are uncorrelated: there is no overall positive or negative association.
    • \(X\) and \(Y\) are uncorrelated if and only if \(\text{E}(XY)= \text{E}(X)\text{E}(Y)\)
  • But be careful: if \(X\) and \(Y\) are uncorrelated there can still be a relationship between \(X\) and \(Y\); there is just no overall positive or negative association.
  • If \(X\) and \(Y\) are independent then \(\text{Cov}(X,Y)=0\)
  • But the converse is not true; there are many examples of uncorrelated random variables that are not independent.

Example 14.4 The covariance between height in inches and weight in pounds for football players is 96.

  1. What are the measurement units of the covariance?


  2. Suppose height were measured in feet instead of inches. Would the shape of the joint distribution (say, represented by a scatterplot) change? Would the strength of the association between height and weight change? Would the value of covariance change?




  • The numerical value of the covariance depends on the measurement units of both variables, so interpreting it can be difficult.
  • Covariance is a measure of joint association between two random variables that has many nice theoretical properties, but the correlation (coefficient) is often a more practical measure.

14.2 Correlation

  • The correlation (coefficient) between random variables \(X\) and \(Y\) is \[\begin{align*} \text{Corr}(X,Y) & = \text{Cov}\left(\frac{X-\text{E}(X)}{\text{SD}(X)},\frac{Y-\text{E}(Y)}{\text{SD}(Y)}\right)\\ & = \frac{\text{Cov}(X, Y)}{\text{SD}(X)\text{SD}(Y)} \end{align*}\]
  • The correlation for two random variables is the covariance between the corresponding standardized random variables. Therefore, correlation is a standardized measure of the association between two random variables.
  • A correlation coefficient has no units and is measured on a universal scale. Regardless of the original measurement units of the random variables \(X\) and \(Y\) \[ -1\le \textrm{Corr}(X,Y)\le 1 \]
  • \(\textrm{Corr}(X,Y) = 1\) if and only if \(Y=aX+b\) for some \(a>0\)
  • \(\textrm{Corr}(X,Y) = -1\) if and only if \(Y=aX+b\) for some \(a<0\)
  • Therefore, correlation is a standardized measure of the strength of the linear association between two random variables.
  • The closer the correlation is to 1 or \(-1\), the closer the joint distribution of \((X, Y)\) pairs hugs a straight line, with positive or negative slope.
  • Because correlation is computed between standardized random variables, correlation is not affected by a linear rescaling of either variable (e.g., a change in measurement units from minutes to seconds)
  • Covariance is the correlation times the product of the standard deviations. \[ \text{Cov}(X, Y) = \text{Corr}(X, Y)\text{SD}(X)\text{SD}(Y) \]

Example 14.5 The covariance between height in inches and weight in pounds for football players is 96. Heights have mean 74 and SD 3 inches. Weights have mean 250 pounds and SD 45 pounds. Find the correlation between weight and height.