Covariance and correlation
Covariance and correlation measure linear association — not all forms of dependence.
$$\mathrm{cov}(x,y)=\frac{1}{N}\sum_i (x_i-\mu_x)(y_i-\mu_y)$$
$$\rho_{x,y}=\frac{\mathrm{cov}(x,y)}{\sigma_x \sigma_y}\in[-1,1]$$
Reuse the familiar lists
$x=\\{{1,8,4,10,2\\}}$, $y=\\{{7,5,4,3,6\\}}$ with means 5. Covariance $= -4$, correlation $\\rho \\approx -0.82$: when $x$ is above its mean, $y$ tends to be below its mean.
Common mix-up
Zero correlation does not prove independence (nonlinear dependence can hide). Independence does imply zero covariance.
Key takeaways
- Correlation ∈ [−1, 1] for linear association.
- Sign shows direction of the linear link.
- Zero correlation ⇏ independence.