Honey, I Shrunk the Sample Covariance Matrix

Ledoit, O., & Wolf, M. (2004). Honey, I Shrunk the Sample Covariance Matrix. The Journal of Portfolio Management, 30(4), 110–119. https://doi.org/10.3905/jpm.2004.110

Mean-variance optimisation (MVO) proposed by Markowitz provides theoretical guarantees, and can be coded to run very efficiently, but it needs good inputs. In particular, it requires a good risk model, that is, a good estimator of covariance.

The sample covariance is the default choice, but often has coefficients with extreme errors which are particularly dangerous in MVO because the optimiser is likely to make large allocations based on these coefficients. One possible improvement is to move extreme values towards the centre, in a process called shrinkage.

Formal description of the portfolio optimisation problem

A manager’s portfolio can be thought of as a position in the benchmark (with a weight vector $w_B$) and an additional long/short position in certain stocks – the active weights $x$ (also a vector). Clearly, because the portfolio weights ($w_P$) sum to one, we have $w_P = w_B + x$.

The expected stock excess returns are given by the mean portfolio returns less the returns from the benchmark.

\[\alpha = \mu - w_B^T\mu\]

The optimisation problem can then be laid out as follows (where $\Sigma$ denotes the covariance matrix and $\mathbf{1}$ is a conforming vector of ones):

\[\begin{equation*} \begin{aligned} & \underset{x}{\text{minimize}} & & x^T\Sigma x \\ & \text{subject to} & & x^T\alpha \geq g \\ &&& x^T\mathbf{1} = 0 \\ &&& x \geq -w_B \\ &&& x \leq c\mathbf{1} - w_B \\ \end{aligned} \end{equation*}\]

The constraints on this optimisation are respectively interpreted as follows:

Shrinkage estimation of the sample covariance matrix

\[\delta F + (1-\delta)S, \qquad 0 \leq \delta \leq 1\]

The shrinkage target

\[f_{ii} = s_{ii} \qquad f_{ij}= \bar{r}\sqrt{s_{ii}s_{jj}}\]

The shrinkage coefficient

The shrinkage coefficient depends on an estimator $\hat{\kappa}$ where:

\[\hat{\kappa} = \frac{\hat{\pi} - \hat{\rho}}{\hat{\gamma}}\]

Let $T$ denote the number of price datapoints and $y_{it}$ be the returns of the $i$th security at time t.

$\hat{\pi}$ estimates the sum of asymptotic variances of the entries of the sample covariance matrix scaled by $\sqrt{T}$:

\[\hat{\pi} = \sum_i^N \sum_j^N \hat{\pi}_{ij} ~~ \text{with} ~~ \hat{\pi}_{ij} = \frac 1 T \sum_{t=1}^T \{(y_{it} - \bar{y}_i)(y_{jt} - \bar{y}_j) -s_{ij}\}^2\]

$\hat{\rho}$ estimates the sum of asymptotic covariances of the entries of the shrinkage target with the entries of the sample covariance matrix, scaled by $\sqrt{T}$

\[\hat{\rho} = \sum_{i=1}^N \hat{\pi}_{ij} + \sum_{i=1}^N \sum_{j=1,j \neq i}^N \frac{\bar{r}}{2} \left( \sqrt{\frac{s_{jj}}{s_{ii}}}\hat{\theta}_{ii,ij} + \sqrt{\frac{s_{ii}}{s_{jj}}}\hat{\theta}_{jj,ij} \right)\]


\[\hat{\theta}_{ii,ij} = \frac 1 T \sum_{t=1}^T \{(y_{it} - \bar{y}_i)^2 -s_{ii}\}\{(y_{it} - \bar{y}_i)(y_{jt} - \bar{y}_j) -s_{ij}\}\]

$\hat{\gamma}$ estimates the misspecification of the shrinkage target:

\[\hat{\gamma} = \sum_{i=1}^N\sum_{j=1}^N (f_{ij} - s_{ij})^2\]

The optimal shrinkage constant is given by $\frac{\hat{\kappa}}{T}$, but in principle because this value can be greater than 1 or less than 0, we truncate it with:

\[\hat{\delta} = \max \left \{ 0, \min \left \{ \frac{\hat{\kappa}}{T},1 \right \} \right \}\]

Empirical evaluation