Conditioning of LSE I

Geometry of the Normal Equations

Formulation

A\mathbf{x}=\mathbf{b}

\mathbf{\hat{y}} = A(A^*A)^{-1}A^*\mathbf{b}

\kappa_{b \mapsto \hat{y}} = \frac{||\delta \hat{y}|| / ||\hat{y}||}{||\delta b|| / ||b||} = \frac{1}{\cos \theta}

Core Idea

This visualization shows how $\mathbf{b}$ is projected onto the plane defined by $A$ , and why the sensitivity to perturbations scales like $\frac{1}{\cos\theta}$ .

Normalize $||\mathbf{b}|| = 1$
Then $||\mathbf{\hat{y}}|| = \cos\theta$
As $\theta \to 90^\circ$ , $\cos\theta$ becomes small
So the sensitivity $\frac{1}{\cos\theta}$ blows up

Geometric Controls

θ (theta)45°

Angle between b and its projection ŷ.

φ (phi)45°

Direction of the perturbation δb within Span(A).

||ŷ|| = cos(θ)

0.7071

Sensitivity = 1/cos(θ)

1.4142

Click & drag to rotate camera