NLA Visualizations

LSE Stability I

Geometry of the Normal Equations

Formulation

Ax=bA\mathbf{x}=\mathbf{b}
y^=A(A∗A)−1A∗b\mathbf{\hat{y}} = A(A^*A)^{-1}A^*\mathbf{b}
κb↦y^=∣∣δy^∣∣/∣∣y^∣∣∣∣δb∣∣/∣∣b∣∣=1cos⁡θ\kappa_{b \mapsto \hat{y}} = \frac{||\delta \hat{y}|| / ||\hat{y}||}{||\delta b|| / ||b||} = \frac{1}{\cos \theta}

Core Idea

This visualization shows how b is projected onto the plane defined by A, and why the sensitivity to perturbations scales like 1/cos(θ).

  • Normalize ||b|| = 1
  • Then ||ŷ|| = cos(θ)
  • As θ approaches 90°, cos(θ) becomes small
  • So the sensitivity 1/cos(θ) blows up

Geometric Controls

45°

Angle between b and its projection ŷ.

45°

Direction of the perturbation δb within Span(A).

||ŷ|| = cos(θ)
0.7071
Sensitivity = 1/cos(θ)
1.4142
Click & drag to rotate camera