3. Stationary Iterative Methods

Visualizing iterative processes and error decomposition

Formulation

To solve $A\\mathbf{x} = \\mathbf{b}$ , we split $A = M - N$ .

\\mathbf{x} = M^{-1}N\\mathbf{x} + M^{-1}\\mathbf{b}

\\mathbf{x}^{(k)} = M^{-1}N\\mathbf{x}^{(k-1)} + M^{-1}\\mathbf{b}

We define the iteration matrix $G = M^{-1}N$ . The error $\\mathbf{e}^{(k)} = \\mathbf{x}^{(k)} - \\mathbf{x}$ follows the simple recurrence:

\\mathbf{e}^{(k)} = G \\mathbf{e}^{(k-1)}

Geometric Model

G = \begin{bmatrix} 0.8 & 2.0 \\ 0 & 0.5 \end{bmatrix}

Spectral Radius: $\\rho(G) = 0.8 < 1$

Transient Growth

Even though $\\rho(G) < 1$ , the highly non-normal matrix $G$ causes the error norm to oscillate and even grow initially before eventually decaying. Notice how the error vector first converges to the dominant direction before steadily decreasing.

Controls

θInitial Guess

\\mathbf{x}^{(0)}

135°

Iteration: 0

Solution Space (x)

Error Space (e)