Iterative or Indirect method is a method of considering an approximation to the true solution and, if successful, obtain better and better approximations from a computational cycle repeated as often as may be necessary for achieving a required accuracy.
The iterative methods shall be applied if matrices have large main diagonal entries or if a large system is sparse (having many zero coefficients).
Gauss Siedel Iteration Method
A = I + L + U
where
I - n x n unit matrix
L - Lower triangle matrix with zero main diagonals
U - Upper triangle matrix with zero main diagonals
Substituting the A in Ax = b
(I + L + U)x = b
x = b - Lx - Ux
The each component is replaced an approximation of a component by a corresponding new approximation as soon as the latter has been computed.
x(m+1) = b - Lx(m+1) - Ux(m)
A sufficient condition for convergence is
||C|| < 1 where ||C|| is matrix norm
The Gauss–Seidel iteration belongs to a class of methods often called relaxation methods. The method is also referred as successive corrections.
Jacobi Iteration Method
This method is referred as Simultaneous corrections method as no component of an approximation is used until all the components have been computed.
x(m+1) = b + (I - A)x(m)
This method converges for every choice of x if and only if the spectral radius of I - A is less than 1.