Suppose you want to fit m data points to your model y = α xβ + e.
Your m data points are (x1,y1),...,(xm,ym). And your initial guess for the parameters is α0,β0,e0.
Let λ = (α β e
), a 3x1 matrix holding your parameters.
Define d Ui = yi - (α xβ i
+ e) as the residue values. The linearized estimate of these values isd U i = ∂y(xi) ∂α d α + ∂y(xi) ∂β d β + ∂y(xi) ∂e d e
In our model, the partial derivatives are easily computed,∂y ∂α = x β , ∂y ∂β = x β log x , ∂y ∂e = 1
Then we can write this in matrix form,
If we let A be the lefthand side matrix and d λ the righthand side, thend U = A d λ
Multiplying both sides by AT using linear algebra we can solve the following equation for d λ,a d λ = b
where a≡ATA and b≡ATd U.
This small change d λ is added to our original guess λ0. With that we recompute d U, and solve the linear algebra problem again. The process continues until d λ<ε, where ε is a limit we wish to reach.