I am trying to write a MATLAB code of the Levenberg-Marquardt Algorithm based on Section III in the paper "Training Feedforward networks with the Marquardt Algorithm" by Matin.T.Hagan. So considering I have a set of training inputs and a set of Target inputs, my question is

- How do I calculate the Jacobian Matrix ? I understand that it is a matrix of the partial derivatives of the vector being calculated. But am not sure how do I actually use the Jacobian command that is in MATLAB to compute the jacobian matrix ? .

hoping somone can help me out. Thank you .

I would be very surprised if there is such a command.

The Jacobian is a matrix of partial derivatives of the

function you attempt to optimize. If you have an analytic

expression for the fucntion, compute the partial derivatives

analytically and arrange in a matrix.

If you can't find the analytic expressions for the partial

derivatives, you can come up with all kinds of numerical

estimates. But be aware that not all such estimates are

very good. In which case you might prefer to use a method

that does not require the Jacobian at all.

All this is standard material in an intro class on

numerical optimization. Find a textbook and read.

Rune

MATLAB has two potentially relevant commands. The jacobian function

in the symbolic computation toolbox computes a symbolic Jacobian. The

numjac command uses finite difference approximations to compute the

jacobian of a function at a particular point.

in the symbolic computation toolbox computes a symbolic Jacobian. The

numjac command uses finite difference approximations to compute the

jacobian of a function at a particular point.

================

jacobian() is a command in the Symbolic Math Toolbox, so first off, you have to have that. If you do have that, its usage is documented here, with an example:

http://www.mathworks.com/access/helpdesk/help/toolbox/symbolic/jacobian.html

In the Optimization toolbox help on the L-M method ( http://www.mathworks.com/help/toolbox/optim/ug/brnoybu.html #f204)

it mentions that the "ScaleProblem" option can be used to improve the performance of a poorly scaled non-linear least squares problem by multiplying the Hessian by the lambda parameter.

Basically, I am wondering how this helps exactly. Is it a type of matrix normalization?

Any reference material which described the LM method from an engineering stand point would be greatly appreciated.

Thank you!

it mentions that the "ScaleProblem" option can be used to improve the performance of a poorly scaled non-linear least squares problem by multiplying the Hessian by the lambda parameter.

Basically, I am wondering how this helps exactly. Is it a type of matrix normalization?

Any reference material which described the LM method from an engineering stand point would be greatly appreciated.

Thank you!

No, it's a clever strategy of adaptive (L2) regularization within the (Newton) iterations, applied on non-quadratic objective function to make the convergence more robust. Just google it. http://www.cs.nyu.edu/ ~roweis/notes/lm.pdf

Bruno

Hello Mr.Mano Samuel,

I am also trying to write own code for Levenberg-Marquardt Algorithm for feed forward neural network, i hope you would have programmed it so can you please help me out in programming the same. i am new in using matlab so can you please help me out to program for Levenberg-Marquardt Algorithm

Thanking you

raj

Similar Threads

1. Levenberg-Marquardt (trainlm) and Nash-Sutcliffe efficiency coefficient - MATLAB

2. C-MEX file for Levenberg-Marquardt

3. Help needed for using Levenberg-Marquardt method - MATLAB

4. More help needed for using Levenberg-Marquardt method

Dear all, First of all, a special thank you to Marcelo Marazzi, who have gave me tremendous help in the previous message. Here I would like to further and detail the problem I encounter: As was described in the last letter, "When I use the function lsqnonlin(...) provided by MATLAB to solve the nonlinear least-squares problems, the first thing I need to do is to provide a problem-dependent function that will be passed to lsqnonlin. Here, I have F = SUM(||mi - Mi||^2), where Mi = 1/(h3*vi')*[h1*vi', h2*vi']', where h1, h2, h3, vi are all vectors of the same size, and both mi and Mi are vectors with two elements. Although I have tried millions of times, there is still something wrong with the expression I used to calculate F in the function." Source code %//////////////////////////////////////////////////////// % The maximum likelihood estimate function of homography %//////////////////////////////////////////////////////// function F = HMaxLikeFunc(H, dataFileName0, dataFileNameN) % Initialize: extend the data points' dimension to 3D by assuming Z=1 f0 = load(dataFileName0); fn = load(dataFileNameN); data_0 = zeros(4,3); data_0(:,3) = 1; data_n = zeros(4,2); for i = 1:4, data_0(i,1) = f0(i,1); data_0(i,2) = f0(i,2); data_n(i,1) = fn(i,1); data_n(i,2) = fn(i,2); end k = 1:4; F = data_n(k,:) - 1 ./ (H(3,:) * data_0(k,:)') * [H(1,:) * data_0(k,:)', H(2,:) * data_0(k,:)']; %The very place I call the above function %//////////////////////////////////////////////////////// % Obtain the homography: dataFileName0 is the data from % the model plane while dataFileNameN from the nth image %//////////////////////////////////////////////////////// function H = GetH(dataFileName0, dataFileNameN) % Obtain the initial estimate of the homography H0 = zeros(3,3); H0 = GetHInit(dataFileName0, dataFileNameN); % Calculate the homography by solving the nonlinear least squares % with the Levenberg-Marquardt algorithm H = zeros(3,3); Z = []; H = lsqnonlin(@HMaxLikeFunc, H0, Z, Z, Z, dataFileName0,dataFileNameN); ///////////////////////////////////////////////////////////// And here is the error I get: Error using ==> * Inner matrix dimensions must agree. Error in ==> C:\MATLAB6p5\toolbox\optim\lsqnonlin.m On line 121 ==> [x,Resnorm,FVAL,EXITFLAG,OUTPUT,LAMBDA,JACOB] = ... Error in ==> D:\RESEARCH\MATLAB\TEST\getHomography.m (GetH) On line 136 ==> H = lsqnonlin(@HMaxLikeFunc, H0, Z, Z, Z, dataFileName0,dataFileNameN); Error in ==> D:\RESEARCH\MATLAB\TEST\getHomography.m On line 4 ==> H = GetH(dataFileName0,dataFileNameN); ////////////////////////////////////////////////////////////// In the F expression, the part "1 ./ (H(3,:) * data_0(k,:)')" is supposed to return a scalar, while it actually becomes a vector, and I really can't figure out what's going on here. The above formula is the least-squares problem I need to solve. Hopefully all the information presented here is adequate for you. Thank you very much for your time and patience.

5. Levenberg Marquardt for Fundamental Matrix - MATLAB

6. Levenberg-Marquardt with restricted parameter values

I want to use a Levenberg-Marquardt function on the problem of fitting the parameters of an ellipse to some observed points. I have used a function called lma.m but it sometimes gives imaginary results. I think it is because the formula for the geometric shape is defined only for a certain range of x-values: center coordinate +/- radius. As the center and the radius are parameters and the LM-function tries different values of them, it sometimes happens that some observed points are not within that range. This results in the square root out of a negative number and that seems to put the lma.m off track. It works alright when I choose observed points which lie close above the center of the ellipse. This obviously gives enough margin for the lma.m to change parameter values without landing outside the legal range of x-values. However, such points give large variance in the estimates, since they are almost on a straight line which closely fits many different elliptic shapes. I imagine that the problem could be solved by IMPOSING RESTRICTIONS on the lm-function to not choose values of the center and axis such that any observed points fall outside of the legal range of center +/- radius. Alternatively, maybe the function describing the ellipse could be altered so that it is defined for all x-values, l using the elliptic formula for the interesting range, and some other value for all other values. This must be a fairly general problem and I'm looking for a LM- function where this can be done. It could be m-file or c++, or some more general solution. I use a formula y = f(x &and parameters) for an elliptic arc, since it is only (parts of) the "upper half" of an ellipse which interests me.

7. Levenberg Marquardt - MATLAB

8. More help needed for using Levenberg-Marquardt

Do not start a new thread, stick to the original thread as long as you discuss the same problem. Junfeng Chen wrote: > > > Dear all, > > First of all, a special thank you to Marcelo Marazzi, who have gave > me tremendous help in the previous message. Here I would like to > further and detail the problem I encounter: > > As was described in the last letter, "When I use the function > lsqnonlin(...) provided by MATLAB to solve > the nonlinear least-squares problems, the first thing I need to do > is > to provide a problem-dependent function that will be passed to > lsqnonlin. Here, I have > > F = SUM(||mi - Mi||^2), where Mi = 1/(h3*vi')*[h1*vi', h2*vi']', > where h1, h2, h3, vi are all vectors of the same size, and both mi > and Mi are vectors with two elements. > > Although I have tried millions of times, there is still something > wrong with the expression I used to calculate F in the function." > > Source code > %//////////////////////////////////////////////////////// > % The maximum likelihood estimate function of homography > %//////////////////////////////////////////////////////// > function F = HMaxLikeFunc(H, dataFileName0, dataFileNameN) > % Initialize: extend the data points' dimension to 3D by assuming > Z=1 > > f0 = load(dataFileName0); > fn = load(dataFileNameN); > data_0 = zeros(4,3); > data_0(:,3) = 1; > data_n = zeros(4,2); > for i = 1:4, > data_0(i,1) = f0(i,1); > data_0(i,2) = f0(i,2); > data_n(i,1) = fn(i,1); > data_n(i,2) = fn(i,2); > end > k = 1:4; > F = data_n(k,:) - 1 ./ (H(3,:) * data_0(k,:)') * [H(1,:) * > data_0(k,:)', H(2,:) * data_0(k,:)']; > > %The very place I call the above function > %//////////////////////////////////////////////////////// > % Obtain the homography: dataFileName0 is the data from > % the model plane while dataFileNameN from the nth image > %//////////////////////////////////////////////////////// > function H = GetH(dataFileName0, dataFileNameN) > % Obtain the initial estimate of the homography > H0 = zeros(3,3); > H0 = GetHInit(dataFileName0, dataFileNameN); > % Calculate the homography by solving the nonlinear least squares > % with the Levenberg-Marquardt algorithm > H = zeros(3,3); > Z = []; > H = lsqnonlin(@HMaxLikeFunc, H0, Z, Z, Z, > dataFileName0,dataFileNameN); > > ///////////////////////////////////////////////////////////// > And here is the error I get: > > Error using ==> * > Inner matrix dimensions must agree. > > Error in ==> C:\MATLAB6p5\toolbox\optim\lsqnonlin.m > On line 121 ==> [x,Resnorm,FVAL,EXITFLAG,OUTPUT,LAMBDA,JACOB] = > ... > > Error in ==> D:\RESEARCH\MATLAB\TEST\getHomography.m (GetH) > On line 136 ==> H = lsqnonlin(@HMaxLikeFunc, H0, Z, Z, Z, > dataFileName0,dataFileNameN); > > Error in ==> D:\RESEARCH\MATLAB\TEST\getHomography.m > On line 4 ==> H = GetH(dataFileName0,dataFileNameN); > > ////////////////////////////////////////////////////////////// > > In the F expression, the part "1 ./ (H(3,:) * data_0(k,:)')" is > supposed to return a scalar, while it actually becomes a vector, > and > I really can't figure out what's going on here. > > > The above formula is the least-squares problem I need to solve. > Hopefully all the information presented here is adequate for you. > Thank you very much for your time and patience.