MATLAB >> MLE vs. FMINSEARCH

by OR Stats » Wed, 28 May 2008 18:15:04 GMT

Hello All:

What is the difference between using the MLE() function
versus using FMINSEARCH()? Is there a consensus on which is
better for parameter estimation?

MATLAB >> MLE vs. FMINSEARCH

by Simon Preston » Wed, 28 May 2008 19:03:05 GMT



<g1jbb8$3m1$ XXXX@XXXXX.COM >...

A quick look at the code of mle shows that it uses fminsearch:
edit mle

Best wishes, S

MATLAB >> MLE vs. FMINSEARCH

by John D'Errico » Wed, 28 May 2008 19:23:02 GMT


<g1jbb8$3m1$ XXXX@XXXXX.COM >...

Whats the difference? They are different
tools, designed for different purposes,
using different algorithms.

Lets see. The mle code is in the stats
toolbox. It is a trust region optimizer,
that needs the gradient of its objective
function, which it computes for you. At
least this is how one of the options
works based on my quick examination
of the gamfit code. I assume they all
use a similar optimization. The mle
code has the knowledge built inside it
to choose its own starting values, often
a difficult part of any optimization.

The fminsearch code is a generic tool,
a Nelder/Meade optimizer that is not
specific to any objective function type.
It is a polytope method (some call it
a simplex method, but that tends to
confuse other people who think it a
linear programming tool.) You must
supply your own objective function, in
this case for the likelihood function,
probably to minimize the negative log
of the likelihood. You must supply your
own starting values for fminsearch.

Which is better? The mle code is more
sophisticated. It is designed to solve
mle problems. Is it better? Sometimes.
It is more sophisticated. Is the more
sophisticated tool always better? Who
knows? What are your skills? Perhaps
you have some inside knowledge that
provides you with the nearly perfect
set of starting values for fminsearch.

On the other hand, if you don't know
the difference between these tools and
could not figure it out yourself, I'd trust
the person who coded the mle code to
have done their job well.

John

MATLAB >> MLE vs. FMINSEARCH

by John D'Errico » Wed, 28 May 2008 19:26:02 GMT

"Simon Preston" <preston.simon+ XXXX@XXXXX.COM > wrote in
message <g1je59$rum$ XXXX@XXXXX.COM >...


Not that I saw. But I only looked at one of the
modules. Different modules may use different
basic optimizers.

Regardless, there are still huge differences,
since you must write your own objective
function for fminsearch. You must choose
your own starting values for fminsearch. mle
does all of these things for you, apparently
even choosing the underlying optimizer.

John

MATLAB >> MLE vs. FMINSEARCH

by John D'Errico » Wed, 28 May 2008 19:28:02 GMT

"Simon Preston" <preston.simon+ XXXX@XXXXX.COM > wrote in
message <g1je59$rum$ XXXX@XXXXX.COM >...


Looking more closely, you can choose
between fminsearch and fmincon as the
basic optimizer. My other comment still
apply.

John

MATLAB >> MLE vs. FMINSEARCH

by Simon Preston » Wed, 28 May 2008 19:48:02 GMT


Agreed! I didn't mean to give the impression that the two
functions were the same - only that mle has at its core a
call to fminsearch (or similar basic optimizer).

Best wishes, S

MATLAB >> MLE vs. FMINSEARCH

by Peter Perkins » Wed, 28 May 2008 21:51:13 GMT


John's right about that, mostly.

MLE allows you to specify one of about two dozen "canned" univariate
distributions to fit, and it calls things like GAMFIT or NORMFIT to do the work.
Those functions are written specifically for a particular distribution, and so
they take advantage of whatever they can in terms of starting values and so on.
Sometimes those functions compute the MLEs explicitly (like NORMFIT, when
there's no censoring), sometimes they use FMINSEARCH, sometimes they use a trust
region optimizer. Sometimes they find a zero of the log-likelihood gradient.
Depends on the distribution, we've tried to use the method that works best in
terms of actually finding the estimates. They all do maximum likelihood (except
for trivialities like NORMFIT using the unbiased estimator for sigma^2, not the
MLE).

On the other hand, MLE also allows you to provide a function or functions (in
the firm of function handles) that define the log-likelihood of a distribution
that you define. In that case, MLE uses FMINSEARCH, but you can also tell it to
use FMINCON if you have the Optimization Toolbox. FMINCON is usually more
efficient, but FMINSEARCH can be more robust in some odd cases. You also have
to provide a starting value.

Hope this helps.

- Peter Perkins
The MathWorks, Inc.

MATLAB >> MLE vs. FMINSEARCH

by OR Stats » Thu, 29 May 2008 19:40:19 GMT

Peter Perkins < XXXX@XXXXX.COM > wrote
in message <g1jo0h$a0e$ XXXX@XXXXX.COM >...

univariate
NORMFIT to do the work.
particular distribution, and so
starting values and so on.
(like NORMFIT, when
sometimes they use a trust
log-likelihood gradient.
that works best in
maximum likelihood (except
for sigma^2, not the
function or functions (in
log-likelihood of a distribution
you can also tell it to
is usually more
cases. You also have
Thanks to all. This would seem to suggest that we can
optimize any function that we define; for the case of
parameter estimation, our objective can be for example a
inverse fast fourier transform IFFT. In other words, we can
simply define a function for MLE/FMINSEARCH(), in which that
function is the [log of] IFFT? In this case, MLE FMINSEARCH
AND FMINCON would all be customizable for the search
algorithm that we prefer?

MATLAB >> MLE vs. FMINSEARCH

by Peter Perkins » Thu, 29 May 2008 22:06:49 GMT


I don't know what that means. Certainly, FMINSEARCH can optimize any objective
that you can express as an M function that accepts a vector of parameters and
returns an objective value.

But (the function) MLE is pretty specific in what it can do: you have to
provide a vector of data, and you have to either specify the name of a
probability distribution, or provide something like a log-likelihood function.
Anything more general than that, and you might as well use FMINSEARCH or FMINCON
directly.

Similar Threads

1. fminsearch mle parameters - MATLAB

2. matlab code for mle using fminsearch

Hi~^^

I'm a kind of beginner of Matlab, but I may use it a lot in
statistical analysis. I've ever used SAS for statistical analysis.

I'd like to know what would be the matlab code for estimating beta in
maximum likelyhood by using fminsearch function of matlab.

Can I have some idea, please?

Then, I'd wait for your replay. Thank you!

3. fminsearch in script vs. function - MATLAB

4. fminsearch vs fmincon

Hello all,

i am currently using the function fminsearch shown below to fit a 2d
gaussian (IntermPic) over X,Y space using the initial parameters
(Param).

 Fitted = fminsearch(@(Param) GausResid(Param, X, Y,
IntermPic),Param);

I was wondering if anyone can tell me how to set boundary conditions
to Param using such an approach - is fmincon needed?

cheers
Jon

5. fminsearch vs. fminunc - MATLAB

6. AMD vs P4 vs Pentium M & xP vs Win2000

7. print vs saveas vs file/save as - MATLAB

8. Matlab vs C++ vs C#

Which of these should a person interested in financial modeling learn?