lsqnonlin + Tikhonov regularization

[murof htam]

Hey All,

It seems lot of experts here. I really appreciate if anyone can help me to answer my questions. It has bothered me for long time.

I am using LSQNONLIN to solve a non-linear least squares optimization problem. The input data were sampled from different regions of an image each time, these data are all around a center point. Their values are Gaussian distribution, but a little bit shift. The objective is to find the center point. I think, theoretically, each time LSQNONLIN should converge at the same point (the peak one), because the sample data are all around that. However, I got different results each time. It is unstable, even the Standard Deviation is very small.

So my questions are:
1. Is the problem caused by ill-conditioned at the solution point or other reasons?

2. Can the Tikhonov regularization help to fix that?

3. How to implement the regularization for LSQNONLIN in matlab? It is also great if you can refer me to other sources.

Many thanks in advance!

Murof

[John D'Errico]

In article

Post by murof htam
It seems lot of experts here. I really appreciate if anyone can help me to
answer my questions. It has bothered me for long time.
I am using LSQNONLIN to solve a non-linear least squares optimization
problem. The input data were sampled from different regions of an image each
time, these data are all around a center point. Their values are Gaussian
distribution, but a little bit shift. The objective is to find the center
point. I think, theoretically, each time LSQNONLIN should converge at the
same point (the peak one), because the sample data are all around that.
However, I got different results each time. It is unstable, even the Standard
Deviation is very small.
1. Is the problem caused by ill-conditioned at the solution point or other reasons?
2. Can the Tikhonov regularization help to fix that?
3. How to implement the regularization for LSQNONLIN in matlab? It is also
great if you can refer me to other sources.

I give an example of a Tikhonov regularization (though
I did not call it that) in my optimization tips and
tricks doc. Find it here. (Beware the line wrapped URL.)

http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=
8553&objectType=FILE

Section 4, where I discuss ridge regression, has an
example where I build a (very) simple spline model,
then append a regularizer based on the curvature. In
effect, its all done by adding extra data points to
the problem.

Your regularizer would work the same way. Add extra
"data" points for lsqnonlin to work on.

I can't really answer your first question, because
I don't have enough information.

Will a regularizer help to fix it? Clearly a regularizer
will bias the result towards what you want to see.
But since I don't really know what was the cause of
your problem, I can't say how to fix it.

HTH,
John D'Errico

[murof htam]

Really appreciated, John! Just sent you an email, I didn't post it here, cuz it is too long.