Does FindFit use symbolic differentiation when optimizing by least squares a non-linear function?

=================

=================

2 Answers

2

=================

FindFit does perform symbolic differentiation by default. It has the option to use finite differences instead via the option Gradient -> “FiniteDifference”.

There is worked example in the documentation here.

Yes it does. You can see this by using a function that prints the argument passed to the function evaluated in the FindFit procedure.

For example you could use this little setup:

data = Table[{x, 2.5*Exp[x]}, {x, 0, 1, 0.1}];

f[a_, x_] := Block[{}, Print[x]; a*Exp[x]]

FindFit[data, f[a, x], {a}, x]

What you will notice, when you execute the code, is that in addition to the correct result a = 2.5 the Print routine yields the output x. This means that the function f has been evaluated symbolically by Mathematica, otherwise you would have gotten a list of numerical values, which would correspond to points where f would have been evaluated numerically.

1

Since FindFit is not HoldAll, before FindFit is executed, f[a, x] is evaluated and the result is passed to FindFit. I’m afraid this example does not show what FindFit does with the expression it gets. — BTW, I think you’re probably right about FindFit differentiating, but I don’t know how to prove it as yet.

– Michael E2

Feb 3 ’15 at 21:58

Ok, I did not realise that. Thanks for the info. I will leave this answer up for now, because I think your comment is interesting, but I am glad to remove it. Sorry for the misinformation.

– Wizard

Feb 3 ’15 at 22:08

I’m not sure if derivatives are used, but you get slight differences using a symbolic function and a numeric black box. For example, f[a_, x_] := a*Exp[x]; g[a_?NumericQ, x_?NumericQ] := a*Exp[x]; yield values of a that differ by 10^15. The symbolic one is about ten times faster on your example. It suggests derivatives, imo, but perhaps it’s not conclusive.

– Michael E2

Feb 3 ’15 at 22:21

Ok, now this example convinces me that Mathematica does not use symbolic differentiation: data = Table[{x, Pi*x}, {x, 0, 1, 1}]; FindFit[data, a*x, {a}, x] yields {a -> 3.14159} and not {a -> Pi}, which should be the case if Mathematica would differentiate symbolically.

– Wizard

Feb 6 ’15 at 22:14

1

Pi == a /. FindFit[data, a*x, {a}, x] evaluates to True, though, and working precision is MachinePrecision. FindFit cannot (or will not) return exact, infinite precision answers. The result is the same as Pi to 150 digits if I use WorkingPrecision -> 150.

– Michael E2

Feb 6 ’15 at 22:42