Printing from the function to be minimized in FindMinimum

I know that there is an option EvaluationMonitor for various numerical computation and plotting functions.

But I am curious, why a simple Print statement in the function to be minimized does not print. Consider this example:

Clear[x, y, z];
f[{x_, y_, z_}] := (Print[
“in function f called with these arguments: “, {x, y, z}];
1 + x^2 + y^2 + z^2);
FindMinimum[f[{x, y, z}], {{x, 1}, {y, 22}, {z, -3}}]

in function f called with these arguments: {x,y,z}

{1., {x -> 0., y -> 0., z -> 0.}}

The result is OK. But I would have expected the first print output to have been

in function f called with these arguments: {1,22,-3}

Did I get that completely wrong?
Or is FindMinimum using some fancy Compiled feature defeating the Print statement?

=================

3

May be because of this? FindMinimum first localizes the values of all variables, then evaluates f with the variables being symbolic, and then repeatedly evaluates the result numerically so you are looking at first call of the function? You could use StepMonitor to see the steps.
– Nasser
Sep 10 at 7:20

Mathematica does not need to evaluate f numerically to find the minimum. It can do it analytically. Try adding a ?NumericQ to force numeric evaluation.
Sep 10 at 7:22

@mikado I thought about this first, but the result returned is numerical (i.e. real). it has decimals on it. So it must have used numerical method.
– Nasser
Sep 10 at 7:24

1

@nasser Mathematica will do symbolic analysis before attempting to solve the problem numerically and may then minimise the simplified expression (rather than the original f)
Sep 10 at 7:30

=================

1

=================

The essence of the problem is already explained by mikado in the comment:

Mathematica will do symbolic analysis before attempting to solve the
problem numerically and may then minimize the simplified expression
(rather than the original f)

One solution is to prevent symbolic analysis of the function by turning it into a black-box function:

Clear[f, x, y, z];
f[{x_?NumericQ, y_?NumericQ, z_?NumericQ}] := (Print[
“in function f called with these arguments: “, {x, y, z}];
1 + x^2 + y^2 + z^2);
FindMinimum[f[{x, y, z}], {{x, 1}, {y, 22}, {z, -3}}]

But it is recommended don’t do this when it isn’t strictly necessary, and instead use EvaluationMonitor for obtaining the evaluation points:

Clear[x, y, z, f];
f[{x_, y_, z_}] := (Print[“in function f called with these arguments: “, {x, y, z}];
1 + x^2 + y^2 + z^2);
FindMinimum[f[{x, y, z}], {{x, 1}, {y, 22}, {z, -3}},
EvaluationMonitor :> Print[{x, y, z}]]

If you compare the evaluation points taken with the two above approaches, you will see that in the second case the minimum is found in 2 evaluations of the objective functions (and it is exact minimum!), while in the first case much more points are taken and the minimum isn’t exact. The explanation is given above by mikado: symbolic analysis may allow to get the minimum without searching at all. See also the comment by Michael E2 under this answer.

Alexey, after Clear[f, x, y, z]; f[{x_NumericQ, y_NumericQ, z_NumericQ}] := (Print[“in function f called with these arguments: “, {x, y, z}]; 1 + x^2 + y^2 + z^2); FindMinimum claims “The function value f[{1.,22.,-3.}] is not a real number at {x,y,z} = {1.,22.,-3.}.” The function f applied to these arguments evaluates to itself! Did I get something wrong with your answer?
Sep 10 at 11:18

Alexey, after removing the NumericQ after the parameters and adding EvaluationMonitor according to your hint, I see tht in this special case the minimum really is found in a single step. If I change my example to 1 + x (x – 1)^2 + (y + 2)^2 + (z – 3.3)^2), it takes three steps according to EvaluationMonitor, but Print just leaves a single “in function f called with these arguments: {x,y,z}”. Mikado’s argument seems to be right.
Sep 10 at 11:26

@AdalbertHanßen To your first comment: you have removed the ? before NumericQ – that’s why it doesn’t work. Please copy the code from my answer without editing, it works just fine!
– Alexey Popkov
Sep 10 at 11:34

1

@AdalbertHanßen When f[{x, y, z}] evaluates to an expression that can be symbolically differentiated, FindMinimum will use the symbolic derivative for Newton’s method. For a quadratic function, the derivative is linear, and the problem turns into an exact, one-step, linear algebra problem. When f[{x, y, z}] remains unevalauted (the ?NumericQ case), then it will numerically approximate the gradient, and the slight numerical error may lead to extra steps to converge.
– Michael E2
Sep 10 at 16:17

Tanks to all. This was a good learning lesson. Yes, with the question mark before the NumericQ, i.e. f[{x_?NumericQ, y_?NumericQ, z_?NumericQ}] := (Print[“in function f called with these arguments: “, {x, y, z}]; 1 + x (x – 1)^2 + (y + 2)^2 + (z – 3.3)^2); the print commands work. The same applies for ListPlot which sometimes is helpfull to see how numerical approximations improve the result.