Difference between maxiter and maxfun parameters in function fmin_l_bfgs_b

Question

In the help of function:

maxfun : int
Maximum number of function evaluations.
maxiter : int
Maximum number of iterations.

But what is the maximum number of iterations and the number of function evaluation?

For me, the number of iterations was directly related to the number of function evaluations, but they give different results. How the number of function assessments is calculated?

Answer 1

I may be wrong, but when you optimize a function, you may have to evaluate the optimal value of outputs by performing several 'evaluations' each 'iteration'.

In an interaction of the algorithm L-BFGS-B may have to perform (and usually performs) multiple evaluations/value comparisons to arrive at the optimal value for the parameters.

The parameter maxiter controls the number of iterations the algorithm can perform.

Already the parameter maxfun, refers to the number of evaluations/comparisons that the algorithm can make, at each iteration, about the objective function.

Or in other words:

The optimization process of a function may require the optimizer to evaluate the value of the objective function several times with different sets of parameters.

The maximum number of interactions that the optimizer can run is controlled by the parameter maxiter.

The maximum number of times the objective function can be evaluated/tested, with different sets of parameters, is controlled by the parameter maxfun.

For example:

We know that the execution of the optimization algorithm in a given objective function will take 10 interactions, with each interaction requiring an evaluation of the values obtained by 5 times, until the optimal value is reached.

We then have to maxiter shall be equal to 10 and maxfun equal to 10*5 = 50.

Otherwise the function will not run the minimum number of times to have its parameters optimized.