Retrieve Optimal Loss at the end of Ask-and-Tell

[bacalfa]

Not an issue, but a question. How do I retrieve the optimal loss in an ask-and-tell framework? The examples given here show the use of optimizer.value and optimizer.args. What does it mean to have the second element of the tuple in optimizer.value equal to {}? If I only extract args, I'll have to run another simulation to calculate the loss value. I'm hoping to avoid doing that at the end of the ask-and-tell loop.

[jrapin]

How do I retrieve the optimal loss in an ask-and-tell framework?

If you are using the ask-and-tell interface directly, you have full access to all information so you could store the best one yourself right? It's not feasible this way through the minimize method though, there are ways to do it anyway.
The simplest one is only available in the most recent versions: the loss attribute of the recommended parameter should be filled in most cases (sometimes it is not, because some optimizers recommend a point that was not evaluated). This loss attribute is not yet documented, it's still a bit experimental although I think that will stay that way.

What does it mean to have the second element of the tuple in optimizer.value equal to {}?

I guess you are using ng.p.Instrumentation?
This parameter is specifically designed to handle positional and keyword arguments so that you can call func(*param.args, **param.kwargs).
In this case, param.value holds the value (param.args, param.kwargs). If the dict is empty, it basically means that you are not using any keyword argument, and you could use a ng.p.Tuple directly instead

Is it clearer?

[bacalfa]

Got it, thanks! So something like the following?

best_loss = np.inf
for _ in range(optimizer.budget):
    x = optimizer.ask()
    loss = onemax(*x.args, **x.kwargs)
    optimizer.tell(x, loss)
    if loss < best_loss:
        best_loss = loss
best_x = recommendation = optimizer.provide_recommendation().args

What if there are constraints? Does loss always correspond to a feasible solution?

[jrapin]

This should work indeed.
But as mentioned in most cases you do not need to keep track of the best loss and should be able to get it through
optimizer.provide_recommendation().loss

[jrapin]

What if there are constraints? Does loss always correspond to a feasible solution?

Since you have full control of the loss through the ask and tell interface, loss is whatever you provide. If the candidate does not satisfy the constraints, you should probably send a float("inf")  as loss.
It really depends how the constraints are defined. You should not have invalid parameters anyway if you defined it in the Parameter

[bacalfa]

OK! But what if .loss doesn't exist? Is there away to know? Is it solver-dependent?

[jrapin]

It is optimizer dependent indeed. It's mostly optimizers designed to handle strong noise that do not support this feature