Beta-cooling schedule#
One common example is the beta cooling schedule that consists in defining an objective function in the form of
where \(\phi_\text{d}(\mathbf{m})\) is the data misfit term, \(\phi_\text{m}(\mathbf{m})\) is the model norm, and \(\beta_0\) is an initial value for the trade-off parameter, which is usually chosen to be a high value. We proceed to minimize the objective function to obtain an inverted model, and then cool down the trade-off parameter by dividing it by a constant factor. Then we carry another minimization step, and another cooling step, and so on until a certain condition is met, usually that the data is fitted up to their uncertainties.
Beta-cooling: simple#
We can achieve that in the new inversion framework with just a for loop.
For example, consider a DataMisfit object that uses a
LinearRegressor simulation, like the one we defined in
Objective function:
data_misfit
Let’s define a SimpleSmallness regularization:
smallness = ii.SimpleSmallness(data_misfit.n_params)
smallness
And create our objective function with a high value for our starting trade-off parameter beta:
beta_0 = 200.0
regularization = beta_0 * smallness
phi = data_misfit + regularization
phi
Note
A high value for the trade-off parameter depends on the type of problem we are facing. For one problem a value of 200.0 might be high enough, but for other it might need to be several order of magnitudes larger.
Since we can access the value of the trade-off parameter through regularization.multiplier, we can also change it.
Let’s use this capability to implement a simple beta-cooling schedule:
# Define an initial model
model = np.zeros(data_misfit.n_params)
# Perform a beta cooling schedule
while True:
# Minimize the objective function using a conjugate gradient minimizer.
# Overwrite the model variable with the minimized model.
model = ii.minimize.conjugate_gradient(phi, model)
# Define a stopping criterion:
# finish the inversion if the chi factor is below 1.
if data_misfit.chi_factor(model) <= 1.0:
break
# Cool down beta using a factor of 2
regularization.multiplier /= 2.0
The final inverted model will be stored in the model variable:
inverted_model = model.copy()
inverted_model
array([-5.20377095, 1.73075601, -2.26585317, 4.04296808, 1.56254929])
We can also see the final value of the trade-off parameter by displaying the objective function:
phi
Beta-cooling: with custom objects#
The new framework ships some objects that would make writing this inversion in a slicker way.
First, instead of defining our tradeoff parameter as a float and manually cooling it down, we can use a inversion_ideas.hyperparams.CooledMultiplier.
This object acts like a float, but it has an
update() method that can automatically perform the cooling step.
Following the previous example, we can define our objective function as:
# Define trade-off parameter as a hyperparameter object.
# Set its initial value to 200.0 and the factor that will be used to cool it down to 2.0.
beta = ii.hyperparams.CooledMultiplier(200.0, cooling_factor=2.0)
# Use this beta as any other number
phi = data_misfit + beta * smallness
phi
See also
Learn more about custom hyperparameter objects in Hyperparameters.
We can also define a stopping criterion to define when our inversion should stop.
In this case we can make use of inversion_ideas.conditions.ChiTarget to check if the \(\chi\) factor for the data_misfit falls below a desired threshold:
# Define a stopping criterion
chi_target = ii.conditions.ChiTarget(data_misfit, chi_target=1.0)
And finally re-write our inversion in a slicker way:
# Define an initial model
model = np.zeros(data_misfit.n_params)
# Perform a beta cooling schedule
while True:
# Minimize the objective function
model = ii.minimize.conjugate_gradient(phi, model)
# Finish the inversion if the stopping criterion is met
if chi_target(model):
break
# Cool down beta
beta.update()
# Define inverted model as last model in the iterations
inverted_model = model.copy()
inverted_model
array([-5.20377095, 1.73075601, -2.26585317, 4.04296808, 1.56254929])
We can also explore the custom beta hyperparameter and see how its value changed:
print(beta)
CooledMultiplier(25.0)
Hint
We can easily insert any custom code in between iterations. For example we could print information about the inversion process:
# Define objective function
beta = ii.hyperparams.CooledMultiplier(200.0, cooling_factor=2.0)
phi = data_misfit + beta * smallness
# Define an initial model
model = np.zeros(data_misfit.n_params)
# Perform a beta cooling schedule
iteration = 0
while True:
# Minimize the objective function
model = ii.minimize.conjugate_gradient(phi, model)
# Print information
iteration += 1
print(f"Iteration: {iteration:2d}")
print("-------------")
print(f"- model: {model}")
print(f"- beta: {beta}")
print(f"- chi factor: {data_misfit.chi_factor(model)}")
print("")
# Finish the inversion if the stopping criterion is met
if chi_target(model):
break
# Cool down beta
beta.update()
Iteration: 1
-------------
- model: [-5.16015904 1.69081292 -2.25382391 4.00541648 1.54686684]
- beta: 200.0
- chi factor: 37.723368524172706
Iteration: 2
-------------
- model: [-5.18497886 1.71345053 -2.26068489 4.02679262 1.55576705]
- beta: 100.0
- chi factor: 9.546176168694283
Iteration: 3
-------------
- model: [-5.19748988 1.72495586 -2.26412826 4.03756252 1.56027817]
- beta: 50.0
- chi factor: 2.4011920329085124
Iteration: 4
-------------
- model: [-5.20377095 1.73075601 -2.26585317 4.04296808 1.56254929]
- beta: 25.0
- chi factor: 0.6021437352887772