romtools.workflows.inverse.mf_eki_drivers#

Multifidelity ensemble Kalman inversion drivers.

This module extends the single-fidelity EKI workflow with an adaptive low-fidelity surrogate, typically a ROM, to reduce the cost of the inverse solve while preserving high-fidelity correction steps. The implementation uses one high-fidelity ensemble and an additional ROM-only ensemble so that the Kalman update benefits from a larger effective sample set.

Theory

run_mf_eki targets the same observation-matching objective as romtools.workflows.inverse.eki_drivers.run_eki(), but partitions the ensemble into a high-fidelity set \(\mathcal{S}_0\) and an auxiliary ROM-only set \(\mathcal{S}_1\). The high-fidelity model is evaluated on \(\mathcal{S}_0\), while a correlated surrogate is evaluated on both sample sets.

Using the same anomaly notation as single-fidelity EKI, the multifidelity update combines:

  • High-fidelity QoI anomalies from the FOM evaluations on \(\mathcal{S}_0\).

  • Low-fidelity anomalies from ROM evaluations on both \(\mathcal{S}_0\) and \(\mathcal{S}_1\).

The resulting update has a control-variate flavor: the FOM-corrected ROM statistics are used to approximate the cross-covariance and QoI covariance that appear in the Kalman solve, while only the smaller subset of samples requires expensive FOM evaluations.

Adaptive ROM Refresh

The ROM is retrained from iteration data generated by accepted FOM steps. Each iteration measures the relative discrepancy between ROM and FOM QoIs on the shared high-fidelity sample set. When that discrepancy exceeds rom_tolerance, the surrogate is rebuilt from the accumulated training directories, parameters, and QoIs so that the low-fidelity correction remains useful as the ensemble moves.

Step Acceptance

As in run_eki(), the routine tests a scaled update and accepts it only when the mean high-fidelity observation error norm decreases sufficiently. The same growth/decay step-size logic is used, but each accepted step may also update the surrogate state and the stored ROM training history.

Relation to run_eki

If rom_extra_ensemble_size is zero, or if the ROM correction is perfectly collapsed onto the FOM sample set, the algorithm approaches the single-fidelity EKI workflow. The multifidelity path mainly changes how the Kalman statistics are estimated and when low-fidelity models are retrained.

Functions

compute_mf_eki_update(parameter_sample_sets, ...)

Compute the update matrices for the MF-EKI algorithm.

mf_eki_with_auto_rom(model, parameter_space, ...)

Wrapper around run_mf_eki that selects a default ROM surrogate by rom_type.

run_mf_eki(model, rom_model_builder, ...[, ...])

Run a multi-fidelity ensemble Kalman inversion (MF-EKI) workflow.

Classes

GaussianProcessQoiModelBuilderWithTrainingData(...)

romtools.workflows.inverse.mf_eki_drivers.compute_mf_eki_update(parameter_sample_sets, fom_results_for_sample_sets, rom_results_for_sample_sets, observations_covariance, regularization_parameter)[source]#

Compute the update matrices for the MF-EKI algorithm.

romtools.workflows.inverse.mf_eki_drivers.mf_eki_with_auto_rom(model, parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, absolute_eki_directory='/home/runner/work/rom-tools-and-workflows/rom-tools-and-workflows/work/', fom_ensemble_size=10, rom_extra_ensemble_size=30, rom_tolerance=0.005, use_updated_rom_in_update_on_rebuild=False, initial_step_size=0.1, regularization_parameter=0.0001, step_size_growth_factor=1.25, step_size_decay_factor=2.0, max_step_size_decrease_trys=5, relaxation_parameter=1.05, error_norm_tolerance=1e-05, delta_params_tolerance=0.0001, max_rom_training_history=1, max_iterations=50, random_seed=1, fom_evaluation_concurrency=1, rom_evaluation_concurrency=1, restart_file=None, rom_type='gp', rom_args=None)[source]#

Wrapper around run_mf_eki that selects a default ROM surrogate by rom_type.

Parameters:

  • model (QoiModel)

  • parameter_space (ParameterSpace)

  • observations (ndarray)

  • observations_covariance (ndarray)

  • parameter_mins (ndarray)

  • parameter_maxes (ndarray)

  • absolute_eki_directory (str)

  • fom_ensemble_size (int)

  • rom_tolerance (float)

  • use_updated_rom_in_update_on_rebuild (bool)

  • initial_step_size (float)

  • regularization_parameter (float)

  • step_size_growth_factor (float)

  • step_size_decay_factor (float)

  • max_step_size_decrease_trys (int)

  • relaxation_parameter (float)

  • error_norm_tolerance (float)

  • delta_params_tolerance (float)

  • max_rom_training_history (int)

  • max_iterations (int)

  • random_seed (int)

  • fom_evaluation_concurrency (int)

  • rom_evaluation_concurrency (int)

  • restart_file (str)

  • rom_type (str)

  • rom_args (dict | None)

romtools.workflows.inverse.mf_eki_drivers.run_mf_eki(model, rom_model_builder, parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, absolute_eki_directory='/home/runner/work/rom-tools-and-workflows/rom-tools-and-workflows/work/', fom_ensemble_size=10, rom_extra_ensemble_size=30, rom_tolerance=0.005, use_updated_rom_in_update_on_rebuild=True, initial_step_size=0.1, regularization_parameter=0.0001, step_size_growth_factor=1.25, step_size_decay_factor=2.0, max_step_size_decrease_trys=5, relaxation_parameter=1.05, error_norm_tolerance=1e-05, delta_params_tolerance=0.0001, max_rom_training_history=1, max_iterations=50, random_seed=1, fom_evaluation_concurrency=1, rom_evaluation_concurrency=1, restart_file=None)[source]#

Run a multi-fidelity ensemble Kalman inversion (MF-EKI) workflow.

The workflow evaluates the FOM on a smaller ensemble, augments those data with a larger ROM-only ensemble, and forms a multifidelity Kalman update. The ROM is rebuilt from accepted iteration data when its shared-sample QoI error exceeds rom_tolerance.

Parameters:

  • model (QoiModel) – High-fidelity QoiModel used to define the target inverse problem.

  • rom_model_builder (QoiModelBuilderWithTrainingData) – Builder that constructs surrogate QoI models from accumulated training directories and QoI data.

  • parameter_space (ParameterSpace) – ParameterSpace used to draw the initial combined ensemble when restart_file is not provided.

  • observations (ndarray) – Observed QoI vector \(y\).

  • observations_covariance (ndarray) – Observation covariance matrix \(\Gamma_y\) used in the Kalman solve.

  • parameter_mins (ndarray) – Optional lower bounds applied to sampled and updated parameters.

  • parameter_maxes (ndarray) – Optional upper bounds applied to sampled and updated parameters.

  • absolute_eki_directory (str) – Absolute path to the working directory. Each iteration stores FOM and ROM evaluations in iteration-specific subdirectories below this path.

  • fom_ensemble_size (int) – Number of ensemble members evaluated with the high-fidelity model on each iteration.

  • rom_extra_ensemble_size – Number of additional ROM-only ensemble members used to improve multifidelity covariance estimates.

  • rom_tolerance (float) – Relative QoI error threshold on the shared FOM/ROM sample set used to trigger surrogate rebuilding.

  • use_updated_rom_in_update_on_rebuild (bool) – When True, use the updated ROM in the update for that step. When False, use the previous ROM in the update for that step.

  • initial_step_size (float) – Initial multiplier applied to the multifidelity Kalman update directions.

  • regularization_parameter (float) – Tikhonov regularization added to the QoI covariance solve.

  • step_size_growth_factor (float) – Factor used to increase the step size after an accepted iteration.

  • step_size_decay_factor (float) – Factor used to decrease the step size after a rejected trial iteration.

  • max_step_size_decrease_trys (int) – Maximum number of consecutive rejected trial steps before the routine exits.

  • relaxation_parameter (float) – Acceptance threshold on the mean FOM observation-space error norm.

  • error_norm_tolerance (float) – Stop when the mean FOM observation-space error norm falls below this value.

  • delta_params_tolerance (float) – Stop when the norm of the proposed high-fidelity ensemble update falls below this value.

  • max_rom_training_history (int) – Maximum number of accepted iterations whose training data are retained for ROM rebuilding.

  • max_iterations (int) – Maximum number of EKI iterations.

  • random_seed (int) – RNG seed used for the initial ensemble draw.

  • fom_evaluation_concurrency (int) – Number of concurrent FOM evaluations used by each iteration.

  • rom_evaluation_concurrency (int) – Number of concurrent ROM evaluations used by each iteration.

  • restart_file (str) – Optional .npz restart file produced by a prior MF-EKI run. When set, the saved FOM/ROM sample sets, surrogate training history, and step size are restored.

Returns:

Tuple (fom_parameter_samples, fom_qois) for the final accepted high-fidelity sample set.