Ensemble Kalman Inversion (EKI)#

The EKI workflows provide single- and multi-fidelity ensemble Kalman inversion utilities for parameter inference problems. Both workflows are derivative-free with respect to the forward model, support concurrent sample evaluation, and write restart files that can be used to resume long-running solves.

Single-fidelity EKI updates a parameter ensemble with Kalman-style covariance estimates formed from forward-model QoIs. Multi-fidelity EKI preserves the same high-fidelity inverse target while augmenting the update with a surrogate-based control-variate correction built from shared FOM/ROM samples and additional ROM-only samples.

For more detailed algorithmic background, see the function-level API documentation for the single- and multi-fidelity drivers below.

Single-fidelity EKI#

romtools.workflows.inverse.eki_drivers.run_eki(model, parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, absolute_eki_directory='/home/runner/work/rom-tools-and-workflows/rom-tools-and-workflows/work/', ensemble_size=30, initial_step_size=0.1, regularization_parameter=0.0001, step_size_growth_factor=1.25, step_size_decay_factor=2.0, max_step_size_decrease_trys=5, relaxation_parameter=1.05, error_norm_tolerance=1e-05, delta_params_tolerance=1e-06, max_iterations=50, random_seed=1, evaluation_concurrency=1, restart_file=None)[source]

Run a single-fidelity ensemble Kalman inversion (EKI) workflow.

The routine draws or restores a parameter ensemble, evaluates the forward model for every ensemble member, forms the ensemble Kalman correction, and accepts or rejects trial updates based on the reduction in mean QoI error. The workflow is fully derivative-free with respect to model.

Parameters:

  • model (QoiModel) – QoiModel to evaluate at ensemble samples.

  • parameter_space (ParameterSpace) – ParameterSpace used to draw the initial ensemble when restart_file is not provided.

  • observations (ndarray) – Observed QoI vector \(y\).

  • observations_covariance (ndarray) – Observation covariance matrix \(\Gamma_y\) used in the Kalman solve.

  • parameter_mins (ndarray) – Optional lower bounds applied to sampled and updated parameters.

  • parameter_maxes (ndarray) – Optional upper bounds applied to sampled and updated parameters.

  • absolute_eki_directory (str) – Absolute path to the working directory. Each accepted or tested iteration writes into iteration_<k>/run_* subdirectories under this path.

  • ensemble_size (int) – Number of ensemble members used in the EKI update.

  • initial_step_size (float) – Initial multiplier applied to the computed Kalman update directions.

  • regularization_parameter (float) – Tikhonov regularization added to the QoI covariance solve for numerical stability.

  • step_size_growth_factor (float) – Factor used to increase the step size after an accepted iteration.

  • step_size_decay_factor (float) – Factor used to decrease the step size after a rejected trial iteration.

  • max_step_size_decrease_trys (int) – Maximum number of consecutive rejected trial steps before the routine exits.

  • relaxation_parameter (float) – Acceptance threshold on the mean error norm. A trial step is accepted when the new norm is below relaxation_parameter * current_error_norm.

  • error_norm_tolerance (float) – Stop when the mean observation-space error norm falls below this value.

  • delta_params_tolerance (float) – Stop when the norm of the proposed ensemble update falls below this value.

  • max_iterations (int) – Maximum number of EKI iterations.

  • random_seed (int) – RNG seed used for the initial ensemble draw.

  • evaluation_concurrency – Number of concurrent model evaluations used by each EKI iteration.

  • restart_file – Optional .npz restart file produced by a prior EKI run. When set, the saved ensemble, QoIs, error state, and step size are restored instead of drawing a new ensemble.

Returns:

Tuple (parameter_samples, qois) containing the final ensemble and the corresponding QoI matrix from the last accepted iteration.

See also:

Multi-fidelity EKI#

romtools.workflows.inverse.mf_eki_drivers.run_mf_eki(model, rom_model_builder, parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, absolute_eki_directory='/home/runner/work/rom-tools-and-workflows/rom-tools-and-workflows/work/', fom_ensemble_size=10, rom_extra_ensemble_size=30, rom_tolerance=0.005, use_updated_rom_in_update_on_rebuild=True, initial_step_size=0.1, regularization_parameter=0.0001, step_size_growth_factor=1.25, step_size_decay_factor=2.0, max_step_size_decrease_trys=5, relaxation_parameter=1.05, error_norm_tolerance=1e-05, delta_params_tolerance=0.0001, max_rom_training_history=1, max_iterations=50, random_seed=1, fom_evaluation_concurrency=1, rom_evaluation_concurrency=1, restart_file=None)[source]

Run a multi-fidelity ensemble Kalman inversion (MF-EKI) workflow.

The workflow evaluates the FOM on a smaller ensemble, augments those data with a larger ROM-only ensemble, and forms a multifidelity Kalman update. The ROM is rebuilt from accepted iteration data when its shared-sample QoI error exceeds rom_tolerance.

Parameters:

  • model (QoiModel) – High-fidelity QoiModel used to define the target inverse problem.

  • rom_model_builder (QoiModelBuilderWithTrainingData) – Builder that constructs surrogate QoI models from accumulated training directories and QoI data.

  • parameter_space (ParameterSpace) – ParameterSpace used to draw the initial combined ensemble when restart_file is not provided.

  • observations (ndarray) – Observed QoI vector \(y\).

  • observations_covariance (ndarray) – Observation covariance matrix \(\Gamma_y\) used in the Kalman solve.

  • parameter_mins (ndarray) – Optional lower bounds applied to sampled and updated parameters.

  • parameter_maxes (ndarray) – Optional upper bounds applied to sampled and updated parameters.

  • absolute_eki_directory (str) – Absolute path to the working directory. Each iteration stores FOM and ROM evaluations in iteration-specific subdirectories below this path.

  • fom_ensemble_size (int) – Number of ensemble members evaluated with the high-fidelity model on each iteration.

  • rom_extra_ensemble_size – Number of additional ROM-only ensemble members used to improve multifidelity covariance estimates.

  • rom_tolerance (float) – Relative QoI error threshold on the shared FOM/ROM sample set used to trigger surrogate rebuilding.

  • use_updated_rom_in_update_on_rebuild (bool) – When True, use the updated ROM in the update for that step. When False, use the previous ROM in the update for that step.

  • initial_step_size (float) – Initial multiplier applied to the multifidelity Kalman update directions.

  • regularization_parameter (float) – Tikhonov regularization added to the QoI covariance solve.

  • step_size_growth_factor (float) – Factor used to increase the step size after an accepted iteration.

  • step_size_decay_factor (float) – Factor used to decrease the step size after a rejected trial iteration.

  • max_step_size_decrease_trys (int) – Maximum number of consecutive rejected trial steps before the routine exits.

  • relaxation_parameter (float) – Acceptance threshold on the mean FOM observation-space error norm.

  • error_norm_tolerance (float) – Stop when the mean FOM observation-space error norm falls below this value.

  • delta_params_tolerance (float) – Stop when the norm of the proposed high-fidelity ensemble update falls below this value.

  • max_rom_training_history (int) – Maximum number of accepted iterations whose training data are retained for ROM rebuilding.

  • max_iterations (int) – Maximum number of EKI iterations.

  • random_seed (int) – RNG seed used for the initial ensemble draw.

  • fom_evaluation_concurrency (int) – Number of concurrent FOM evaluations used by each iteration.

  • rom_evaluation_concurrency (int) – Number of concurrent ROM evaluations used by each iteration.

  • restart_file (str) – Optional .npz restart file produced by a prior MF-EKI run. When set, the saved FOM/ROM sample sets, surrogate training history, and step size are restored.

Returns:

Tuple (fom_parameter_samples, fom_qois) for the final accepted high-fidelity sample set.

See also: