romtools.workflows.inverse.vi.mf_drivers#
Multifidelity variational inference drivers.
This module extends the single-fidelity Gaussian variational inference workflow with multifidelity Monte Carlo control variates and adaptive reduced-order models. A correlated low-fidelity model is used to reduce the variance of ELBO gradient and Hessian estimators while retaining a high-fidelity target distribution.
Theory
run_mf_vi extends the single-fidelity VI formulation to a two-level
multifidelity setting. The target objective is still the Gaussian VI ELBO
but gradient and curvature estimates are formed by coupling a high-fidelity model with a correlated low-fidelity surrogate, typically a ROM.
As in run_vi(), the routine is derivative-free with respect to the
forward model and uses score-function estimators:
where \(\mathcal{L}(\theta;\zeta)\) denotes the ELBO integrand.
Multifidelity Control Variates
At each iteration, the code draws a high-fidelity sample set \(\mathcal{S}^{\mathrm{HF}}\) and a larger low-fidelity sample set \(\mathcal{S}^{\mathrm{LF}}\), with the high-fidelity samples reused as a subset of the low-fidelity pool. Writing
the multifidelity gradient estimator has the control-variate form
The same construction is used for the Hessian estimator by replacing \(\Xi_g\) with the corresponding second-order score-function quantity. In expectation, these estimators match the single-fidelity high-fidelity estimators, but their variance can be much lower when the ROM remains well correlated with the FOM.
This is the role of the use_mfmc_control_variate and
mfmc_control_variate_mode options. The implementation supports scalar,
componentwise, and matrix-valued control-variate coefficients.
For score_function_entropy_strategy="joint", fitted gradient coefficients
are set to zero when the measured high-fidelity variance is at roundoff. This
avoids amplifying ROM noise at an exact high-fidelity fixed point. Explicitly
fixed coefficients requested through use_mfmc_control_variate=False are
preserved.
Adaptive ROM Use
The low-fidelity model is refreshed on the fly using high-fidelity training data generated by accepted VI iterations. The routine evaluates the current ROM on the coupled high-fidelity sample set, estimates its relative QoI error, and rebuilds the ROM when that error exceeds the requested tolerance. This keeps the low-fidelity correction informative as the variational distribution moves.
Relation to run_vi
If no extra ROM samples are used, or if the low-fidelity correction is turned
off, the method reduces toward the single-fidelity score-function VI
workflow. The optimizer, line search, parameter-bound handling, and optional
Newton curvature model are otherwise shared with run_vi().
Functions
|
Run multi-fidelity VI with MFMC variance-reduced score-function gradients. |
- romtools.workflows.inverse.vi.mf_drivers.run_mf_vi(model, rom_model_builder, prior_parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, initial_variational_parameter_space=None, restart_file=None, optimizer_method='gradient', optimizer_config=None, line_search_method='stochastic_nonmonotone', line_search_config=None, absolute_work_dir=None, fom_sample_size=10, rom_extra_sample_size=30, rom_tolerance=0.005, max_rom_training_history=1, random_seed=1, sampling_method='mc', fom_evaluation_concurrency=10, rom_evaluation_concurrency=1, covariance_regularization=1e-07, restart_files_to_keep=10, rom_retraining_strategy='delayed', gradient_gain_estimation_strategy='same_sample', elbo_scaling_factor='diag_mean', elbo_relative_tolerance=None, baseline_method=None, use_mfmc_control_variate=True, mfmc_control_variate_mode='componentwise', bounded_parameter_handling='transform', transform_interior_margin=1e-08, transform_map='sigmoid', min_physical_variational_std_fraction=1e-08, dispatcher=None, *, absolute_vi_directory=None, score_function_entropy_strategy='analytic', rom_substep_start_iteration=0, rom_substep_end_iteration=None, num_rom_substeps=0, gain_num_folds=5, hessian_gain_estimation_strategy='same_sample', gain_sample_size=None, mfmc_fixed_gain=1.0, _reuse_controller=None)[source]#
Run multi-fidelity VI with MFMC variance-reduced score-function gradients.
- Parameters:
prior_parameter_space – Either GaussianParameterSpace (diagonal VI) or MultivariateGaussianParameterSpace (multivariate VI). Defines the Bayesian prior in physical parameter space.
parameter_mins (ndarray) – Optional lower bounds on parameters.
parameter_maxes (ndarray) – Optional upper bounds on parameters.
initial_variational_parameter_space – Required Gaussian initializer for the variational state in physical parameter space.
restart_file (str) – Optional restart file path. Restart files written by this routine store variational_mean in physical coordinates.
optimizer_method (str) – Optimizer used for variational updates. Supported options are ‘gradient’, ‘adam’, and ‘newton’.
optimizer_config – Method-specific optimizer config. Expected types are VIGradientOptimizerConfig for optimizer_method=’gradient’, VIAdamOptimizerConfig for optimizer_method=’adam’, and VINewtonOptimizerConfig for optimizer_method=’newton’.
line_search_method (str) – Line-search acceptance strategy. Supported options are ‘none’ (fixed-step Newton), ‘legacy’, and ‘stochastic_nonmonotone’. Defaults to ‘stochastic_nonmonotone’.
line_search_config – Method-specific line-search config. Expected types are VILegacyLineSearchConfig for line_search_method=’legacy’ and VIStochasticNonmonotoneLineSearchConfig for line_search_method=’stochastic_nonmonotone’. For Newton without line search, use
line_search_method='none'and optionallyline_search_config=VINoLineSearchConfig(step_size=0.5).sampling_method (str) – Sampling method for variational draws. Supported options are ‘mc’ and ‘rqmc’.
mfmc_control_variate_mode (str) – Control-variate coefficient strategy. ‘componentwise’ computes one scalar coefficient per component. ‘scalar’ computes one shared scalar coefficient (A = alpha I). ‘matrix’ computes a joint linear map across components.
rom_retraining_strategy (str) – ‘same_sample’ adapts immediately using the current HF batch and history; ‘delayed’ trains after acceptance and activates replacements in the next outer iteration.
gain_num_folds (int) – Number of balanced outer folds for gradient ‘k_fold’ gain estimation (default 5, capped at the paired sample count).
hessian_gain_estimation_strategy (str) – Hessian gain fitting; only ‘same_sample’ is supported, independently of the gradient setting.
mfmc_fixed_gain (float) – Finite scalar coefficient used for gradient and Hessian corrections when use_mfmc_control_variate=False (default 1.0). Componentwise gains use this value uniformly; matrix gains use it times the identity. Ignored when fitting gains is enabled.
gain_sample_size – Independent gradient gain batch size. None matches the current trial HF sample count; a positive integer fixes the count.
gradient_gain_estimation_strategy (str) – Gradient gain fitting. ‘same_sample’ fits one gain on all pairs; ‘loo’ excludes each held-out pair from its gain and baseline fits; ‘k_fold’ excludes each held-out fold and shares one gain per fold; ‘independent’ fits a gain on fresh, separate paired HF/LF samples; ‘history’ importance-weights the accepted HF training window toward the current variational distribution, using the current ROM. The bootstrap trains immediately and uses a fixed unit gain.
transform_interior_margin (float) – Margin used by bounded_parameter_handling=’transform’ to keep mapped samples away from exact bounds.
transform_map (str) – Transform used by bounded_parameter_handling=’transform’. Supported options are ‘sigmoid’ and ‘arctan’.
min_physical_variational_std_fraction (float) – Minimum physical-space variational standard deviation as a fraction of each parameter range when bounded_parameter_handling=’transform’.
score_function_entropy_strategy (str) – Entropy treatment in the score-function gradient.
'analytic'retains the analytic Gaussian entropy gradient;'joint'estimates the fulllog_joint - log_qintegrand. The joint strategy is not a pathwise estimator.dispatcher (BaseDispatcher | None) – Optional (defaults to None, which instantiates a LocalDispatcher). Pass a RemoteDispatcher to send FOM evaluations to the configured remote host (e.g. an HPC cluster). ROM evaluations always run locally, so a RemoteDispatcher requires a relative absolute_work_dir and fom_evaluation_concurrency=1.
model (QoiModel)
rom_model_builder (QoiModelBuilderWithTrainingData)
observations (ndarray)
observations_covariance (ndarray)
absolute_work_dir (str)
fom_sample_size (int)
rom_extra_sample_size (int)
rom_tolerance (float)
max_rom_training_history (int)
random_seed (int)
covariance_regularization (float)
restart_files_to_keep (int)
elbo_relative_tolerance (float)
baseline_method (str)
use_mfmc_control_variate (bool)
bounded_parameter_handling (str)
absolute_vi_directory (str)
rom_substep_start_iteration (int)
rom_substep_end_iteration (int | None)
num_rom_substeps (int)
- Returns:
Tuple of (variational_mean, variational_std, fom_parameter_samples, fom_qois).