romtools.workflows.inverse.mf_vi_drivers#

Multifidelity variational inference drivers.

This module extends the single-fidelity Gaussian variational inference workflow with multifidelity Monte Carlo control variates and adaptive reduced-order models. A correlated low-fidelity model is used to reduce the variance of ELBO gradient and Hessian estimators while retaining a high-fidelity target distribution.

Theory

run_mf_vi extends the single-fidelity VI formulation to a two-level multifidelity setting. The target objective is still the Gaussian VI ELBO

\[\max_{\zeta}\; \mathbb{E}_{\theta \sim q(\theta;\zeta)} \left[ \log p(y,\theta) - \log q(\theta;\zeta) \right],\]

but gradient and curvature estimates are formed by coupling a high-fidelity model with a correlated low-fidelity surrogate, typically a ROM.

As in run_vi(), the routine is derivative-free with respect to the forward model and uses score-function estimators:

\[g(\zeta) = \mathbb{E}_{\theta \sim q(\theta;\zeta)} \left[ \mathcal{L}(\theta;\zeta)\, \nabla_{\zeta}\log q(\theta;\zeta) \right],\]
\[H(\zeta) = \mathbb{E}_{\theta \sim q(\theta;\zeta)} \left[ \mathcal{L}(\theta;\zeta) \left( \nabla_{\zeta}\log q\,\nabla_{\zeta}\log q^{\top} + \nabla_{\zeta}^2 \log q \right) \right],\]

where \(\mathcal{L}(\theta;\zeta)\) denotes the ELBO integrand.

Multifidelity Control Variates

At each iteration, the code draws a high-fidelity sample set \(\mathcal{S}^{\mathrm{HF}}\) and a larger low-fidelity sample set \(\mathcal{S}^{\mathrm{LF}}\), with the high-fidelity samples reused as a subset of the low-fidelity pool. Writing

\[\Xi_g^{\mathrm{HF}}(\theta;\zeta) := \mathcal{L}^{\mathrm{HF}}(\theta;\zeta)\, \nabla_{\zeta}\log q(\theta;\zeta),\]
\[\Xi_g^{\mathrm{LF}}(\theta;\zeta) := \mathcal{L}^{\mathrm{LF}}(\theta;\zeta)\, \nabla_{\zeta}\log q(\theta;\zeta),\]

the multifidelity gradient estimator has the control-variate form

\[\widehat g_{\mathrm{MF}} = \left\langle \Xi_g^{\mathrm{HF}} \right\rangle_{\mathcal{S}^{\mathrm{HF}}} + A \left( \left\langle \Xi_g^{\mathrm{LF}} \right\rangle_{\mathcal{S}^{\mathrm{LF}}} - \left\langle \Xi_g^{\mathrm{LF}} \right\rangle_{\mathcal{S}^{\mathrm{HF}}} \right).\]

The same construction is used for the Hessian estimator by replacing \(\Xi_g\) with the corresponding second-order score-function quantity. In expectation, these estimators match the single-fidelity high-fidelity estimators, but their variance can be much lower when the ROM remains well correlated with the FOM.

This is the role of the use_mfmc_control_variate and mfmc_control_variate_mode options. The implementation supports scalar, componentwise, and matrix-valued control-variate coefficients.

Adaptive ROM Use

The low-fidelity model is refreshed on the fly using high-fidelity training data generated by accepted VI iterations. The routine evaluates the current ROM on the coupled high-fidelity sample set, estimates its relative QoI error, and rebuilds the ROM when that error exceeds the requested tolerance. This keeps the low-fidelity correction informative as the variational distribution moves.

Relation to run_vi

If no extra ROM samples are used, or if the low-fidelity correction is turned off, the method reduces toward the single-fidelity score-function VI workflow. The optimizer, line search, parameter-bound handling, and optional Newton curvature model are otherwise shared with run_vi().

Functions

mf_vi_with_auto_rom(model, ...[, ...])

Wrapper around run_mf_vi that selects a default ROM surrogate by rom_type.

run_mf_vi(model, rom_model_builder, ...[, ...])

Run multi-fidelity VI with MFMC variance-reduced score-function gradients.
romtools.workflows.inverse.mf_vi_drivers.mf_vi_with_auto_rom(model, prior_parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, initial_variational_parameter_space=None, restart_file=None, optimizer_method='gradient', optimizer_config=None, line_search_method='stochastic_nonmonotone', line_search_config=None, absolute_vi_directory='/home/runner/work/rom-tools-and-workflows/rom-tools-and-workflows/work/', fom_sample_size=10, rom_extra_sample_size=30, rom_tolerance=0.005, max_rom_training_history=1, random_seed=1, sampling_method='mc', fom_evaluation_concurrency=1, rom_evaluation_concurrency=1, covariance_regularization=1e-08, restart_files_to_keep=10, correlation_estimator='in_sample', correlation_k_folds=5, elbo_scaling_factor='diag_mean', elbo_relative_tolerance=None, baseline_method=None, use_mfmc_control_variate=True, mfmc_control_variate_mode='componentwise', rom_base_sampling_strategy='coupled', bounded_parameter_handling='transform', transform_interior_margin=1e-08, transform_map='sigmoid', min_physical_variational_std_fraction=1e-08, rom_type='gp', rom_args=None)[source]#

Wrapper around run_mf_vi that selects a default ROM surrogate by rom_type. Accepts the same rom_base_sampling_strategy options as run_mf_vi.

Parameters:

  • model (QoiModel)

  • observations (ndarray)

  • observations_covariance (ndarray)

  • parameter_mins (ndarray)

  • parameter_maxes (ndarray)

  • restart_file (str)

  • optimizer_method (str)

  • line_search_method (str)

  • absolute_vi_directory (str)

  • fom_sample_size (int)

  • rom_extra_sample_size (int)

  • rom_tolerance (float)

  • max_rom_training_history (int)

  • random_seed (int)

  • sampling_method (str)

  • covariance_regularization (float)

  • restart_files_to_keep (int)

  • correlation_estimator (str)

  • correlation_k_folds (int)

  • elbo_relative_tolerance (float)

  • baseline_method (str)

  • use_mfmc_control_variate (bool)

  • mfmc_control_variate_mode (str)

  • rom_base_sampling_strategy (str)

  • bounded_parameter_handling (str)

  • transform_interior_margin (float)

  • transform_map (str)

  • min_physical_variational_std_fraction (float)

  • rom_type (str)

  • rom_args (dict | None)

romtools.workflows.inverse.mf_vi_drivers.run_mf_vi(model, rom_model_builder, prior_parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, initial_variational_parameter_space=None, restart_file=None, optimizer_method='gradient', optimizer_config=None, line_search_method='stochastic_nonmonotone', line_search_config=None, absolute_vi_directory='/home/runner/work/rom-tools-and-workflows/rom-tools-and-workflows/work/', fom_sample_size=10, rom_extra_sample_size=30, rom_tolerance=0.005, max_rom_training_history=1, random_seed=1, sampling_method='mc', fom_evaluation_concurrency=10, rom_evaluation_concurrency=1, covariance_regularization=1e-07, restart_files_to_keep=10, correlation_estimator='in_sample', correlation_k_folds=5, elbo_scaling_factor='diag_mean', elbo_relative_tolerance=None, baseline_method=None, use_mfmc_control_variate=True, mfmc_control_variate_mode='componentwise', rom_base_sampling_strategy='coupled', bounded_parameter_handling='transform', transform_interior_margin=1e-08, transform_map='sigmoid', min_physical_variational_std_fraction=1e-08)[source]#

Run multi-fidelity VI with MFMC variance-reduced score-function gradients.

Parameters:

  • prior_parameter_space – Either GaussianParameterSpace (diagonal VI) or MultivariateGaussianParameterSpace (multivariate VI). Defines the Bayesian prior in physical parameter space.

  • parameter_mins (ndarray) – Optional lower bounds on parameters.

  • parameter_maxes (ndarray) – Optional upper bounds on parameters.

  • initial_variational_parameter_space – Optional Gaussian initializer for the variational state in physical parameter space. Defaults to the prior moments.

  • restart_file (str) – Optional restart file path. Restart files written by this routine store variational_mean in physical coordinates.

  • optimizer_method (str) – Optimizer used for variational updates. Supported options are ‘gradient’ and ‘newton’.

  • optimizer_config – Method-specific optimizer config. Expected types are VIGradientOptimizerConfig for optimizer_method=’gradient’, VINewtonOptimizerConfig for optimizer_method=’newton’.

  • line_search_method (str) – Line-search acceptance strategy. Supported options are ‘legacy’ and ‘stochastic_nonmonotone’. Defaults to ‘stochastic_nonmonotone’.

  • line_search_config – Method-specific line-search config. Expected types are VILegacyLineSearchConfig for line_search_method=’legacy’ and VIStochasticNonmonotoneLineSearchConfig for line_search_method=’stochastic_nonmonotone’.

  • sampling_method (str) – Sampling method for variational draws. Supported options are ‘mc’ and ‘rqmc’.

  • mfmc_control_variate_mode (str) – Control-variate coefficient strategy. ‘componentwise’ computes one scalar coefficient per component. ‘scalar’ computes one shared scalar coefficient (A = alpha I). ‘matrix’ computes a joint linear map across components.

  • rom_base_sampling_strategy (str) – Strategy for ROM-base sample selection. ‘coupled’ (default) reuses the FOM sample set for ROM-base evaluations. ‘separate’ draws an independent ROM-base sample set.

  • transform_interior_margin (float) – Margin used by bounded_parameter_handling=’transform’ to keep mapped samples away from exact bounds.

  • transform_map (str) – Transform used by bounded_parameter_handling=’transform’. Supported options are ‘sigmoid’ and ‘arctan’.

  • min_physical_variational_std_fraction (float) – Minimum physical-space variational standard deviation as a fraction of each parameter range when bounded_parameter_handling=’transform’.

  • model (QoiModel)

  • rom_model_builder (QoiModelBuilderWithTrainingData)

  • observations (ndarray)

  • observations_covariance (ndarray)

  • absolute_vi_directory (str)

  • fom_sample_size (int)

  • rom_extra_sample_size (int)

  • rom_tolerance (float)

  • max_rom_training_history (int)

  • random_seed (int)

  • covariance_regularization (float)

  • restart_files_to_keep (int)

  • correlation_estimator (str)

  • correlation_k_folds (int)

  • elbo_relative_tolerance (float)

  • baseline_method (str)

  • use_mfmc_control_variate (bool)

  • bounded_parameter_handling (str)

Returns:

Tuple of (variational_mean, variational_std, fom_parameter_samples, fom_qois).