romtools.workflows.inverse.mf_vi_drivers

Multifidelity variational inference drivers.

This module extends the single-fidelity Gaussian variational inference workflow with multifidelity Monte Carlo control variates and adaptive reduced-order models. A correlated low-fidelity model is used to reduce the variance of ELBO gradient and Hessian estimators while retaining a high-fidelity target distribution.

Theory

run_mf_vi extends the single-fidelity VI formulation to a two-level multifidelity setting. The target objective is still the Gaussian VI ELBO

\[\max_{\zeta}\; \mathbb{E}_{\theta \sim q(\theta;\zeta)} \left[ \log p(y,\theta) - \log q(\theta;\zeta) \right],\]

but gradient and curvature estimates are formed by coupling a high-fidelity model with a correlated low-fidelity surrogate, typically a ROM.

As in run_vi(), the routine is derivative-free with respect to the forward model and uses score-function estimators:

\[g(\zeta) = \mathbb{E}_{\theta \sim q(\theta;\zeta)} \left[ \mathcal{L}(\theta;\zeta)\, \nabla_{\zeta}\log q(\theta;\zeta) \right],\]

\[H(\zeta) = \mathbb{E}_{\theta \sim q(\theta;\zeta)} \left[ \mathcal{L}(\theta;\zeta) \left( \nabla_{\zeta}\log q\,\nabla_{\zeta}\log q^{\top} + \nabla_{\zeta}^2 \log q \right) \right],\]

where \(\mathcal{L}(\theta;\zeta)\) denotes the ELBO integrand.

Multifidelity Control Variates

At each iteration, the code draws a high-fidelity sample set \(\mathcal{S}^{\mathrm{HF}}\) and a larger low-fidelity sample set \(\mathcal{S}^{\mathrm{LF}}\), with the high-fidelity samples reused as a subset of the low-fidelity pool. Writing

\[\Xi_g^{\mathrm{HF}}(\theta;\zeta) := \mathcal{L}^{\mathrm{HF}}(\theta;\zeta)\, \nabla_{\zeta}\log q(\theta;\zeta),\]

\[\Xi_g^{\mathrm{LF}}(\theta;\zeta) := \mathcal{L}^{\mathrm{LF}}(\theta;\zeta)\, \nabla_{\zeta}\log q(\theta;\zeta),\]

the multifidelity gradient estimator has the control-variate form

\[\widehat g_{\mathrm{MF}} = \left\langle \Xi_g^{\mathrm{HF}} \right\rangle_{\mathcal{S}^{\mathrm{HF}}} + A \left( \left\langle \Xi_g^{\mathrm{LF}} \right\rangle_{\mathcal{S}^{\mathrm{LF}}} - \left\langle \Xi_g^{\mathrm{LF}} \right\rangle_{\mathcal{S}^{\mathrm{HF}}} \right).\]

The same construction is used for the Hessian estimator by replacing \(\Xi_g\) with the corresponding second-order score-function quantity. In expectation, these estimators match the single-fidelity high-fidelity estimators, but their variance can be much lower when the ROM remains well correlated with the FOM.

This is the role of the use_mfmc_control_variate and mfmc_control_variate_mode options. The implementation supports scalar, componentwise, and matrix-valued control-variate coefficients. For score_function_entropy_strategy="joint", fitted gradient coefficients are set to zero when the measured high-fidelity variance is at roundoff. This avoids amplifying ROM noise at an exact high-fidelity fixed point. Explicitly fixed coefficients requested through use_mfmc_control_variate=False are preserved.

Adaptive ROM Use

The low-fidelity model is refreshed on the fly using high-fidelity training data generated by accepted VI iterations. The routine evaluates the current ROM on the coupled high-fidelity sample set, estimates its relative QoI error, and rebuilds the ROM when that error exceeds the requested tolerance. This keeps the low-fidelity correction informative as the variational distribution moves.

Relation to run_vi

If no extra ROM samples are used, or if the low-fidelity correction is turned off, the method reduces toward the single-fidelity score-function VI workflow. The optimizer, line search, parameter-bound handling, and optional Newton curvature model are otherwise shared with run_vi().