romtools.workflows.inverse.vi#
Gaussian variational inference workflows and optimizer configurations.
Use run_vi, run_mf_vi, and mf_vi_with_auto_rom here or through
romtools.workflows.inverse. Implementation modules are private to this package.
- class romtools.workflows.inverse.vi.AdamSolver(learning_rate=None, learning_rate_scale=0.1, beta1=0.9, beta2=0.999, epsilon=1e-08, fisher_damping_initial=0.01, fisher_damping_decay_start=50, fisher_damping_min=1e-06, gradient_clip_norm=1000000.0, parameter_dimension=None)[source]#
Bases:
objectStateful Adam ascent direction for stochastic VI gradients.
- Parameters:
learning_rate (float)
learning_rate_scale (float)
beta1 (float)
beta2 (float)
epsilon (float)
fisher_damping_initial (float)
fisher_damping_decay_start (int)
fisher_damping_min (float)
gradient_clip_norm (float)
parameter_dimension (int)
- load_restart_state_dict(restart_data)[source]#
Restore Adam state and reject incompatible optimizer configuration.
- Return type:
None
- prepare_gradient(gradient, fisher_diagonal=None)[source]#
Apply Fisher damping and clipping without updating Adam moments.
- Parameters:
gradient (ndarray)
- Return type:
ndarray
- class romtools.workflows.inverse.vi.NewtonSolver(regularization, hessian_type='diagonal', regularization_strategy='absolute', additive_regularization=0.0, regularization_epsilon=1e-12, fallback_learning_rate=0.01)[source]#
Bases:
objectGeneric Newton solver supporting diagonal or dense Hessians.
- Parameters:
regularization (float)
hessian_type (str)
regularization_strategy (str)
additive_regularization (float)
regularization_epsilon (float)
fallback_learning_rate (float)
- class romtools.workflows.inverse.vi.SteepestDescentSolver[source]#
Bases:
objectGeneric steepest-descent solver.
- class romtools.workflows.inverse.vi.VIAdamOptimizerConfig(gradient_method='natural', learning_rate=None, learning_rate_scale=0.1, beta1=0.9, beta2=0.999, epsilon=1e-08, fisher_damping_initial=0.01, fisher_damping_decay_start=50, fisher_damping_min=1e-06, gradient_clip_norm=1000000.0, gradient_norm_tolerance=1e-05, max_iterations=1000, max_log_std_update=0.5, min_variational_std=1e-06, max_variational_std=1000000.0)[source]#
Bases:
objectConfiguration for Adam-based stochastic ELBO maximization.
By default Adam is applied to the mean-field Gaussian natural gradient. Following the ABRIS setup, the Fisher matrix is damped early in the optimization and the preconditioned gradient is clipped by its L2 norm. When
learning_rateisNone, the ABRIS convention0.1 / parameter_dimensionis used.- Parameters:
gradient_method (str)
learning_rate (float)
learning_rate_scale (float)
beta1 (float)
beta2 (float)
epsilon (float)
fisher_damping_initial (float)
fisher_damping_decay_start (int)
fisher_damping_min (float)
gradient_clip_norm (float)
gradient_norm_tolerance (float)
max_iterations (int)
max_log_std_update (float)
min_variational_std (float)
max_variational_std (float)
- class romtools.workflows.inverse.vi.VIGradientOptimizerConfig(gradient_method: str = 'standard', gradient_norm_tolerance: float = 1e-05, max_iterations: int = 1000, max_log_std_update: float = 0.5, min_variational_std: float = 1e-06, max_variational_std: float = 1000000.0)[source]#
Bases:
object- Parameters:
gradient_method (str)
gradient_norm_tolerance (float)
max_iterations (int)
max_log_std_update (float)
min_variational_std (float)
max_variational_std (float)
- class romtools.workflows.inverse.vi.VILegacyLineSearchConfig(initial_step_size: float = 0.1, max_step_size: float = inf, step_size_growth_factor: float = 1.01, step_size_decay_factor: float = 3.0, max_step_size_decrease_trys: int = 5, relaxation_parameter: float = 2.05, line_search_objective: str = 'elbo', line_search_sample_growth_factor: float = 1.0, log_std_learning_rate_factor: float = 0.1)[source]#
Bases:
object- Parameters:
initial_step_size (float)
max_step_size (float)
step_size_growth_factor (float)
step_size_decay_factor (float)
max_step_size_decrease_trys (int)
relaxation_parameter (float)
line_search_objective (str)
line_search_sample_growth_factor (float)
log_std_learning_rate_factor (float)
- class romtools.workflows.inverse.vi.VINewtonOptimizerConfig(gradient_norm_tolerance: float = 5e-05, max_iterations: int = 1000, max_log_std_update: float = 0.5, max_mean_update_std: float = None, min_variational_std: float = 1e-08, max_variational_std: float = 1000000.0, newton_metric: str = 'natural', newton_fallback_learning_rate: float = 0.02, newton_regularization: float = 0.0005, newton_hessian_type: str = 'full', newton_curvature_strategy: str = 'lagged', newton_hessian_num_samples: int = None, newton_hessian_averaging_factor: float = 0.25, newton_regularization_strategy: str = 'absolute', newton_additive_regularization: float = 0.0, newton_regularization_epsilon: float = 1e-12, newton_adaptive_regularization: bool = False, newton_regularization_increase_factor: float = 5.0, newton_regularization_decrease_factor: float = 1.25, newton_regularization_max_multiplier: float = 10000.0, newton_additive_regularization_growth_factor: float = 1.0, newton_additive_regularization_max: float = None)[source]#
Bases:
object- Parameters:
gradient_norm_tolerance (float)
max_iterations (int)
max_log_std_update (float)
max_mean_update_std (float)
min_variational_std (float)
max_variational_std (float)
newton_metric (str)
newton_fallback_learning_rate (float)
newton_regularization (float)
newton_hessian_type (str)
newton_curvature_strategy (str)
newton_hessian_num_samples (int)
newton_hessian_averaging_factor (float)
newton_regularization_strategy (str)
newton_additive_regularization (float)
newton_regularization_epsilon (float)
newton_adaptive_regularization (bool)
newton_regularization_increase_factor (float)
newton_regularization_decrease_factor (float)
newton_regularization_max_multiplier (float)
newton_additive_regularization_growth_factor (float)
newton_additive_regularization_max (float)
- class romtools.workflows.inverse.vi.VINoLineSearchConfig(step_size=1.0, log_std_learning_rate_factor=1.0)[source]#
Bases:
objectConstant Newton step without objective-based acceptance or backtracking.
Use with
optimizer_method='newton', line_search_method='none'inrun_vi()orrun_mf_vi(). Existing update limits still apply.log_std_learning_rate_factorscales diagonal-family log-std updates.- Parameters:
step_size (float)
log_std_learning_rate_factor (float)
- class romtools.workflows.inverse.vi.VISampleReuseConfig(enabled=True, history_batches=10, ess_threshold=None, use_score_diagnostic=True, score_error_scale=1.0, periodic_refresh=100, use_hessian_score_diagnostic=True, hessian_score_error_scale=2.0, hessian_relative_standard_error_threshold=0.25)[source]#
Bases:
objectConfiguration for ABRIS-style reuse of previous VI samples.
history_batchescontrols both the maximum moving-window size and the maximum number of fresh batches allowed in one state evaluation. During zero-based iterationi, both limits grow asmin(history_batches, i + 1). Every returned update is formed from the retained archive, including evaluations that sampled fresh batches.- Parameters:
enabled (bool)
history_batches (int)
ess_threshold (float | None)
use_score_diagnostic (bool)
score_error_scale (float)
periodic_refresh (int | None)
use_hessian_score_diagnostic (bool)
hessian_score_error_scale (float)
hessian_relative_standard_error_threshold (float | None)
- class romtools.workflows.inverse.vi.VIStochasticNonmonotoneLineSearchConfig(initial_step_size: float = 0.1, max_step_size: float = inf, step_size_growth_factor: float = 1.05, step_size_decay_factor: float = 2.0, max_step_size_decrease_trys: int = 15, relaxation_parameter: float = 3.05, line_search_objective: str = 'elbo', line_search_nonmonotone_window: int = 5, line_search_armijo_coefficient: float = 1e-06, line_search_uncertainty_sigma: float = 3.0, line_search_sample_growth_factor: float = 1.0, log_std_learning_rate_factor: float = 1.0)[source]#
Bases:
object- Parameters:
initial_step_size (float)
max_step_size (float)
step_size_growth_factor (float)
step_size_decay_factor (float)
max_step_size_decrease_trys (int)
relaxation_parameter (float)
line_search_objective (str)
line_search_nonmonotone_window (int)
line_search_armijo_coefficient (float)
line_search_uncertainty_sigma (float)
line_search_sample_growth_factor (float)
log_std_learning_rate_factor (float)
- romtools.workflows.inverse.vi.mf_vi_with_auto_rom(model, prior_parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, restart_file=None, optimizer_method='gradient', optimizer_config=None, line_search_method='stochastic_nonmonotone', line_search_config=None, absolute_work_dir=None, fom_sample_size=10, rom_extra_sample_size=30, rom_tolerance=0.005, max_rom_training_history=None, random_seed=1, sampling_method='mc', fom_evaluation_concurrency=1, rom_evaluation_concurrency=1, covariance_regularization=1e-08, restart_files_to_keep=10, rom_retraining_strategy='delayed', gradient_gain_estimation_strategy='same_sample', elbo_scaling_factor='diag_mean', elbo_relative_tolerance=None, baseline_method=None, use_mfmc_control_variate=True, mfmc_control_variate_mode='componentwise', bounded_parameter_handling='transform', transform_interior_margin=1e-08, transform_map='sigmoid', min_physical_variational_std_fraction=1e-08, rom_type='gp', rom_args=None, dispatcher=None, *, initial_variational_parameter_space, max_covariance_log_step=1.0, absolute_vi_directory=None, score_function_entropy_strategy='analytic', rom_substep_start_iteration=0, rom_substep_end_iteration=None, num_rom_substeps=0, gain_num_folds=5, hessian_gain_estimation_strategy='same_sample', gain_sample_size=None, mfmc_fixed_gain=1.0, create_run_directories=True, sample_reuse_config=None)[source]#
Wrapper around run_mf_vi that selects a default ROM surrogate by rom_type. Accepts the same ROM-retraining and gain-estimation options as run_mf_vi.
- Parameters:
model (QoiModel)
observations (ndarray)
observations_covariance (ndarray)
parameter_mins (ndarray)
parameter_maxes (ndarray)
restart_file (str)
optimizer_method (str)
line_search_method (str)
absolute_work_dir (str)
fom_sample_size (int)
rom_extra_sample_size (int)
rom_tolerance (float)
max_rom_training_history (int)
random_seed (int)
sampling_method (str)
covariance_regularization (float)
restart_files_to_keep (int)
rom_retraining_strategy (str)
gradient_gain_estimation_strategy (str)
elbo_relative_tolerance (float)
baseline_method (str)
use_mfmc_control_variate (bool)
mfmc_control_variate_mode (str)
bounded_parameter_handling (str)
transform_interior_margin (float)
transform_map (str)
min_physical_variational_std_fraction (float)
rom_type (str)
rom_args (dict | None)
dispatcher (BaseDispatcher | None)
max_covariance_log_step (float)
absolute_vi_directory (str)
score_function_entropy_strategy (str)
rom_substep_start_iteration (int)
rom_substep_end_iteration (int | None)
num_rom_substeps (int)
gain_num_folds (int)
hessian_gain_estimation_strategy (str)
mfmc_fixed_gain (float)
create_run_directories (bool)
- romtools.workflows.inverse.vi.run_mf_vi(model, rom_model_builder, prior_parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, restart_file=None, optimizer_method='gradient', optimizer_config=None, line_search_method='stochastic_nonmonotone', line_search_config=None, absolute_work_dir=None, fom_sample_size=10, rom_extra_sample_size=30, rom_tolerance=0.005, max_rom_training_history=None, random_seed=1, sampling_method='mc', fom_evaluation_concurrency=10, rom_evaluation_concurrency=1, covariance_regularization=1e-07, restart_files_to_keep=10, rom_retraining_strategy='delayed', gradient_gain_estimation_strategy='same_sample', elbo_scaling_factor='diag_mean', elbo_relative_tolerance=None, baseline_method=None, use_mfmc_control_variate=True, mfmc_control_variate_mode='componentwise', bounded_parameter_handling='transform', transform_interior_margin=1e-08, transform_map='sigmoid', min_physical_variational_std_fraction=1e-08, dispatcher=None, *, initial_variational_parameter_space, max_covariance_log_step=1.0, absolute_vi_directory=None, score_function_entropy_strategy='analytic', rom_substep_start_iteration=0, rom_substep_end_iteration=None, num_rom_substeps=0, gain_num_folds=5, hessian_gain_estimation_strategy='same_sample', gain_sample_size=None, mfmc_fixed_gain=1.0, create_run_directories=True, sample_reuse_config=None)[source]#
Run Gaussian multifidelity VI with an adaptive surrogate builder.
- Parameters:
model (QoiModel)
rom_model_builder (QoiModelBuilderWithTrainingData)
observations (ndarray)
observations_covariance (ndarray)
parameter_mins (ndarray)
parameter_maxes (ndarray)
restart_file (str)
optimizer_method (str)
line_search_method (str)
absolute_work_dir (str)
fom_sample_size (int)
rom_extra_sample_size (int)
rom_tolerance (float)
max_rom_training_history (int)
random_seed (int)
sampling_method (str)
covariance_regularization (float)
restart_files_to_keep (int)
rom_retraining_strategy (str)
gradient_gain_estimation_strategy (str)
elbo_relative_tolerance (float)
baseline_method (str)
use_mfmc_control_variate (bool)
mfmc_control_variate_mode (str)
bounded_parameter_handling (str)
transform_interior_margin (float)
transform_map (str)
min_physical_variational_std_fraction (float)
dispatcher (BaseDispatcher | None)
max_covariance_log_step (float)
absolute_vi_directory (str)
score_function_entropy_strategy (str)
rom_substep_start_iteration (int)
rom_substep_end_iteration (int | None)
num_rom_substeps (int)
gain_num_folds (int)
hessian_gain_estimation_strategy (str)
mfmc_fixed_gain (float)
create_run_directories (bool)
sample_reuse_config (VISampleReuseConfig)
- romtools.workflows.inverse.vi.run_vi(model, prior_parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, restart_file=None, optimizer_method='gradient', optimizer_config=None, line_search_method='stochastic_nonmonotone', line_search_config=None, absolute_work_dir=None, sample_size=30, random_seed=1, sampling_method='mc', evaluation_concurrency=1, covariance_regularization=1e-08, restart_files_to_keep=10, elbo_scaling_factor='diag_mean', elbo_relative_tolerance=None, baseline_method=None, bounded_parameter_handling='transform', transform_interior_margin=1e-06, transform_map='sigmoid', min_physical_variational_std_fraction=1e-06, dispatcher=None, *, initial_variational_parameter_space, max_covariance_log_step=1.0, absolute_vi_directory=None, score_function_entropy_strategy='analytic', create_run_directories=True, sample_reuse_config=None)[source]#
Run Gaussian VI; the initializer determines the variational family.
- Parameters:
model (QoiModel)
observations (ndarray)
observations_covariance (ndarray)
parameter_mins (ndarray)
parameter_maxes (ndarray)
restart_file (str)
optimizer_method (str)
line_search_method (str)
absolute_work_dir (str)
sample_size (int)
random_seed (int)
sampling_method (str)
covariance_regularization (float)
restart_files_to_keep (int)
elbo_relative_tolerance (float)
baseline_method (str)
bounded_parameter_handling (str)
transform_interior_margin (float)
transform_map (str)
min_physical_variational_std_fraction (float)
dispatcher (BaseDispatcher | None)
max_covariance_log_step (float)
absolute_vi_directory (str)
score_function_entropy_strategy (str)
create_run_directories (bool)
sample_reuse_config (VISampleReuseConfig)
Modules
Accepted-iteration lifecycle for delayed MF-VI ROM adaptation. |
|
Public entry points for diagonal and full-covariance variational inference. |
|
Shared density, baseline, and uncertainty helpers for VI estimators. |
|
Variational inference drivers for Gaussian posterior approximations. |
|
Run-directory policy for VI and MFVI model evaluations. |
|
Full-covariance Gaussian VI wrappers. |
|
Full-covariance multifidelity variational inference. |
|
Cross-fitted ACV gains from cached score/weight moments. |
|
Utilities for full-covariance Gaussian variational inference. |
|
Multifidelity variational inference drivers. |
|
Natural-coordinate Newton support for full-covariance Gaussian VI/MF-VI. |
|
Optimization methods and configs used by inverse VI workflows. |
|
ROM-only optimizer substeps for multifidelity variational inference. |
|
Importance-sampling sample reuse for VI and MFVI. |