romtools.workflows.inverse.vi#

Gaussian variational inference workflows and optimizer configurations.

Use run_vi, run_mf_vi, and mf_vi_with_auto_rom here or through romtools.workflows.inverse. Implementation modules are private to this package.

class romtools.workflows.inverse.vi.AdamSolver(learning_rate=None, learning_rate_scale=0.1, beta1=0.9, beta2=0.999, epsilon=1e-08, fisher_damping_initial=0.01, fisher_damping_decay_start=50, fisher_damping_min=1e-06, gradient_clip_norm=1000000.0, parameter_dimension=None)[source]#

Bases: object

Stateful Adam ascent direction for stochastic VI gradients.

Parameters:
  • learning_rate (float)

  • learning_rate_scale (float)

  • beta1 (float)

  • beta2 (float)

  • epsilon (float)

  • fisher_damping_initial (float)

  • fisher_damping_decay_start (int)

  • fisher_damping_min (float)

  • gradient_clip_norm (float)

  • parameter_dimension (int)

load_restart_state_dict(restart_data)[source]#

Restore Adam state and reject incompatible optimizer configuration.

Return type:

None

prepare_gradient(gradient, fisher_diagonal=None)[source]#

Apply Fisher damping and clipping without updating Adam moments.

Parameters:

gradient (ndarray)

Return type:

ndarray

resolved_learning_rate(gradient)[source]#

Return the learning rate used for the supplied VI gradient.

Parameters:

gradient (ndarray)

Return type:

float

restart_state_dict()[source]#

Return restart-safe Adam state and configuration metadata.

Return type:

dict

class romtools.workflows.inverse.vi.NewtonSolver(regularization, hessian_type='diagonal', regularization_strategy='absolute', additive_regularization=0.0, regularization_epsilon=1e-12, fallback_learning_rate=0.01)[source]#

Bases: object

Generic Newton solver supporting diagonal or dense Hessians.

Parameters:
  • regularization (float)

  • hessian_type (str)

  • regularization_strategy (str)

  • additive_regularization (float)

  • regularization_epsilon (float)

  • fallback_learning_rate (float)

class romtools.workflows.inverse.vi.SteepestDescentSolver[source]#

Bases: object

Generic steepest-descent solver.

class romtools.workflows.inverse.vi.VIAdamOptimizerConfig(gradient_method='natural', learning_rate=None, learning_rate_scale=0.1, beta1=0.9, beta2=0.999, epsilon=1e-08, fisher_damping_initial=0.01, fisher_damping_decay_start=50, fisher_damping_min=1e-06, gradient_clip_norm=1000000.0, gradient_norm_tolerance=1e-05, max_iterations=1000, max_log_std_update=0.5, min_variational_std=1e-06, max_variational_std=1000000.0)[source]#

Bases: object

Configuration for Adam-based stochastic ELBO maximization.

By default Adam is applied to the mean-field Gaussian natural gradient. Following the ABRIS setup, the Fisher matrix is damped early in the optimization and the preconditioned gradient is clipped by its L2 norm. When learning_rate is None, the ABRIS convention 0.1 / parameter_dimension is used.

Parameters:
  • gradient_method (str)

  • learning_rate (float)

  • learning_rate_scale (float)

  • beta1 (float)

  • beta2 (float)

  • epsilon (float)

  • fisher_damping_initial (float)

  • fisher_damping_decay_start (int)

  • fisher_damping_min (float)

  • gradient_clip_norm (float)

  • gradient_norm_tolerance (float)

  • max_iterations (int)

  • max_log_std_update (float)

  • min_variational_std (float)

  • max_variational_std (float)

class romtools.workflows.inverse.vi.VIGradientOptimizerConfig(gradient_method: str = 'standard', gradient_norm_tolerance: float = 1e-05, max_iterations: int = 1000, max_log_std_update: float = 0.5, min_variational_std: float = 1e-06, max_variational_std: float = 1000000.0)[source]#

Bases: object

Parameters:
  • gradient_method (str)

  • gradient_norm_tolerance (float)

  • max_iterations (int)

  • max_log_std_update (float)

  • min_variational_std (float)

  • max_variational_std (float)

class romtools.workflows.inverse.vi.VILegacyLineSearchConfig(initial_step_size: float = 0.1, max_step_size: float = inf, step_size_growth_factor: float = 1.01, step_size_decay_factor: float = 3.0, max_step_size_decrease_trys: int = 5, relaxation_parameter: float = 2.05, line_search_objective: str = 'elbo', line_search_sample_growth_factor: float = 1.0, log_std_learning_rate_factor: float = 0.1)[source]#

Bases: object

Parameters:
  • initial_step_size (float)

  • max_step_size (float)

  • step_size_growth_factor (float)

  • step_size_decay_factor (float)

  • max_step_size_decrease_trys (int)

  • relaxation_parameter (float)

  • line_search_objective (str)

  • line_search_sample_growth_factor (float)

  • log_std_learning_rate_factor (float)

class romtools.workflows.inverse.vi.VINewtonOptimizerConfig(gradient_norm_tolerance: float = 5e-05, max_iterations: int = 1000, max_log_std_update: float = 0.5, max_mean_update_std: float = None, min_variational_std: float = 1e-08, max_variational_std: float = 1000000.0, newton_metric: str = 'natural', newton_fallback_learning_rate: float = 0.02, newton_regularization: float = 0.0005, newton_hessian_type: str = 'full', newton_curvature_strategy: str = 'lagged', newton_hessian_num_samples: int = None, newton_hessian_averaging_factor: float = 0.25, newton_regularization_strategy: str = 'absolute', newton_additive_regularization: float = 0.0, newton_regularization_epsilon: float = 1e-12, newton_adaptive_regularization: bool = False, newton_regularization_increase_factor: float = 5.0, newton_regularization_decrease_factor: float = 1.25, newton_regularization_max_multiplier: float = 10000.0, newton_additive_regularization_growth_factor: float = 1.0, newton_additive_regularization_max: float = None)[source]#

Bases: object

Parameters:
  • gradient_norm_tolerance (float)

  • max_iterations (int)

  • max_log_std_update (float)

  • max_mean_update_std (float)

  • min_variational_std (float)

  • max_variational_std (float)

  • newton_metric (str)

  • newton_fallback_learning_rate (float)

  • newton_regularization (float)

  • newton_hessian_type (str)

  • newton_curvature_strategy (str)

  • newton_hessian_num_samples (int)

  • newton_hessian_averaging_factor (float)

  • newton_regularization_strategy (str)

  • newton_additive_regularization (float)

  • newton_regularization_epsilon (float)

  • newton_adaptive_regularization (bool)

  • newton_regularization_increase_factor (float)

  • newton_regularization_decrease_factor (float)

  • newton_regularization_max_multiplier (float)

  • newton_additive_regularization_growth_factor (float)

  • newton_additive_regularization_max (float)

class romtools.workflows.inverse.vi.VINoLineSearchConfig(step_size=1.0, log_std_learning_rate_factor=1.0)[source]#

Bases: object

Constant Newton step without objective-based acceptance or backtracking.

Use with optimizer_method='newton', line_search_method='none' in run_vi() or run_mf_vi(). Existing update limits still apply. log_std_learning_rate_factor scales diagonal-family log-std updates.

Parameters:
  • step_size (float)

  • log_std_learning_rate_factor (float)

class romtools.workflows.inverse.vi.VISampleReuseConfig(enabled=True, history_batches=10, ess_threshold=None, use_score_diagnostic=True, score_error_scale=1.0, periodic_refresh=100, use_hessian_score_diagnostic=True, hessian_score_error_scale=2.0, hessian_relative_standard_error_threshold=0.25)[source]#

Bases: object

Configuration for ABRIS-style reuse of previous VI samples.

history_batches controls both the maximum moving-window size and the maximum number of fresh batches allowed in one state evaluation. During zero-based iteration i, both limits grow as min(history_batches, i + 1). Every returned update is formed from the retained archive, including evaluations that sampled fresh batches.

Parameters:
  • enabled (bool)

  • history_batches (int)

  • ess_threshold (float | None)

  • use_score_diagnostic (bool)

  • score_error_scale (float)

  • periodic_refresh (int | None)

  • use_hessian_score_diagnostic (bool)

  • hessian_score_error_scale (float)

  • hessian_relative_standard_error_threshold (float | None)

class romtools.workflows.inverse.vi.VIStochasticNonmonotoneLineSearchConfig(initial_step_size: float = 0.1, max_step_size: float = inf, step_size_growth_factor: float = 1.05, step_size_decay_factor: float = 2.0, max_step_size_decrease_trys: int = 15, relaxation_parameter: float = 3.05, line_search_objective: str = 'elbo', line_search_nonmonotone_window: int = 5, line_search_armijo_coefficient: float = 1e-06, line_search_uncertainty_sigma: float = 3.0, line_search_sample_growth_factor: float = 1.0, log_std_learning_rate_factor: float = 1.0)[source]#

Bases: object

Parameters:
  • initial_step_size (float)

  • max_step_size (float)

  • step_size_growth_factor (float)

  • step_size_decay_factor (float)

  • max_step_size_decrease_trys (int)

  • relaxation_parameter (float)

  • line_search_objective (str)

  • line_search_nonmonotone_window (int)

  • line_search_armijo_coefficient (float)

  • line_search_uncertainty_sigma (float)

  • line_search_sample_growth_factor (float)

  • log_std_learning_rate_factor (float)

romtools.workflows.inverse.vi.mf_vi_with_auto_rom(model, prior_parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, restart_file=None, optimizer_method='gradient', optimizer_config=None, line_search_method='stochastic_nonmonotone', line_search_config=None, absolute_work_dir=None, fom_sample_size=10, rom_extra_sample_size=30, rom_tolerance=0.005, max_rom_training_history=None, random_seed=1, sampling_method='mc', fom_evaluation_concurrency=1, rom_evaluation_concurrency=1, covariance_regularization=1e-08, restart_files_to_keep=10, rom_retraining_strategy='delayed', gradient_gain_estimation_strategy='same_sample', elbo_scaling_factor='diag_mean', elbo_relative_tolerance=None, baseline_method=None, use_mfmc_control_variate=True, mfmc_control_variate_mode='componentwise', bounded_parameter_handling='transform', transform_interior_margin=1e-08, transform_map='sigmoid', min_physical_variational_std_fraction=1e-08, rom_type='gp', rom_args=None, dispatcher=None, *, initial_variational_parameter_space, max_covariance_log_step=1.0, absolute_vi_directory=None, score_function_entropy_strategy='analytic', rom_substep_start_iteration=0, rom_substep_end_iteration=None, num_rom_substeps=0, gain_num_folds=5, hessian_gain_estimation_strategy='same_sample', gain_sample_size=None, mfmc_fixed_gain=1.0, create_run_directories=True, sample_reuse_config=None)[source]#

Wrapper around run_mf_vi that selects a default ROM surrogate by rom_type. Accepts the same ROM-retraining and gain-estimation options as run_mf_vi.

Parameters:
  • model (QoiModel)

  • observations (ndarray)

  • observations_covariance (ndarray)

  • parameter_mins (ndarray)

  • parameter_maxes (ndarray)

  • restart_file (str)

  • optimizer_method (str)

  • line_search_method (str)

  • absolute_work_dir (str)

  • fom_sample_size (int)

  • rom_extra_sample_size (int)

  • rom_tolerance (float)

  • max_rom_training_history (int)

  • random_seed (int)

  • sampling_method (str)

  • covariance_regularization (float)

  • restart_files_to_keep (int)

  • rom_retraining_strategy (str)

  • gradient_gain_estimation_strategy (str)

  • elbo_relative_tolerance (float)

  • baseline_method (str)

  • use_mfmc_control_variate (bool)

  • mfmc_control_variate_mode (str)

  • bounded_parameter_handling (str)

  • transform_interior_margin (float)

  • transform_map (str)

  • min_physical_variational_std_fraction (float)

  • rom_type (str)

  • rom_args (dict | None)

  • dispatcher (BaseDispatcher | None)

  • max_covariance_log_step (float)

  • absolute_vi_directory (str)

  • score_function_entropy_strategy (str)

  • rom_substep_start_iteration (int)

  • rom_substep_end_iteration (int | None)

  • num_rom_substeps (int)

  • gain_num_folds (int)

  • hessian_gain_estimation_strategy (str)

  • mfmc_fixed_gain (float)

  • create_run_directories (bool)

romtools.workflows.inverse.vi.run_mf_vi(model, rom_model_builder, prior_parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, restart_file=None, optimizer_method='gradient', optimizer_config=None, line_search_method='stochastic_nonmonotone', line_search_config=None, absolute_work_dir=None, fom_sample_size=10, rom_extra_sample_size=30, rom_tolerance=0.005, max_rom_training_history=None, random_seed=1, sampling_method='mc', fom_evaluation_concurrency=10, rom_evaluation_concurrency=1, covariance_regularization=1e-07, restart_files_to_keep=10, rom_retraining_strategy='delayed', gradient_gain_estimation_strategy='same_sample', elbo_scaling_factor='diag_mean', elbo_relative_tolerance=None, baseline_method=None, use_mfmc_control_variate=True, mfmc_control_variate_mode='componentwise', bounded_parameter_handling='transform', transform_interior_margin=1e-08, transform_map='sigmoid', min_physical_variational_std_fraction=1e-08, dispatcher=None, *, initial_variational_parameter_space, max_covariance_log_step=1.0, absolute_vi_directory=None, score_function_entropy_strategy='analytic', rom_substep_start_iteration=0, rom_substep_end_iteration=None, num_rom_substeps=0, gain_num_folds=5, hessian_gain_estimation_strategy='same_sample', gain_sample_size=None, mfmc_fixed_gain=1.0, create_run_directories=True, sample_reuse_config=None)[source]#

Run Gaussian multifidelity VI with an adaptive surrogate builder.

Parameters:
  • model (QoiModel)

  • rom_model_builder (QoiModelBuilderWithTrainingData)

  • observations (ndarray)

  • observations_covariance (ndarray)

  • parameter_mins (ndarray)

  • parameter_maxes (ndarray)

  • restart_file (str)

  • optimizer_method (str)

  • line_search_method (str)

  • absolute_work_dir (str)

  • fom_sample_size (int)

  • rom_extra_sample_size (int)

  • rom_tolerance (float)

  • max_rom_training_history (int)

  • random_seed (int)

  • sampling_method (str)

  • covariance_regularization (float)

  • restart_files_to_keep (int)

  • rom_retraining_strategy (str)

  • gradient_gain_estimation_strategy (str)

  • elbo_relative_tolerance (float)

  • baseline_method (str)

  • use_mfmc_control_variate (bool)

  • mfmc_control_variate_mode (str)

  • bounded_parameter_handling (str)

  • transform_interior_margin (float)

  • transform_map (str)

  • min_physical_variational_std_fraction (float)

  • dispatcher (BaseDispatcher | None)

  • max_covariance_log_step (float)

  • absolute_vi_directory (str)

  • score_function_entropy_strategy (str)

  • rom_substep_start_iteration (int)

  • rom_substep_end_iteration (int | None)

  • num_rom_substeps (int)

  • gain_num_folds (int)

  • hessian_gain_estimation_strategy (str)

  • mfmc_fixed_gain (float)

  • create_run_directories (bool)

  • sample_reuse_config (VISampleReuseConfig)

romtools.workflows.inverse.vi.run_vi(model, prior_parameter_space, observations, observations_covariance, parameter_mins=None, parameter_maxes=None, restart_file=None, optimizer_method='gradient', optimizer_config=None, line_search_method='stochastic_nonmonotone', line_search_config=None, absolute_work_dir=None, sample_size=30, random_seed=1, sampling_method='mc', evaluation_concurrency=1, covariance_regularization=1e-08, restart_files_to_keep=10, elbo_scaling_factor='diag_mean', elbo_relative_tolerance=None, baseline_method=None, bounded_parameter_handling='transform', transform_interior_margin=1e-06, transform_map='sigmoid', min_physical_variational_std_fraction=1e-06, dispatcher=None, *, initial_variational_parameter_space, max_covariance_log_step=1.0, absolute_vi_directory=None, score_function_entropy_strategy='analytic', create_run_directories=True, sample_reuse_config=None)[source]#

Run Gaussian VI; the initializer determines the variational family.

Parameters:
  • model (QoiModel)

  • observations (ndarray)

  • observations_covariance (ndarray)

  • parameter_mins (ndarray)

  • parameter_maxes (ndarray)

  • restart_file (str)

  • optimizer_method (str)

  • line_search_method (str)

  • absolute_work_dir (str)

  • sample_size (int)

  • random_seed (int)

  • sampling_method (str)

  • covariance_regularization (float)

  • restart_files_to_keep (int)

  • elbo_relative_tolerance (float)

  • baseline_method (str)

  • bounded_parameter_handling (str)

  • transform_interior_margin (float)

  • transform_map (str)

  • min_physical_variational_std_fraction (float)

  • dispatcher (BaseDispatcher | None)

  • max_covariance_log_step (float)

  • absolute_vi_directory (str)

  • score_function_entropy_strategy (str)

  • create_run_directories (bool)

  • sample_reuse_config (VISampleReuseConfig)

Modules

adaptation

Accepted-iteration lifecycle for delayed MF-VI ROM adaptation.

api

Public entry points for diagonal and full-covariance variational inference.

common

Shared density, baseline, and uncertainty helpers for VI estimators.

drivers

Variational inference drivers for Gaussian posterior approximations.

evaluation

Run-directory policy for VI and MFVI model evaluations.

full_covariance_drivers

Full-covariance Gaussian VI wrappers.

full_covariance_mf_drivers

Full-covariance multifidelity variational inference.

gains

Cross-fitted ACV gains from cached score/weight moments.

gaussian

Utilities for full-covariance Gaussian variational inference.

mf_drivers

Multifidelity variational inference drivers.

newton

Natural-coordinate Newton support for full-covariance Gaussian VI/MF-VI.

optimization

Optimization methods and configs used by inverse VI workflows.

rom_substeps

ROM-only optimizer substeps for multifidelity variational inference.

sample_reuse

Importance-sampling sample reuse for VI and MFVI.