romtools.hpc.dispatchers#

class romtools.hpc.dispatchers.LocalDispatcher(sampling_directory='hpctools', logger=None)[source]#

Bases: BaseDispatcher

LocalDispatcher is a subclass of BaseDispatcher that implements the core functionality for dispatching ROM workflows on the local machine. It overrides methods to set up directories and execute commands without SSH, making it suitable for local execution.

Parameters:

  • sampling_directory (str)

  • logger (Logger)

dispatch(cmd, run_directory=None)[source]#
Returns:

sacct format string of job exit code + linux signal number (always 0 in local) example: ‘0:0’

Parameters:

  • cmd (str)

  • run_directory (str)

Return type:

Result

get(remote_path, local_path)[source]#

Local ‘get’ is just a copy from remote_path to local_path.

Parameters:

  • remote_path (str)

  • local_path (str)

Return type:

None

np_savez(path, **arrays)[source]#

Write multiple arrays to a .npz file. The .npz file is written directly to the specified path.

Parameters:

path (str)

Return type:

None

put(local_path, remote_path)[source]#

Local ‘put’ is just a copy from local_path to remote_path.

Parameters:

  • local_path (str)

  • remote_path (str)

Return type:

None

class romtools.hpc.dispatchers.RemoteDispatcher(sampling_directory='hpctools', logger=None, connection=None)[source]#

Bases: BaseDispatcher

Main class of ROM’s HPC tools. Establishes SSH connection to remote host, dispatches desired workflows, and transfers results back to the local machine.

Parameters:

  • logger (Logger) – An instance of the Logger class for logging

  • sampling_directory (str) – An optional string for your local output directory

  • connection (Connection | None)

The basic command is therefore:

ssh user@remote -p port

dispatch(cmd=None, run_directory=None, with_slurm=True)[source]#

Main method of the Dispatcher. Dispatches provided work to the remote host, polls the job, and collects results.

Parameters:

  • cmd (str) – The command to run in the SLURM job. Should be executable from self.config.get(“remote_root”) or the specified run_directory. If not provided, dispatcher must be configured with a SLURM script that includes the command to run.

  • run_directory (str) – The directory in which to execute the command or SLURM job on the remote host. If not provided, defaults to self.config.get(“remote_root”).

  • with_slurm (bool) – If True, the command will be run as a SLURM job. If False, the command will be executed directly without SLURM.

Return type:

Result

Returns the SLURM job exit code in sacct format of 0:0, or “No SLURM JOB submitted.”.

np_savez(path, **arrays)[source]#
Write multiple arrays to a .npz file.

  • If a connection exists, the .npz file is first written to a local temp directory and then uploaded to the remote host.

  • If no connection exists, the .npz file is written directly to the specified path.

Parameters:

path (str)

Return type:

None

Modules

base_dispatcher

local_dispatcher

remote_dispatcher