romtools.hpc.dispatchers.remote_dispatcher#

Functions

run_local_bash(cmd)

Classes

RemoteDispatcher([sampling_directory, ...])

Main class of ROM's HPC tools.
class romtools.hpc.dispatchers.remote_dispatcher.RemoteDispatcher(sampling_directory='hpctools', logger=None, connection=None)[source]#

Bases: BaseDispatcher

Main class of ROM’s HPC tools. Establishes SSH connection to remote host, dispatches desired workflows, and transfers results back to the local machine.

Parameters:

  • logger (Logger) – An instance of the Logger class for logging

  • sampling_directory (str) – An optional string for your local output directory

  • connection (Connection | None)

The basic command is therefore:

ssh user@remote -p port

dispatch(cmd=None, run_directory=None, with_slurm=True)[source]#

Main method of the Dispatcher. Dispatches provided work to the remote host, polls the job, and collects results.

Parameters:

  • cmd (str) – The command to run in the SLURM job. Should be executable from self.config.get(“remote_root”) or the specified run_directory. If not provided, dispatcher must be configured with a SLURM script that includes the command to run.

  • run_directory (str) – The directory in which to execute the command or SLURM job on the remote host. If not provided, defaults to self.config.get(“remote_root”).

  • with_slurm (bool) – If True, the command will be run as a SLURM job. If False, the command will be executed directly without SLURM.

Return type:

Result

Returns the SLURM job exit code in sacct format of 0:0, or “No SLURM JOB submitted.”.

np_savez(path, **arrays)[source]#
Write multiple arrays to a .npz file.

  • If a connection exists, the .npz file is first written to a local temp directory and then uploaded to the remote host.

  • If no connection exists, the .npz file is written directly to the specified path.

Parameters:

path (str)

Return type:

None