Base Module

class ablinfer.DispatchBase(config: Mapping[KT, VT_co] = None)

Base class for dispatching to models.

This class cannot actually be instantiated; users are intended to instantiate a subclass that dispatches using the appropriate method (e.g. ablinfer.docker.DispatchDocker).

The general procedure when running is:

1. _validate_config()
2. _validate_model_config()
3. _make_fmap()
4. _make_flags()
5. _make_command()
6. _run_processing() (dispatches required nodes to _clone() to copy them)
7. _save_input()
8. _run_command()
9. _load_output()
10. _run_processing()
11. _cleanup()

Functions are expected to raise exceptions if they encounter any unfixable errors. If an exception is raised, _cleanup_all() will be called.

run(model: Mapping[KT, VT_co], model_config: Mapping[KT, VT_co], progress: Callable[[ablinfer.constants.DispatchStage, float, float, str], None] = <built-in function print>) → None

Run the model.

This is the entry point

Parameters:

• model – The model specification.
• model_config – The model configuration.
• progress – an optional function accepting a DispatchStage, a float on [0,1] representing progress in the current stage, a float on [0,1] representing progress in the current operation, and a string with detailed info.

The following functions make up the actual functionality of the class, and at least the abstract ones must be overriden in derivative classes.

_validate_config() → None

Validate the non-model configuration.

_validate_model_config() → None

Validate the model configuration.

This first normalizes the model config, which also validates it. It then ensures that no inputs are null, no two outputs are the same, and no outputs are the same as inputs.

Note that when overriding this method without delegating here, the new method must normalize the model config by calling :func:ablinfer.model.normalize_model_config.

_make_fmap() → Mapping[str, str]

Make the file map.

Returns:The created filemap, mapping the input/output name to its path as seen by whatever the command will be dispatched to (e.g. for Docker, this would be the path inside the container).

_make_fmap_helper(actual_path: str) → Mapping[str, str]

_make_flags(fmap: Mapping[str, str]) → List[str]

Construct the flags to pass to the command.

Parameters:fmap – The generated fmap.

_make_command(flags: List[str]) → List[str]

Make the final command.

Parameters:flags – A list of the flags to be included. Returns:The final command, to be passed to _run_command_().

_run_processing(inp: bool) → None

Run pre/post processing.

Parameters:inp – Whether to do the input or output section.

_save_input(fmap: Mapping[str, str]) → None

Save the input files at the appropriate locations for running the model.

See the ablinfer.docker.DispatchDocker class for an example. This function will need to be overridden to reflect whatever node format your subclass uses.

_run_command(cmd: List[str]) → None

Run the actual command.

Must raise DispatchException if the called process runs into a problem.

Parameters:cmd – The command to execute.

_load_output(fmap: Mapping[str, str]) → None

Put the output files into the appropriate locations.

It is this function’s responsibility to set self._output_files as the files are created, so that they can be removed properly if the command fails.

Parameters:fmap – The file map.

_cleanup(error: Exception = None) → None

Run successful cleanup.

This function should conduct all of the cleanup necessary for a successful run. This will also be called if the run fails. This means that this function should run checks of its own to see what actually needs to be cleaned up.

Parameters:error – The exception, if one occurred.

_cleanup_all(error: Exception) → None

Cleanup everything.

This will only be called if an error is raised.

Parameters:error – The exception that occurred.

exception ablinfer.DispatchException(msg, stderr: str = None)

To be raised if a process runs into a problem.

Parameters:stderr – The stderr output of the process, if any.