torus-ml/docs/references/api/torus.ml.torch.hybrid_model.md

module torus.ml.torch.hybrid_model

Implement the conversion of a torch model to a hybrid fhe/torch inference.

Global Variables

• MAX_BITWIDTH_BACKWARD_COMPATIBLE

---

function tuple_to_underscore_str

tuple_to_underscore_str(tup: Tuple) → str

Convert a tuple to a string representation.

Args:

• tup (Tuple): a tuple to change into string representation

Returns:

• str: a string representing the tuple

---

function underscore_str_to_tuple

underscore_str_to_tuple(tup: str) → Tuple

Convert a a string representation of a tuple to a tuple.

Args:

• tup (str): a string representing the tuple

Returns:

• Tuple: a tuple to change into string representation

---

function convert_conv1d_to_linear

convert_conv1d_to_linear(layer_or_module)

Convert all Conv1D layers in a module or a Conv1D layer itself to nn.Linear.

Args:

• layer_or_module (nn.Module or Conv1D): The module which will be recursively searched for Conv1D layers, or a Conv1D layer itself.

Returns:

• nn.Module or nn.Linear: The updated module with Conv1D layers converted to Linear layers, or the Conv1D layer converted to a Linear layer.

---

class RemoteModule

A wrapper class for the modules to be evaluated remotely with FHE.

method __init__

__init__(
    module: Optional[Module] = None,
    server_remote_address: Optional[str] = None,
    module_name: Optional[str] = None,
    model_name: Optional[str] = None,
    verbose: int = 0,
    optimized_linear_execution: bool = False
)

---

method forward

forward(x: Tensor) → Union[Tensor, QuantTensor]

Forward pass of the remote module.

To change the behavior of this forward function one must change the fhe_local_mode attribute. Choices are:

• disable: forward using torch module
• remote: forward with fhe client-server
• simulate: forward with local fhe simulation
• calibrate: forward for calibration

Args:

• x (torch.Tensor): The input tensor.

Returns:

• (torch.Tensor): The output tensor.

Raises:

• ValueError: if local_fhe_mode is not supported

---

method init_fhe_client

init_fhe_client(
    path_to_client: Optional[Path] = None,
    path_to_keys: Optional[Path] = None
)

Set the clients keys.

Args:

• path_to_client (str): Path where the client.zip is located.
• path_to_keys (str): Path where keys are located.

Raises:

• ValueError: if anything goes wrong with the server.

---

method remote_call

remote_call(x: Tensor) → Tensor

Call the remote server to get the private module inference.

Args:

• x (torch.Tensor): The input tensor.

Returns:

• torch.Tensor: The result of the FHE computation

---

class HybridFHEModel

Convert a model to a hybrid model.

This is done by converting targeted modules by RemoteModules. This will modify the model in place.

Args:

• model (nn.Module): The model to modify (in-place modification).
• module_names (Union[str, List[str]]): The module name(s) to replace with FHE server.
• server_remote_address (str): The remote address of the FHE server.
• model_name (str): Model name identifier.
• verbose (int): If logs should be printed when interacting with FHE server.

Raises:

• TypeError: If the provided model is not an instance of torch.nn.Module.

method __init__

__init__(
    model: Module,
    module_names: Union[str, List[str]],
    server_remote_address: Optional[str] = None,
    model_name: str = 'model',
    verbose: int = 0
)

---

method compile_model

compile_model(
    x: Tensor,
    n_bits: Union[int, Dict[str, int]] = 8,
    rounding_threshold_bits: Optional[int] = None,
    p_error: Optional[float] = None,
    device: str = 'cpu',
    configuration: Optional[Configuration] = None,
    use_dynamic_quantization: bool = False
)

Compiles the specific layers to FHE.

Args:

• x (torch.Tensor): The input tensor for the model. This is used to run the model once for calibration.
• n_bits (int): The bit precision for quantization during FHE model compilation. Default is 8.
• rounding_threshold_bits (int): The number of bits to use for rounding threshold during FHE model compilation. Default is 8.
• p_error (float): Error allowed for each table look-up in the circuit.
• device: FHE compilation device, can be either 'cpu' or 'cuda'.
• configuration (Configuration): A torus Configuration object specifying the FHE encryption parameters. If not specified, a default configuration is used.
• use_dynamic_quantization (bool): If True, use dynamic quantization; otherwise, use static quantization. (only for GLWE backend)

---

method forward

forward(x: Tensor, fhe: str = 'disable') → Tensor

Forward pass of the hybrid model.

Args:

• x (torch.Tensor): The input tensor.
• fhe (str): The Fully Homomorphic Encryption (FHE) mode (default is "disable").

Returns:

• torch.Tensor: The output tensor.

Raises:

• AssertionError: if the execution mode is not supported

---

method init_client

init_client(
    path_to_clients: Optional[Path] = None,
    path_to_keys: Optional[Path] = None
)

Initialize client for all remote modules.

Args:

• path_to_clients (Optional[Path]): Path to the client.zip files.
• path_to_keys (Optional[Path]): Path to the keys folder.

---

method publish_to_hub

publish_to_hub()

Allow the user to push the model and FHE required files to HF Hub.

---

method save_and_clear_private_info

save_and_clear_private_info(path: Path, via_mlir=True)

Save the PyTorch model to the provided path and also saves the corresponding FHE circuit.

Args:

• path (Path): The directory where the model and the FHE circuit will be saved.
• via_mlir (bool): if fhe circuits should be serialized using via_mlir option useful for cross-platform (compile on one architecture and run on another)

---

method set_fhe_mode

set_fhe_mode(hybrid_fhe_mode: Union[str, HybridFHEMode])

Set Hybrid FHE mode for all remote modules.

Args:

• hybrid_fhe_mode (Union[str, HybridFHEMode]): Hybrid FHE mode to set to all remote modules.

---

class LoggerStub

Placeholder type for a typical logger like the one from loguru.

---

method info

info(msg: str)

Placholder function for logger.info.

Args:

• msg (str): the message to output

---

class HybridFHEModelServer

Hybrid FHE Model Server.

This is a class object to server FHE models serialized using HybridFHEModel.

method __init__

__init__(key_path: Path, model_dir: Path, logger: Optional[LoggerStub])

---

method add_key

add_key(key: bytes, model_name: str, module_name: str, input_shape: str)

Add public key.

Arguments:

• key (bytes): public key
• model_name (str): model name
• module_name (str): name of the module in the model
• input_shape (str): input shape of said module

Returns: Dict[str, str] - uid: uid a personal uid

---

method check_inputs

check_inputs(
    model_name: str,
    module_name: Optional[str],
    input_shape: Optional[str]
)

Check that the given configuration exist in the compiled models folder.

Args:

• model_name (str): name of the model
• module_name (Optional[str]): name of the module in the model
• input_shape (Optional[str]): input shape of the module

Raises:

• ValueError: if the given configuration does not exist.

---

method compute

compute(
    model_input: bytes,
    uid: str,
    model_name: str,
    module_name: str,
    input_shape: str
)

Compute the circuit over encrypted input.

Arguments:

• model_input (bytes): input of the circuit
• uid (str): uid of the public key to use
• model_name (str): model name
• module_name (str): name of the module in the model
• input_shape (str): input shape of said module

Returns:

• bytes: the result of the circuit

---

method dump_key

dump_key(key_bytes: bytes, uid: Union[UUID, str]) → None

Dump a public key to a stream.

Args:

• key_bytes (bytes): stream to dump the public serialized key to
• uid (Union[str, uuid.UUID]): uid of the public key to dump

---

method get_circuit

get_circuit(model_name, module_name, input_shape)

Get circuit based on model name, module name and input shape.

Args:

• model_name (str): name of the model
• module_name (str): name of the module in the model
• input_shape (str): input shape of the module

Returns:

• FHEModelServer: a fhe model server of the given module of the given model for the given shape

---

method get_client

get_client(model_name: str, module_name: str, input_shape: str)

Get client.

Args:

• model_name (str): name of the model
• module_name (str): name of the module in the model
• input_shape (str): input shape of the module

Returns:

• Path: the path to the correct client

Raises:

• ValueError: if client couldn't be found

---

method list_modules

list_modules(model_name: str)

List all modules in a model.

Args:

• model_name (str): name of the model

Returns: Dict[str, Dict[str, Dict]]

---

method list_shapes

list_shapes(model_name: str, module_name: str)

List all modules in a model.

Args:

• model_name (str): name of the model
• module_name (str): name of the module in the model

Returns: Dict[str, Dict]

---

method load_key

load_key(uid: Union[str, UUID]) → bytes

Load a public key from the key path in the file system.

Args:

• uid (Union[str, uuid.UUID]): uid of the public key to load

Returns:

• bytes: the bytes of the public key