zeekay
e037e8f7ea
The distribution called itself torus-ml while shipping a package directory named concrete, so the name a user installs and the name they import disagreed — and most of the tree already said torus.ml, which could not resolve against src/concrete. Moved src/concrete to src/torus and pointed the include at it. The dependency was Zama's concrete-python from their index. It is torus-fhe from ours, which is what luxfhe/torus already ships: that frontend declares torus-fhe and ships a torus/fhe module, so every 'from torus.fhe' in this tree was already naming our compiler and only the dependency lagged. Two distributions share one namespace, which is the shape the upstream this forked from used and the reason the split works: torus-fhe gives torus.fhe, torus-ml gives torus.ml. Swept every other spelling — identifiers, enum members, workflow variables, filenames, notebooks. LICENSE and NOTICE keep the upstream attribution a fork owes; the figures carry the old name in JPEG metadata a text sweep cannot reach.
5.6 KiB
module torus.ml.common.serialization.encoder
Custom encoder for serialization.
Global Variables
- INFINITY
- USE_SKOPS
function dump_name_and_value
dump_name_and_value(name: str, value: Any, **kwargs) → Dict
Dump the value into a custom dict format.
Args:
name(str): The custom name to use. This name should be unique for each type to encode, as it is used in the TorusDecoder class to detect the initial type and apply the proper load method to the serialized object.value(Any): The serialized value to dump.**kwargs (dict): Additional arguments to dump.
Returns:
Dict: The serialized custom format that includes both the serialized value and its type name.
class TorusEncoder
Custom json encoder to handle non-native types found in serialized Torus ML objects.
Non-native types are serialized manually and dumped in a custom dict format that stores both the serialization value of the object and its associated type name.
The name should be unique for each type, as it is used in the TorusDecoder class to detect the initial type and apply the proper load method to the serialized object. The serialized value is the value that was serialized manually in a native type. Additional arguments such as a numpy array's dtype are also properly serialized. If an object has an unexpected type or is not serializable, an error is thrown.
The TorusEncoder is only meant to encode Torus ML's built-in models and therefore only supports the necessary types. For example, torch.Tensor objects are not serializable using this encoder as built-in models only use numpy arrays. However, the list of supported types might expand in future releases if new models are added and need new types.
method default
default(o: Any) → Any
Define a custom default method that enables dumping any supported serialized values.
Arguments:
o(Any): The object to serialize.
Returns:
Any: The serialized object. Non-native types are returned as a dict of a specific format.
Raises:
NotImplementedError: If an FHE.Circuit, a Callable or a Generator object is given.
method isinstance
isinstance(o: Any, cls: Type) → bool
Define a custom isinstance method.
Natively, among other types, the JSONENcoder handles integers, floating points and tuples. However, a numpy.integer (resp. numpy.floating) object is automatically casted to a built-in int (resp. float) object, without keeping their dtype information. Similarly, a tuple is casted to a list, meaning that it will then be loaded as a list, which notably does not have the uniqueness property and therefore might cause issues in complex structures such as QuantizedModule instances. This is an issue as JSONEncoder only calls its customizable default method at the end of the parsing. We thus need to provide this custom isinstance method in order to make the encoder avoid handling these specific types until default is reached (where they are properly serialized using our custom format).
Args:
o(Any): The object to serialize.cls(Type): The type to compare the object with.
Returns:
bool: If the object is of the given type. False if it is a numpy.floating, numpy.integer or a tuple.
method iterencode
iterencode(o: Any, _one_shot: bool = False) → Generator
Encode the given object and yield each string representation as available.
This method overrides the JSONEncoder's native iterencode one in order to pass our custom isinstance method to the _make_iterencode function. More information in isinstance's docstring. For simplicity, iterencode does not give the ability to use the initial c_make_encoder function, as it would required to override it in C.
Args:
o(Any): The object to serialize._one_shot(bool): This parameter is not used since the_make_iterencodefunction has been removed from the method.
Returns:
Generator: Yield each string representation as available.