torus/frontends/torus-python/examples/floating_point/floating_point.ipynb

Floating point management in Torus

In this tutorial, we are going to explain how to manage circuits with floating points.

As we explain in the documentation, TFHE operations are limited to integers. However, it is most of the time not a limit, since it is possible to turn floating-point operations into integer operations. This is what we are going to study in this tutorial.

In [1]:

import numpy as np
from torus import fhe
from time import time

from numpy.random import randint
from numpy.random import rand
from numpy import round

Starting with an integer circuit

Let's start with a very simple circuit, directly on integers. We'll take an example inspired by the README example.

In [2]:

nb_bits = 4
length_inputset = 100
nb_test_samples = 16


@fhe.compiler({"x": "encrypted", "y": "encrypted"})
def add_integers(x, y):
    return x + y


# Compile
inputset = [(randint(2**nb_bits), randint(2**nb_bits)) for _ in range(length_inputset)]
circuit = add_integers.compile(inputset)

# Check
time_begin = time()

for _ in range(nb_test_samples):
    random_sample = (randint(2**nb_bits), randint(2**nb_bits))
    assert circuit.encrypt_run_decrypt(*random_sample) == add_integers(*random_sample)

print(
    "Compilation and test look good, with FHE execution time of about "
    f"{(time() - time_begin) / nb_test_samples:.2f} seconds per inference"
)

Compilation and test look good, with FHE execution time of about 0.00 seconds per inference

Here, we have defined a function add_integers, which takes 2 encrypted inputs and add them. We have compiled this function using an inputset of random inputs of nb_bits = 4 bits. At the end, we check the FHE execution, by comparing encrypt_run_decrypt execution (which conveniently pack encryption, FHE run and decryption together, for testing purposes) and the clear execution add_integers(*random_sample).

Let's even see the MLIR circuit

In [3]:

print(circuit.mlir)

module {
  func.func @main(%arg0: !FHE.eint<5>, %arg1: !FHE.eint<5>) -> !FHE.eint<5> {
    %0 = "FHE.add_eint"(%arg0, %arg1) : (!FHE.eint<5>, !FHE.eint<5>) -> !FHE.eint<5>
    return %0 : !FHE.eint<5>
  }
}

Here we see that inputs and outputs are considered as 5b integers

Trying to use it with floats

Now, let's try to use this circuit with floats. As you can imagine, it's not going to work.

In [4]:

random_sample = (1.5, 2.42)
try:
    circuit.encrypt_run_decrypt(*random_sample)
except Exception as err:
    print(err)
    pass

Expected argument 0 to be EncryptedScalar<uint5> but it's EncryptedScalar<float64>

It raises an error ValueError: Expected argument 0 to be EncryptedScalar<uint5> but it's EncryptedScalar<float64>. Let's see now how to deal with this situation.

Creating a circuit for floats

What we are going to do is:

• chose a scaling factor
• multiply floats by this scaling factor
• round to integer
• encrypt
• make FHE computations over integers, classically
• decrypt the result
• unscale

scaling_factor = 100

Here, we chose a scaling factor which is a power of 10, to be easier to read, but it could be anything, including a float value.

In [5]:

max_value_for_floats = 2.5
scaling_factor = 100


@fhe.compiler({"x": "encrypted", "y": "encrypted"})
def add_integers(x, y):
    return x + y


def are_almost_the_same(a, b, threshold):
    abs_diff = abs(b - a)
    assert abs_diff <= threshold, f"Too far {a=} {b=} {abs_diff=} > {threshold=}"


# Compile
inputset = [
    (
        round(scaling_factor * rand() * max_value_for_floats).astype(np.uint32),
        round(scaling_factor * rand() * max_value_for_floats).astype(np.uint32),
    )
    for _ in range(length_inputset)
]
circuit = add_integers.compile(inputset)

# Check
verbose = True
time_begin = time()

for _ in range(nb_test_samples):
    # Take a random float input
    random_sample = (rand() * max_value_for_floats, rand() * max_value_for_floats)
    if verbose:
        print("Showing an example")
        print(f"{random_sample=}")

    # Scale it and round
    scaled_sample = (
        round(random_sample[0] * scaling_factor).astype(np.uint32),
        round(random_sample[1] * scaling_factor).astype(np.uint32),
    )
    if verbose:
        print(f"{scaled_sample=}")

    # Encrypt
    encrypted_scaled_sample = circuit.encrypt(*scaled_sample)

    # Computations in FHE
    encrypted_scaled_result = circuit.run(*encrypted_scaled_sample)

    # Decrypt
    scaled_result = circuit.decrypt(encrypted_scaled_result)
    if verbose:
        print(f"{scaled_result=}")

    # Unscale
    result = scaled_result * 1.0 / scaling_factor

    bounded_error = 2.0 / scaling_factor
    clear_result = random_sample[0] + random_sample[1]

    if verbose:
        print()
        print(f"FHE:   {result}")
        print(f"clear: {clear_result}")
        print(f"bounded error: {bounded_error}")

    are_almost_the_same(result, clear_result, bounded_error)

    verbose = False

print(
    "\nCompilation and test look good, with FHE execution time of about "
    f"{(time() - time_begin) / nb_test_samples:.2f} seconds per inference"
)

Showing an example
random_sample=(1.583812144515164, 0.38770109380525164)
scaled_sample=(158, 39)
scaled_result=197

FHE:   1.97
clear: 1.9715132383204157
bounded error: 0.02

Compilation and test look good, with FHE execution time of about 0.00 seconds per inference

Here, our testing loop checks that the results are the expected ones, with a tolerance error of bounded_error = 2. / scaling_factor. We also see one example:

• random_sample is a pair of floats
• scaled_sample is this same pair multiplied by the scaling factor and rounded to integers
• scaled_result is the result of the FHE computation, over integers
• at the end, we see the FHE result and the clear result. They are close but not the same, due to rounding errors.

To have more precise computations, one needs to take a larger scaling factor. However, keep in mind it has impact on performances, since integers will be larger.

In [6]:

print(circuit.mlir)

module {
  func.func @main(%arg0: !FHE.eint<9>, %arg1: !FHE.eint<9>) -> !FHE.eint<9> {
    %0 = "FHE.add_eint"(%arg0, %arg1) : (!FHE.eint<9>, !FHE.eint<9>) -> !FHE.eint<9>
    return %0 : !FHE.eint<9>
  }
}

Here, we see that our integers in the circuits have 9 bits, which corresponds to code integers which are smaller than max_value_for_floats * scaling_factor = 250 plus 1 for the addition carry. Taking scaling_factor = 1000 for example would give more precise results (bounded_error reduces from 0.02 to 0.0002) but now the circuits would have integers of 13 bits.

As our circuits doesn't have programmable bootstrapping (PBS), it doesn't have a big impact, but if we had non linear operations (as we have in the following), the timing impact will be much more important.

More complex case with Programmable Bootstrapping

Let's do the same for computing the norm of a vector. Here, we'll have a square root so some PBS.

In [7]:

max_value_for_floats = 2.5
scaling_factor = 8
nb_test_samples = 10


@fhe.compiler({"x": "encrypted", "y": "encrypted"})
def norm(x, y):
    return np.round(fhe.univariate(lambda x: np.sqrt(x))(x**2 + y**2)).astype(np.int64)


# Compile
inputset = [
    (
        round(scaling_factor * rand() * max_value_for_floats).astype(np.uint32),
        round(scaling_factor * rand() * max_value_for_floats).astype(np.uint32),
    )
    for _ in range(length_inputset)
]
circuit = norm.compile(inputset, show_mlir=False)

print(f"Maximal bitwidth in the circuit: {circuit.graph.maximum_integer_bit_width()}\n")

# Check
verbose = True
time_begin = time()

for _ in range(nb_test_samples):
    # Take a random float input
    random_sample = (rand() * max_value_for_floats, rand() * max_value_for_floats)
    if verbose:
        print("Showing an example")
        print(f"{random_sample=}")

    # Scale it and round
    scaled_sample = (
        round(random_sample[0] * scaling_factor).astype(np.uint32),
        round(random_sample[1] * scaling_factor).astype(np.uint32),
    )
    if verbose:
        print(f"{scaled_sample=}")

    # Encrypt
    encrypted_scaled_sample = circuit.encrypt(*scaled_sample)

    # Computations in FHE
    encrypted_scaled_result = circuit.run(*encrypted_scaled_sample)

    # Decrypt
    scaled_result = circuit.decrypt(encrypted_scaled_result)
    if verbose:
        print(f"{scaled_result=}")

    # Unscale
    result = scaled_result * 1.0 / scaling_factor

    bounded_error = 2.0 / scaling_factor
    clear_result = np.sqrt(random_sample[0] ** 2 + random_sample[1] ** 2)

    if verbose:
        print()
        print(f"FHE:   {result}")
        print(f"clear: {clear_result}")
        print(f"bounded error: {bounded_error}")

    are_almost_the_same(result, clear_result, bounded_error)

    verbose = False

print(
    "\nCompilation and test look good, with FHE execution time of about "
    f"{(time() - time_begin) / nb_test_samples:.2f} seconds per inference"
)

Maximal bitwidth in the circuit: 10

Showing an example
random_sample=(0.6597925446542308, 2.2648103856817494)
scaled_sample=(5, 18)
scaled_result=19

FHE:   2.375
clear: 2.358960000736176
bounded error: 0.25

Compilation and test look good, with FHE execution time of about 11.16 seconds per inference

We can remark that here, the function is much slower, since it includes PBS. Furthermore, the input of the PBS is as large as twice the square of typical inputs, which can quickly become large. That's why we have had to reduce the scaling factor and the precision. When more precision is needed, one would have to be a bit more patient.

Final example

We finish with a more complex example, where one needs to understand what kind of computations are done and cancel some scaling factors during the computation, to ensure correct computations.

Let's suppose we want to convert f(x, y) = (x**2 + 4y) / 1.33 to FHE. What we'll do is that, we'll scale x and y with a scaling factor scaling_factor, before encryption. Then, in x**2, scaling factors will multiply (so we'll have a scaling factor of scaling_factor**2) while in 4y scaling factor would remain as scaling_factor. To keep the addition homogeneous, we have two possibilities:

• multiply 4y by scaling_factor in the FHE circuit, and at the end, unscale by dividing by scaling_factor**2
• or, divide x**2 by scaling_factorin the FHE circuit, and at the end, unscale by dividing by scaling_factor. We have chose the second approach, since it avoids to enlarge too much integers, making that our FHE execution is more efficient.

The error bound is more complicated to compute: we can use an estimation.

In [8]:

max_value_for_floats = 2
scaling_factor = 8
nb_test_samples = 8


def special_function_in_clear(x, y):
    u = x**2
    v = u + 4 * y
    return v / 1.33


@fhe.compiler({"x": "encrypted", "y": "encrypted"})
def special_function(x, y):
    u = fhe.univariate(lambda x: x**2 // scaling_factor)(x)
    v = u + 4 * y
    return np.round(fhe.univariate(lambda x: x / 1.33)(v)).astype(np.int64)


# Compile
inputset = [
    (
        round(scaling_factor * rand() * max_value_for_floats).astype(np.uint32),
        round(scaling_factor * rand() * max_value_for_floats).astype(np.uint32),
    )
    for _ in range(length_inputset)
]
circuit = special_function.compile(inputset, show_mlir=False)

print(f"Maximal bitwidth in the circuit: {circuit.graph.maximum_integer_bit_width()}\n")

# Check
verbose = True
time_begin = time()

for _ in range(nb_test_samples):
    # Take a random float input
    random_sample = (rand() * max_value_for_floats, rand() * max_value_for_floats)
    if verbose:
        print("Showing an example")
        print(f"{random_sample=}")

    # Scale it and round
    scaled_sample = (
        round(random_sample[0] * scaling_factor).astype(np.uint32),
        round(random_sample[1] * scaling_factor).astype(np.uint32),
    )
    if verbose:
        print(f"{scaled_sample=}")

    # Encrypt
    encrypted_scaled_sample = circuit.encrypt(*scaled_sample)

    # Computations in FHE
    encrypted_scaled_result = circuit.run(*encrypted_scaled_sample)

    # Decrypt
    scaled_result = circuit.decrypt(encrypted_scaled_result)
    if verbose:
        print(f"{scaled_result=}")

    # Unscale
    result = scaled_result * 1.0 / scaling_factor

    bounded_error = 3.0 / scaling_factor
    clear_result = special_function_in_clear(*random_sample)

    if verbose:
        print()
        print(f"FHE:   {result}")
        print(f"clear: {clear_result}")
        print(f"bounded error: {bounded_error}")

    are_almost_the_same(result, clear_result, bounded_error)

    verbose = False

print(
    "\nCompilation and test look good, with FHE execution time of about "
    f"{(time() - time_begin) / nb_test_samples:.2f} seconds per inference"
)

Maximal bitwidth in the circuit: 7

Showing an example
random_sample=(0.43374356260757363, 1.6491834206477578)
scaled_sample=(3, 13)
scaled_result=40

FHE:   5.0
clear: 5.10140388022146
bounded error: 0.375

Compilation and test look good, with FHE execution time of about 0.82 seconds per inference

Conclusion

As one knows, floats are not natively supported in TFHE and thus in Torus. However, we have shown in this tutorial that by scaling floats to integers, it's completely possible to make the computations in FHE. Finally, we'll add that such techniques, called quantization techniques, are directly integrated into TorusML for what is related to machine learning. More information are also given in the TorusML documentation, see https://docs.luxfhe.com/torus-ml/explanations/quantization.