Rspress

# #Eager Mode

The horizon_plugin_pytorch currently supports quantization using the eager mode, however, we no longer recommend using this mode. For quantization in eager mode, currently you are still supported by our horizon_plugin_pytorch. The overall flow of the Eager mode is also based on PyTorch's quantization interface and ideas, so we recommend that you read the Eager mode section of the PyTorch official document.

## #Difference from fx mode

The main differences between using eager mode and fx mode in horizon_plugin_pytorch are:

- The eager mode only supports module operators. Before carryout model quantization, you need to manually replace the functional operators in the floating-point model with Module-type operators in PyTorch or proprietary operators defined in horizon_plugin_pytorch, including but not limited to:

Original floating-point operator | Operator to be replaced
--- | ---
torch.nn.functional.relu | torch.nn.ReLU()
a + b / torch.add | horizon.nn.quantized.FloatFunctional().add
Tensor.exp | horizon.nn.Exp()
torch.nn.functional.interpolate | horizon.nn.Interpolate()

- You must manually define the operators to be fused and explicitly call the fusion function before carryout model quantization, and also specify the use of the `fuser_func` provided in horizon_plugin_pytorch when calling it. As shown below:

`import torch from torch import nn import horizon_plugin_pytorch as horizon class ConvBNReLU(nn.Sequential): def __init__(self, in_channels, out_channels, kernel_size): super(ConvBNReLU, self).__init__( nn.Conv2d( in_channels=in_channels, out_channels=out_channels, kernel_size=kernel_size ), nn.BatchNorm2d(num_features=out_channels), nn.ReLU() ) # specify the operators that can be fuse def fuse_model(self): torch.quantization.fuse_modules( self, ['0', '1', '2'], inplace=True, # specify the fuse function provided in horizon_plugin_pytorch fuser_func=horizon.quantization.fuse_known_modules, ) float_model = ConvBNReLU(1, 1, 1) # call the fuse function explicitly float_model.fuse_model() print(float_model) # ConvBNReLU( # (0): ConvReLU2d( # (0): Conv2d(1, 1, kernel_size=(1, 1), stride=(1, 1)) # (1): ReLU() # ) # (1): Identity() # (2): Identity() # )`

- The eager mode cannot use `QconfigSetter` because there is no computation graph, and the qconfig can only be configured by setting attributes.

  `model.qconfig = QConfig(...)`

On This Page