Quantization for Ryzen AI

Ryzen AI IPU best performances are achieved using quantized models. There are two different ways to quantize models for Ryzen AI IPU:

through Vitis AI Quantizer, used in Optimum’s RyzenAIOnnxQuantizer, which is designed for ONNX model quantization. Currently supports quantising timm models using dynamic and static quantization methods.
through Brevitas library, used in Optimum’s BrevitasQuantizer. Brevitas allows to quantize directly PyTorch models, which may be optionally exported to ONNX. This is recommended to quantize other models.

Quantization using RyzenAIOnnxQuantizer

🤗 Optimum AMD provides a Ryzen AI Quantizer that enables you to apply quantization on many models hosted on the Hugging Face Hub using the AMD Vitis AI Quantizer.

RyzenAI Quantizer provides an easy-to-use Post Training Quantization (PTQ) flow on the pre-trained model saved in the ONNX format. It generates a quantized ONNX model ready to be deployed with the Ryzen AI.

The Quantizer supports various configuration and functions to quantize models targeting for deployment on IPU_CNN, IPU_Transformer and CPU.

The RyzenAIOnnxQuantizer can be initialized using the from_pretrained method, either from a local model folder or a model hosted on Hugging Face Hub:

>>> from optimum.amd.ryzenai import RyzenAIOnnxQuantizer

>>> quantizer = RyzenAIOnnxQuantizer.from_pretrained("path/to/model")

Below you will find an easy end-to-end example on how to quantize a VGG model from Timm library.

To begin, export the VGG model to ONNX using Optimum Exporters. Ensure static shapes are specified for inference.
Create a preprocessing function to handle specific image format conversions and apply necessary transformations to prepare the input for the model.
Initialize the RyzenAI quantizer (RyzenAIOnnxQuantizer) and configure the quantization settings using AutoQuantizationConfig. The recommended quantization configuration for CNN models to be deployed on the IPU is loaded using ipu_cnn_config.
Obtain a calibration dataset using the quantizer’s get_calibration_dataset method. This dataset is crucial for computing quantization parameters during the quantization process.
Run the quantizer with the specified quantization configuration and calibration data. The quantization parameters computed during this process are embedded as constants in the quantized model.
The resulting quantized model is saved in the specified quantization directory.

>>> from functools import partial
>>> import timm

>>> from optimum.amd.ryzenai import AutoQuantizationConfig, RyzenAIOnnxQuantizer
>>> from optimum.exporters.onnx import main_export
>>> from transformers import PretrainedConfig

>>> # Define paths for exporting ONNX model and saving quantized model
>>> export_dir = "/path/to/vgg_onnx"
>>> quantization_dir = "/path/to/vgg_onnx_quantized"

>>> # Specify the model ID from Timm
>>> model_id = "timm/vgg11.tv_in1k"

>>> # Step 1: Export the model to ONNX format using Optimum Exporters
>>> main_export(
...     model_name_or_path=model_id,
...     output=export_dir,
...     task="image-classification",
...     opset=13,
...     batch_size=1,
...     no_dynamic_axes=True,
... )

>>> # Step 2: Preprocess configuration and data transformations
>>> config = PretrainedConfig.from_pretrained(export_dir)
>>> data_config = timm.data.resolve_data_config(pretrained_cfg=config.pretrained_cfg)
>>> transforms = timm.data.create_transform(**data_config, is_training=False)

>>> def preprocess_fn(ex, transforms):
...     image = ex["image"]
...     if image.mode == "L":
...       # Convert greyscale to RGB if needed
...       print("WARNING: converting greyscale to RGB")
...       image = image.convert("RGB")
...     pixel_values = transforms(image)
...     return {"pixel_values": pixel_values}

>>> # Step 3: Initialize the RyzenAIOnnxQuantizer with the exported model
>>> quantizer = RyzenAIOnnxQuantizer.from_pretrained(export_dir)

>>> # Step 4: Load recommended quantization config for model
>>> quantization_config = AutoQuantizationConfig.ipu_cnn_config()

>>> # Step 5: Obtain a calibration dataset for computing quantization parameters
>>> train_calibration_dataset = quantizer.get_calibration_dataset(
...     "imagenet-1k",
...     preprocess_function=partial(preprocess_fn, transforms=transforms),
...     num_samples=100,
...     dataset_split="train",
...     preprocess_batch=False,
...     streaming=True,
... )

>>> # Step 6: Run the quantizer with the specified configuration and calibration data
>>> quantizer.quantize(
...     quantization_config=quantization_config,
...     dataset=train_calibration_dataset,
...     save_dir=quantization_dir
... )

Quantization using BrevitasQuantizer

Coming soon.

< > Update on GitHub