How to convert a PyTorch model to ONNX

· Category: AI & Machine Learning

Short answer

ONNX is an open format for representing machine learning models that enables deployment across diverse frameworks and hardware accelerators.

Steps

  1. Put your PyTorch model in evaluation mode and define example input tensors.
  2. Call torch.onnx.export with the model, inputs, output path, and input names.
  3. Specify dynamic_axes if the batch size or sequence length varies at runtime.
  4. Verify the exported model using ONNX Runtime or the onnx checker.
  5. Optimize the graph with ONNX Runtime tools or convert to TensorRT for GPU inference.

Tips

  • Trace the model with representative inputs to avoid unsupported dynamic control flow.
  • Simplify the model by removing training-only layers before export.
  • Use opset versions compatible with your target inference engine.
  • Test numerical parity between PyTorch and ONNX outputs on a validation set.

Common issues

  • Unsupported operations in the ONNX opset causing export failures.
  • Dynamic shapes not declared leading to fixed batch sizes in inference.
  • Slight numerical differences between frameworks due to implementation details.
  • Complex model architectures with data-dependent control flow failing to trace.

Example

import torch

dummy_input = torch.randn(1, 3, 224, 224)
torch.onnx.export(model, dummy_input, 'model.onnx',
                  input_names=['input'], output_names=['output'],
                  dynamic_axes={'input': {0: 'batch_size'}})

This code exports a PyTorch model to ONNX with support for variable batch sizes, enabling deployment in diverse inference engines.