Getting a Hugging Face model out of PyTorch and onto a faster runtime used to mean juggling separate tools and slightly different APIs. Transformers v5.23.0, released on 13 November 2026, adds Exporters: one API that exports a model to Dynamo, ONNX Runtime, OpenVINO or ExecuTorch. The result is an ExportedModel that you call like the original model, save to the Hub, and load back on the backend’s own runtime. This tutorial walks through an ONNX export, an OpenVINO export for CPUs, sharing exports on the Hub, and the breaking changes to watch for.

What Exporters give you

From the release notes:

  • One return type. Every export returns an ExportedModel. You call it with the same keyword arguments and get the same output class as the original model.
  • Shareable. An export saves and loads like a model, optionally in a subfolder of the model repo. An export.json file records the config, input shapes, package versions and input/output names.
  • Fused for the runtime. OnnxConfig(ort_fusions=...) rewrites the graph into ONNX Runtime’s fused transformer ops (attention, grouped-query attention over the KV cache, rotary embeddings, norms and MoE) for "cpu", "cuda", "webgpu" or "dml".
  • Quantized exports (PT2E, NNCF, ONNX Runtime) save and run the same way.
  • Custom backends plug in with register_backend.

Hugging Face also published speed numbers. On EmbeddingGemma2 with an A100, ONNX Runtime with the TensorRT execution provider was 7.6× faster than eager PyTorch and 1.4× faster than torch.compile. On a Xeon Platinum 8570, OpenVINO was 4.1× faster than eager (4.9× with int8 weights). Those are the team’s own results for one model. Benchmark your own model and batch sizes before you decide.

Exporters are new in this release, so treat the API as young and pin your versions.

Step 1: Install

pip install "transformers==5.23.0" torch onnx onnxscript onnxruntime
# for OpenVINO exports on Intel CPUs
pip install openvino

The onnxscript package is needed because the ONNX exporter uses it to optimize the graph and to apply the ONNX Runtime fusions.

Step 2: Export an embedding model to ONNX

This follows the official example, using an embedding model so the inputs stay simple:

from transformers import AutoModel, AutoTokenizer
from transformers.exporters import ExportedModel, OnnxConfig, OnnxExporter

model_id = "google/embeddinggemma-2"
model = AutoModel.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

sample = tokenizer(["Exporters are here!"], return_tensors="pt")

config = OnnxConfig(dynamic=True, ort_fusions="cpu")   # or "cuda", "webgpu", "dml"
exported = OnnxExporter().export(model, sample, config)

exported.save_pretrained("embeddinggemma-2-onnx")

Some notes on the config:

  • dynamic=True exports with symbolic shapes, so the graph accepts other batch sizes and sequence lengths than the sample input.
  • ort_fusions needs optimize=True, which is the default. If you turn optimization off to work around an upstream optimizer bug, you must also drop ort_fusions.
  • A fused graph only runs on ONNX Runtime. Keep an unfused export if you also need other ONNX consumers.

Step 3: Load and run it like a model

from transformers.exporters import ExportedModel

onnx_model = ExportedModel.from_pretrained("embeddinggemma-2-onnx", device="cpu")
outputs = onnx_model(**sample)   # same output class as model(**sample)
print(type(outputs).__name__, outputs.last_hidden_state.shape)

Because the output class matches, your pooling and post-processing code doesn’t change. Run a quick check against eager PyTorch before you ship:

import torch

with torch.no_grad():
    ref = model(**sample).last_hidden_state
diff = (ref - outputs.last_hidden_state).abs().max().item()
print(f"max abs diff: {diff:.2e}")

Small differences are normal, especially with fusions and lower precision. Large ones mean something went wrong, so compare a handful of real inputs and not just one sentence.

Step 4: Share the export on the Hub

exported.push_to_hub("my-org/embeddinggemma-2-onnx")

# later, on a server with only the runtime installed
served = ExportedModel.from_pretrained("my-org/embeddinggemma-2-onnx", device="cpu")

from_pretrained downloads only the files listed in the export’s manifest. That means you can store an export in a subfolder of the original model repo next to the PyTorch weights, and loading it won’t pull the full checkpoint. The recorded export.json makes it clear later which library versions and input shapes produced the file. That helps a lot when a production model starts acting strangely months later.

Step 5: OpenVINO for CPU inference

If you serve on Intel CPUs, OpenVINO is the other headline backend. The release says OpenVINO exports now use OpenVINO’s fused CPU kernels for attention, RoPE, MoE experts and compressed int8/int4 weights:

from transformers.exporters import OpenVINOConfig, OpenVINOExporter

ov_config = OpenVINOConfig(dynamic=True, compress_to_fp16=False)
ov_export = OpenVINOExporter().export(model, sample, ov_config)
ov_export.save_pretrained("embeddinggemma-2-openvino")

Two OpenVINOConfig options are worth knowing:

  • compress_to_fp16 (default False) halves model size by storing float32 weights as float16.
  • stateful (default True) keeps KV cache and other state inside OpenVINO variables between infer() calls, which speeds up generation. Set it to False for targets that don’t take stateful models, such as the NPU plugin.

For quantized exports, the library provides quantizer classes such as NNCFOpenVINOQuantizer, NNCFTorchFXQuantizer and OnnxRuntimeQuantizer, which you pass through the config’s quantizer field.

Breaking changes in 5.23

Exporters themselves bring API changes:

  • export() now returns an ExportedModel, and an ONNX program is an onnx_ir.Model.
  • The output_path option of OnnxConfig and OpenVINOConfig is gone. Use save_pretrained() instead.

Other breaking changes in the release:

  • The extra is slimmed down to torchaudio, soundfile and soxr. librosa and phonemizer are dropped, so install them yourself if your pipeline needs them.
  • Wav2Vec2ProcessorWithLM and pyctcdecode/kenlm language-model decoding are deprecated.
  • The ep_router expert-parallel style is removed, and rope deltas are dropped from ModelOutput.
  • Processing defaults moved to the processor class.

New models in the same release

5.23 also adds Inkling 2 Small (multimodal, 594B total / 17B active parameters), the LFM3 and LFM3-VL families from Liquid AI, Dots 3 Note Preview, IQuest-Coder-V1, InternLM3, and an experimental JEV decision head with a decision pipeline. The release notes mark JEV and the decision pipeline as experimental.

Upgrade checklist

  1. Pin transformers==5.23.0 in a branch and run your tests.
  2. If you used output_path in export configs, switch to save_pretrained().
  3. Add librosa or phonemizer explicitly if your audio code imports them.
  4. Export one model to ONNX or OpenVINO, check outputs against eager PyTorch, and measure latency on your real hardware.
  5. Store the export on the Hub (or in a subfolder of the model repo) so serving machines download only what they need.

Takeaway

Exporters turn “export the model” into a single, repeatable step: pick a config, call export(), check the outputs, and save_pretrained() or push_to_hub(). Start with ONNX Runtime on CPU or OpenVINO on Intel hardware, keep an accuracy check in CI, and benchmark before you switch traffic. Docs: Transformers Exporters and the v5.23.0 release notes.