Exporters

Export any PreTrainedModel to ONNX, ExecuTorch, or a standalone PyTorch program, regardless of the target runtime.

exporter = DynamoExporter()  # or OnnxExporter, ExecutorchExporter
config = DynamoConfig(dynamic=True)
exported = exporter.export(model, inputs, config=config)

The exporters live inside Transformers instead of a downstream library, so architecture changes, new attention patterns, and custom cache types are supported at export time as soon as they land in the modeling code.

The exporters are experimental. Many of the patches in this module work around specific upstream bugs (Torch, ONNX Script, ONNX Runtime, ExecuTorch) and will be removed as soon as the fix lands upstream. Until the API stabilizes, treat the patches as tied to the versions used in the test suite. Pin those versions in production tooling, and expect new patches to appear and old ones to disappear as upstream changes land.

Exporter	Program (`.program`)	Saved as	Runtime
`DynamoExporter`	`ExportedProgram`	`model.pt2`	Any PyTorch runtime, AOT compilation
`OnnxExporter`	`onnx_ir.Model`	`model.onnx`	Any ONNX runtime (ORT, TensorRT, OpenVINO)
`ExecutorchExporter`	`ExecutorchProgramManager`	`model.pte`	Mobile and edge devices (ExecuTorch)
`OpenVINOExporter`	`openvino.Model`	`openvino_model.xml` + `.bin`	OpenVINO runtime (Intel CPU/GPU/NPU)

Every export returns an ExportedModel, whose components map names to ExportedComponents. A component holds the backend’s program (.program) and the ExportMetadata recorded while tracing it (input and output names, precision, traced input shapes, export config). ~HfExporter.export produces a single "model" component, so its program is directly exported.program. ~ExportedModel.save_pretrained writes one file per component, an export.json manifest with each component’s file and metadata, and the model config.

~ExportedModel.from_pretrained loads it back, from a local directory or a Hub repo, with the same download kwargs as from_pretrained(). Pass subfolder for an export kept next to the model’s weights; only the export’s own files are downloaded. Dynamo, ONNX, and OpenVINO programs load as the type the export returned. A loaded ONNX graph leaves its weights in model.onnx.data until they are read. An ExecuTorch .pte loads as an executorch.runtime.Program, since the ExecutorchProgramManager can’t be rebuilt from the file.

from transformers.exporters import ExportedModel

exported.program  # the backend's program
exported.save_pretrained("exported-model")  # model.<suffix>, export.json, config.json
loaded = ExportedModel.from_pretrained("exported-model")

A directory holds one export, since export.json and config.json have fixed names. Keep exports to several formats in their own subfolders (onnx/, openvino/) and load them with subfolder="onnx".

AutoHfExporter picks the right exporter from a config, and AutoExportConfig picks the right config class from a dict. Both follow the same auto-class pattern in Transformers, which is useful when the backend is selected at runtime instead of hardcoded at the call site.

from transformers.exporters import AutoExportConfig, AutoHfExporter

export_config_dict = {"export_format": "onnx", "dynamic": True}
config = AutoExportConfig.from_dict(export_config_dict)
exporter = AutoHfExporter.from_config(config)

exported = exporter.export(model, inputs, config=config)

Installation

Install the dependencies for the backend you plan to export to.

The versions below are the ones the exporter test suite is pinned against. Newer or older releases often work, but the exporter patches target a specific API surface, so for production tooling pin these and expect HfExporter to log a warning when it detects drift.

Dynamo

ONNX

ExecuTorch XNNPACK

ExecuTorch MLX

OpenVINO

Export a model

All exporters share the same interface. Create an exporter with a config, and call export().

Switch between runtimes by swapping the exporter class.

Dynamo

ONNX

ExecuTorch

OpenVINO

Run an export

An ExportedModel runs an export on its backend’s runtime (torch.export’s module, ONNX Runtime, OpenVINO, or the ExecuTorch runtime) and is called like the model it came from: same kwargs, same ModelOutput class. The compute runs in the backend, which is often faster than eager PyTorch (OpenVINO on CPU, ONNX Runtime on CUDA), and quantized exports run the same way. Inputs the export wasn’t traced with are dropped, so a processor’s whole output can be passed.

A fresh export runs on first call (on the backend’s default device, or the one ~ExportedModel.to picks), and ExportedModel.from_pretrained loads a saved one from a local directory or the Hub, ready to run.

from transformers.exporters import ExportedModel

outputs = exported(**inputs)

exported.save_pretrained("qwen3-onnx")
model = ExportedModel.from_pretrained("qwen3-onnx", device="cuda")
logits = model(**inputs).logits

device picks where it runs: an ONNX Runtime provider ("cpu", "cuda"), an OpenVINO device ("cpu", "xpu" for an Intel GPU, or AUTO by default), or where a torch.export module is moved. An ExecuTorch program runs where it was lowered for. Exports with several components (generation) don’t run yet.

Dynamic shapes

Passing dynamic=True marks every tensor dimension as dynamic so the exported graph accepts inputs of any size at runtime without retracing.

For fine-grained control over which dimensions are dynamic, pass explicit dynamic_shapes instead, which is forwarded directly to torch.export.export.

Dynamo

ONNX

ExecuTorch

OpenVINO

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.exporters import DynamoExporter, DynamoConfig

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
inputs = tokenizer(["Hello, world!", "Hi"], padding=True, return_tensors="pt")

batch = torch.export.Dim("batch", min=1, max=32)
seq = torch.export.Dim("seq", min=1, max=2048)

exporter = DynamoExporter()
config = DynamoConfig(
    dynamic_shapes={"input_ids": {0: batch, 1: seq}, "attention_mask": {0: batch, 1: seq}},
    # Emit data-dependent shape guards as runtime asserts instead of failing the export when a
    # guard wouldn't hold across the explicit symbolic range. Most LLMs need this under fine-grained
    # ``Dim(min=, max=)`` bounds. Not needed with ``dynamic=True`` / ``Dim.AUTO``, where torch.export
    # infers shape relations instead of verifying them against user-stated bounds.
    prefer_deferred_runtime_asserts_over_guards=True,
)
exported = exporter.export(model, inputs, config=config)

Generative models

For autoregressive generation, the model’s forward has different shapes at the prefill step (full prompt, no KV cache) versus the decode step (single token, populated KV cache). Exporters expose ~HfExporter.export_for_generation, which splits both stages and exports each.

For multi-modal generative models, the prefill additionally splits into an image or audio encoder, the language model, and lm_head. Encoder and language-model discovery uses get_encoder() (modality="image" or "audio") and get_decoder() accessors, so any new architecture using these work out of the box.

A projector component appears only when the model exposes one under an attribute name (multi_modal_projector, connector, embed_vision, embed_audio). Qwen2-VL below folds its projector into the vision tower, so its component dict has no separate multi_modal_projector key. New architectures must align their projector attribute to one of these names instead of growing the list.

Dynamo

ONNX

ExecuTorch

OpenVINO

from transformers import AutoModelForImageTextToText, AutoProcessor
from transformers.exporters import DynamoExporter, DynamoConfig

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen2-VL-2B-Instruct")
processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-2B-Instruct")
messages = [{"role": "user", "content": [{"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"}, {"type": "text", "text": "Describe this image."}]}]
text = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=text, images=messages[0]["content"][0]["url"], return_tensors="pt").to(model.device)

exporter = DynamoExporter()
config = DynamoConfig(dynamic=True)
exported = exporter.export_for_generation(model, inputs, config=config)
# exported: ExportedModel with "image_encoder", "language_model", "lm_head" and "decode",
# each an ExportedComponent whose `.program` is an ExportedProgram

The exported components are independent graphs, not a ready-to-run inference pipeline. The caller is responsible for running each encoder, projecting embeddings, and orchestrating the generation loop.

How export_for_generation works

decompose_for_generation() runs model.generate(**inputs, max_new_tokens=2) once and hooks model.forward to capture the real prefill and decode kwargs (and the per-submodule kwargs via hooks on each encoder/projector/language model if the model is multi-modal). That’s why it works for any architecture, including decoder-only, SSM, encoder-decoder, and multi-modal models, without per-model glue. export_for_generation is a one-liner over it.

The capture runs the model eagerly on inputs, so pass small but representative values, such as a short prompt, a single small image, or a few audio frames. The exported program isn’t tied to those sizes (dynamic shapes still flow through), but smaller capture inputs make decompose_for_generation cheaper and keep symbolic-shape inference tractable.

Call decompose_for_generation directly to act between decomposing and exporting, such as running an eager forward for verification, swapping a submodule’s inputs, or skipping a stage.

from transformers.exporters.utils import decompose_for_generation

components = decompose_for_generation(model, inputs)
# {"image_encoder": (submodel, fwd_kwargs), "language_model": (...), ..., "decode": (...)}

exported = {}
for name, (submodel, subinputs) in components.items():
    eager_outputs = submodel(**subinputs)  # sanity-check the eager forward before exporting
    exported[name] = exporter.export(submodel, subinputs, config=config)

Multi-token decode

By default the decode component is a single-token step — one query token against the KV cache — so torch.export specializes its query-sequence axis to 1. Pass multi_token_decode=True to capture decode as a multi-token decode instead: decompose_for_generation() merges two consecutive decode steps (it captures with max_new_tokens=3) into one forward, so that axis stays symbolic. A single graph then serves every query length — one token (ordinary decoding), many tokens at once (continuation-from-past, e.g. accepting a chunk of speculative tokens), and a plain prefill when the cache is empty.

Dynamo

ONNX

ExecuTorch

The query axis only stays symbolic under a dynamic-shape export (dynamic=True); a static export freezes it at the captured length, giving a fixed multi-token graph. It composes with the static KV cache below — the merged decode writes each step’s tokens into the fixed-size cache in place, and the cache handles where they land internally.

Static KV cache

The ExecuTorch examples in this section, including zero-copy updates and the decode loop, are XNNPACK-only. The MLX exporter rejects StaticCache; use DynamicCache with MLX.

generate() grows a DynamicCache by default, reallocating as the sequence extends — a moving target for an exported graph. A static cache is a fixed-size buffer, allocated once and written in place at the current position each step. Combined with a multi-token decode it collapses generation into a single exported graph: the decode graph takes a fixed-size cache and a variable number of query tokens, so one graph serves both the prompt (empty cache → prefill) and each generated token (populated cache → decode). Export it by forwarding a GenerationConfig with cache_implementation="static" (and a max_cache_len) alongside multi_token_decode=True:

Dynamo

ONNX

ExecuTorch XNNPACK

The decode graph now has two symbolic axes — the query length (how many tokens you feed) and the cache length (max_cache_len, resizable at load time). dynamic=True marks these (and every other axis) Dim.AUTO, so the exported graph accepts any prompt length and cache size at load time.

Zero-copy in-place updates

The static cache is passed in and mutated in place, so one buffer carries state across decode steps with no host copies — as long as the runtime binds the caller’s buffers rather than copying through its own arena. What that takes is the only per-backend part left:

Dynamo — the exported program models the cache write as a USER_INPUT_MUTATION, so calling exported.runners["decode"](...) updates the cache tensors you pass in directly. Reuse the same tensors each step; nothing to configure.
ONNX Runtime — the decode graph exposes the cache as matched input.<name> / output.<name> pairs. ORT’s CudaSession.set_buffer_sharing (onnxruntime.transformers.io_binding_helper) binds each pair to one device buffer, so the cache is read and updated in place across the loop with no host round-trips.
ExecuTorch (XNNPACK) — turn off the memory-planning allocations on ExecutorchConfig so the in-place write can land in the caller’s own tensor (see the reference for what each flag does):
```
config = ExecutorchConfig(
    backend="xnnpack",
    dynamic=True,
    alloc_graph_input=False,
    alloc_graph_output=False,
    alloc_mutable_buffers=False,
)
```
The zero-copy in-place write also needs the caller to bind output buffers at runtime via Method::set_output_data_ptr — not surfaced by the Python runtime (executorch.runtime.Method exposes only execute/set_inputs/get_outputs). The flags above set it up, but the in-place write is a C++-only path (see the ExecuTorch decode-loop example below). From Python, read the updated cache back from the method outputs each step.

Decode-loop inference examples

The loop is the same shape on every backend — it’s the same graph throughout. Start from an empty fixed-size cache, feed the whole prompt once (empty cache → prefill), then one token at a time (populated cache → decode). Each call passes input_ids, a causal attention_mask, and position_ids (advanced by the number of new tokens each step), plus the cache, and gets back logits for every query position. Where each token lands in the cache is tracked internally by the static cache, so there’s nothing extra to thread through the call. How the cache is set up differs per runtime (a StaticCache object for Dynamo, raw device buffers for ONNX Runtime, caller arrays in C++ for ExecuTorch), so each tab builds its own below. The Dynamo and ONNX Runtime tabs update the cache in place; ExecuTorch’s in-place path is C++ (its Python runtime can’t, as noted above).

Dynamo

ONNX Runtime

ExecuTorch XNNPACK

torch.export records the static-cache write as a USER_INPUT_MUTATION, so the decode runner updates the StaticCache you pass in directly — one cache carries state across the whole loop with nothing to bind or thread back out. The cache has to be initialized up front (torch.export bakes the allocated K/V into the input spec, so a lazy blank cache won’t match) — but the saved program carries its own example_inputs, so reuse that already-initialized StaticCache template, reset to empty:

import copy
import torch
from transformers.exporters import ExportedModel

exported = ExportedModel.from_pretrained("qwen3-static", device="cuda")
decode = exported.runners["decode"]

# the program carries an initialized StaticCache template — reuse it (reset to empty)
_, example_kwargs = exported.components["decode"].program.example_inputs
past_key_values = copy.deepcopy(example_kwargs["past_key_values"])
past_key_values.reset()

def causal_mask(positions, cache_len):   # [1, 1, len(positions), cache_len]
    return (torch.arange(cache_len, device="cuda")[None, :] <= positions[:, None])[None, None]

# prefill: the whole prompt in one call
positions = torch.arange(prompt_len, device="cuda")
logits = decode(input_ids=prompt_ids, attention_mask=causal_mask(positions, max_cache_len),
                position_ids=positions[None], past_key_values=past_key_values).logits
next_token = logits[:, -1:].argmax(-1)

# decode: query=1 buffers reused in place
input_ids = torch.empty((1, 1), dtype=torch.long, device="cuda")
position_ids = torch.empty((1, 1), dtype=torch.long, device="cuda")
attention_mask = torch.empty((1, 1, 1, max_cache_len), dtype=torch.bool, device="cuda")
slots = torch.arange(max_cache_len, device="cuda")
for position in range(prompt_len, max_cache_len):
    input_ids.copy_(next_token)
    position_ids.fill_(position)
    attention_mask[0, 0, 0].copy_(slots <= position)
    logits = decode(input_ids=input_ids, attention_mask=attention_mask,
                    position_ids=position_ids, past_key_values=past_key_values).logits
    next_token = logits[:, -1:].argmax(-1)

ONNX Runtime runs the graph as-is; the in-place cache update is done with ORT’s CudaSession (onnxruntime.transformers.io_binding_helper), a thin wrapper over ORT io-binding. set_buffer_sharing binds a cache input.<name> and its matching output.<name> to one device buffer, so the mutated K/V/counter are written straight back into the input; allocate_buffers allocates the remaining (non-shared) outputs — here just logits; and infer(feed_dict) binds your CUDA tensors by pointer and runs. The cache buffers come straight from the graph’s own input metadata (get_inputs() shape and type), so no model config is needed — the one symbolic axis (cache length) becomes max_cache_len:

import torch
import onnxruntime as ort
from onnxruntime.transformers.io_binding_helper import CudaSession, TypeHelper

def causal_mask(positions, cache_len):
    return (torch.arange(cache_len, device="cuda")[None, :] <= positions[:, None])[None, None]

session = ort.InferenceSession("decode.onnx", providers=["CUDAExecutionProvider"])
cuda = CudaSession(session, torch.device("cuda"))

# fresh device cache buffers built from each cache input's own shape/dtype; share each
# input.<name>/output.<name> pair on one buffer so the update lands in place
cache = {}
for i in session.get_inputs():
    if not i.name.startswith("input."):
        continue
    name = i.name[len("input.") :]
    dims = [max_cache_len if isinstance(d, str) and not d.isdigit() else int(d) for d in i.shape]
    cache[name] = torch.zeros(dims, dtype=TypeHelper.ort_type_to_torch_type(i.type), device="cuda")
    cuda.set_buffer_sharing(f"input.{name}", f"output.{name}")
cache_feed = {f"input.{name}": buf for name, buf in cache.items()}

vocab_size = next(o.shape[-1] for o in session.get_outputs() if o.name.endswith("logits"))

# prefill: the whole prompt in one call
positions = torch.arange(prompt_len, device="cuda")
cuda.allocate_buffers({"logits": (1, prompt_len, vocab_size)})
out = cuda.infer({"input_ids": prompt_ids, "attention_mask": causal_mask(positions, max_cache_len),
                  "position_ids": positions[None], **cache_feed})
next_token = out["logits"][:, -1:].argmax(-1)

# decode: query=1 buffers reused in place
cuda.allocate_buffers({"logits": (1, 1, vocab_size)})
input_ids = torch.empty((1, 1), dtype=torch.long, device="cuda")
position_ids = torch.empty((1, 1), dtype=torch.long, device="cuda")
attention_mask = torch.empty((1, 1, 1, max_cache_len), dtype=torch.bool, device="cuda")
slots = torch.arange(max_cache_len, device="cuda")
for position in range(prompt_len, max_cache_len):
    input_ids.copy_(next_token)
    position_ids.fill_(position)
    attention_mask[0, 0, 0].copy_(slots <= position)
    out = cuda.infer({"input_ids": input_ids, "attention_mask": attention_mask,
                      "position_ids": position_ids, **cache_feed})
    next_token = out["logits"][:, -1:].argmax(-1)

ExecuTorch’s on-device runtime is C++, and the in-place cache update relies on Method::set_output_data_ptr — not surfaced by the Python runtime (executorch.runtime.Method exposes only execute/set_inputs/get_outputs), so the zero-copy decode is a C++-only path. Bind each mutated-cache output onto its matching cache input buffer, and the new K/V/counter land in the caller’s StaticCache buffers with no copies. As in the other tabs the shapes and sizes come from the artifact itself — here the program’s method_meta (input_tensor_meta/output_tensor_meta → TensorInfo::nbytes()), the C++ has no Python runtime to query:

#include <executorch/extension/data_loader/file_data_loader.h>
#include <executorch/extension/tensor/tensor_ptr.h>
#include <executorch/runtime/executor/method.h>
#include <executorch/runtime/executor/program.h>

using namespace executorch::runtime;
using executorch::extension::FileDataLoader;
using executorch::extension::make_tensor_ptr;

// load the exported decode and its `forward` method (Result error-checks elided for brevity)
auto loader = FileDataLoader::from("decode.pte");
auto program = Program::load(&loader.get());

// method-execution memory: a fixed arena for bookkeeping + one buffer per the method's memory plan
std::array<uint8_t, 4 * 1024 * 1024> arena;
MemoryAllocator method_allocator(arena.size(), arena.data());
auto meta = program->method_meta("forward");
std::vector<std::vector<uint8_t>> planned(meta->num_memory_planned_buffers());
std::vector<Span<uint8_t>> planned_spans;
for (size_t i = 0; i < planned.size(); ++i) {
    planned[i].resize(meta->memory_planned_buffer_size(i).get());
    planned_spans.push_back({planned[i].data(), planned[i].size()});
}
HierarchicalAllocator planned_allocator({planned_spans.data(), planned_spans.size()});
MemoryManager memory_manager(&method_allocator, &planned_allocator);
auto decode = std::move(program->load_method("forward", &memory_manager).get());

// shapes and sizes come from method_meta — no external config. Inputs are [ids, mask, position_ids,
// cache×N]; outputs are [returned cache×N, logits, mutated cache×N].
const size_t num_cache_tensors = meta->num_inputs() - 3;
const size_t logits_out_idx = num_cache_tensors;
std::vector<size_t> cache_nbytes(num_cache_tensors), cache_out_idx(num_cache_tensors);
for (size_t i = 0; i < num_cache_tensors; ++i) {
    cache_nbytes[i] = meta->input_tensor_meta(3 + i)->nbytes();
    cache_out_idx[i] = num_cache_tensors + 1 + i;   // mutated cache = the last N outputs
}
const size_t logits_nbytes = meta->output_tensor_meta(logits_out_idx)->nbytes();

// one set of StaticCache buffers (K/V + per-layer counters) is reused across steps, bound in place each call
auto forward = [&](const TensorPtr& input_ids, const TensorPtr& mask, const TensorPtr& position_ids) {
    decode.set_input(EValue(*input_ids), 0);
    decode.set_input(EValue(*mask), 1);
    decode.set_input(EValue(*position_ids), 2);
    for (int i = 0; i < num_cache_tensors; ++i)
        decode.set_input(EValue(*cache_tensor[i]), 3 + i);
    // bind each mutated-cache output onto that same input tensor's data → the write lands in place, zero copies
    for (int i = 0; i < num_cache_tensors; ++i)
        decode.set_output_data_ptr(cache_tensor[i]->mutable_data_ptr(), cache_nbytes[i], cache_out_idx[i]);
    decode.set_output_data_ptr(logits_data, logits_nbytes, logits_out_idx);
    decode.execute();                 // cache updated in place; logits written to logits_data
    return argmax_last(logits_data);  // greedy pick
};

// prefill the whole prompt, then decode one token per step — the cache carries in place across all calls
int64_t next_token = forward(prompt_ids, prompt_mask, prompt_positions);
for (int64_t position = prompt_len; position < max_cache_len; ++position) {
    next_token = forward(make_tensor_ptr({1, 1}, &next_token, ScalarType::Long),
                         causal_mask(position),  // [1, 1, 1, max_cache_len] bool
                         make_tensor_ptr({1, 1}, &position, ScalarType::Long));
}

ONNX Runtime fusions

Set ort_fusions to rewrite an ONNX export into ONNX Runtime’s fused transformer ops before it’s saved. ONNX Runtime runs the fused graph much faster than the plain one. The fusions cover RMS and layer norms, attention (MultiHeadAttention, or GroupQueryAttention when there’s a key-value cache), rotary embeddings, GELU, and projections that share an input. Each fusion gives the same result as the ops it replaces.

Set ort_fusions to the execution provider the graph will run on. Each provider gets the fused ops it supports.

config = OnnxConfig(ort_fusions="cpu")
config = OnnxConfig(ort_fusions="cuda")
config = OnnxConfig(ort_fusions="webgpu")
config = OnnxConfig(ort_fusions="dml")

	`"cpu"`	`"cuda"`	`"webgpu"`	`"dml"`
Attention with a key-value cache	`GroupQueryAttention`, rotary included	`GroupQueryAttention` with its query and key norms; the prefill keeps `MultiHeadAttention` for now (slower)	`GroupQueryAttention`, rotary as its own `RotaryEmbedding`	`MultiHeadAttention`, rotary as its own `RotaryEmbedding`
Other attention	plain ops for now (slower)	`MultiHeadAttention`	`MultiHeadAttention`	`MultiHeadAttention`
Mixture of experts	unfused	`MoE`	unfused	unfused
CUDA graphs	—	static exports can replay	—	—

A "cuda" graph with MoE doesn’t run on CPU, since ONNX Runtime’s MoE only exists for CUDA. A "webgpu" graph runs in the browser with onnxruntime-web, or natively with onnxruntime-webgpu. A "dml" graph runs on Windows with onnxruntime-directml.

The fused ops aren’t standard ONNX, so other ONNX consumers such as TensorRT or OpenVINO can’t read them. The default is ort_fusions=None, which keeps the graph standard. If you need the model on several runtimes, export it once for each.

The fusions run before OnnxRuntimeQuantizer quantizes the graph, as ONNX Runtime recommends. They’re skipped when the model is quantized before export, and they require optimize=True (the default). The "cuda" fusions need onnxruntime 1.28 or later.

Quantization

Every export config takes a quantizer that quantizes the model during the export, on the torch.export FX graph (PT2E or NNCF, for every backend) or on the converted ONNX or OpenVINO model (ONNX Runtime or NNCF). See Quantizing exports for the quantizers, their backends, calibration, and how to add one.

Limitations and workarounds

torch.export, torch.onnx.export, and ExecuTorch each have rough edges around specific PyTorch patterns. The exporters work around these with a small set of reversible patches and FX-level fixes applied at well-defined points in the export flow. None of this is visible from the public export API, but the most common things to know:

FlashAttention and FlexAttention are not exportable on any backend. sdpa is the preferred setting and eager also works (slower). Set one of them on the model before calling export if it’s using something else.
grouped_mm traces fine through DynamoExporter and is auto-translated for OnnxExporter. For ExecutorchExporter with either XNNPACK or MLX, the exporter swaps MoE experts to batched_mm before export.
Sliding-window layers (Mistral, Gemma 3) export with a full key-value cache; the sliding mask still applies the window.

Next steps

Add export support for a new architecture or backend with the patch and fix registries in Extending the exporters.
Quantize an export, or add a quantizer, in Quantizing exports.

Update on GitHub