Hugging Face's logo

Transformers documentation

Exporters

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v5.14.0).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Exporters

New export backends can be added to Transformers by subclassing HfExporter.

Learn how to use the built-in exporters in the Exporters guide.

AutoHfExporter

class transformers.exporters.AutoHfExporter

< >

( )

The Auto-HF expoerter class that takes care of automatically instantiating to the correct HfExporter given the ExportConfig.

from_pretrained

< >

( pretrained_model_name_or_path**kwargs )

Load an exporter instance from a pretrained model/checkpoint that ships an export config.

Not implemented yet — placeholder for a first-class “export recipe” workflow.

The idea: model owners publish an export_config.json (or an export_config field in config.json) alongside their weights on the Hub. That file captures the settings the owner has already validated for their architecture — the target format (dynamo / onnx / executorch), exact dynamic-shape specs (e.g. text_ids dynamic to 4096, image tiles fixed at 448, batch=1 for edge deployment), strict flag, ONNX opset, prefill vs. decode layout, ExecuTorch backend choice, and any other knob that today lives as tribal knowledge in a README or a private notebook.

Consumers then get the owner-validated export in one call:

exporter = AutoHfExporter.from_pretrained("org/model-name")
program = exporter.export(model, inputs)

Composes with the [register_export_input_preparer] registry: the owner supplies the shape spec via export_config.json, transformers supplies the data-dependent precomputations (cu_seqlens, vision position ids, window indices, …) for that architecture. Together they cover the two hard parts of exporting new models — knowing the right shape contract and preparing the right inputs — so downstream users don’t re-derive either from scratch (and don’t break in production when they get it wrong).

supports_export_format

< >

( export_config_dict: dict )

Return True if the provided dict describes an export_format that has both a registered config class and a registered exporter class. Warns with an actionable message when the format is missing entirely, unknown, or only half-registered.

AutoExportConfig

class transformers.exporters.AutoExportConfig

< >

( )

The Auto-HF export config class that takes care of automatically dispatching to the correct export config given an export config stored in a dictionary.

HfExporter

class transformers.exporters.HfExporter

< >

( )

Abstract base class for all Transformers exporters.

Subclass and implement ~HfExporter.export to add a new export backend.

export

< >

( model: PreTrainedModelsample_inputs: MutableMapping[str, torch.Tensor | Cache]config: ExportConfigMixin )

Parameters

  • model (PreTrainedModel) — The model to export.
  • sample_inputs (dict[str, torch.Tensor | Cache]) — Forward kwargs — what you’d pass to model(**sample_inputs). These are used directly as the example inputs during tracing. For an autoregressive decode-step export, this means you need to include past_key_values, cache_position, etc. If you only have generation-style inputs, use ~HfExporter.export_for_generation instead — it runs model.generate for you and exports each stage.
  • config (ExportConfigMixin) — Backend-specific configuration.

Export the model and return the backend-specific program object.

export_for_generation

< >

( model: PreTrainedModelsample_inputs: MutableMapping[str, torch.Tensor | Cache]config: ExportConfigMixin | dict[str, ExportConfigMixin] ) → dict[str, Any]

Parameters

  • model (PreTrainedModel) — The generative model to export. Must support model.generate(**sample_inputs).
  • sample_inputs (dict[str, torch.Tensor | Cache]) — Generate kwargs — what you’d pass to model.generate(**sample_inputs) (typically input_ids + attention_mask, plus any modality inputs like pixel_values / input_features for multi-modal models). Per-stage forward kwargs are captured internally.
  • config (ExportConfigMixin or dict[str, ExportConfigMixin]) — Backend-specific configuration. Pass a single config to apply to every component, or a dict keyed by component name (e.g. "image_encoder", "language_model", "lm_head", "decode") to override per-component — all component names must be present in the dict.

Returns

dict[str, Any]

{component_name: backend_specific_artifact} — same keys as decompose_for_generation(). Values are whatever ~HfExporter.export returns for the concrete backend (ExportedProgram, ONNXProgram, ExecutorchProgramManager).

Decompose a generative model and export each component independently.

Thin wrapper around decompose_for_generation() that calls ~HfExporter.export on every returned (submodel, forward_inputs) pair. If you need the intermediate (submodel, forward_inputs) pairs (for verification, custom inputs, skipping a stage, …), call decompose_for_generation() directly.

validate_environment

< >

( *args**kwargs )

Check required_packages are installed and warn on version drift from tested_versions.

DynamoExporter

class transformers.exporters.DynamoExporter

< >

( )

Exporter that converts a PreTrainedModel to an ExportedProgram.

Example:

>>> from transformers.exporters.exporter_dynamo import DynamoExporter, DynamoConfig

>>> exporter = DynamoExporter()
>>> exported = exporter.export(model, inputs, config=DynamoConfig(dynamic=True))
>>> outputs = exported.module()(**inputs)

export

< >

( model: PreTrainedModelsample_inputs: MutableMapping[str, Any]config: DynamoConfig | dict[str, Any] )

OnnxExporter

class transformers.exporters.OnnxExporter

< >

( )

Exporter that converts a PreTrainedModel to an ONNX ONNXProgram.

Example:

>>> from transformers.exporters.exporter_onnx import OnnxExporter, OnnxConfig

>>> exporter = OnnxExporter()
>>> onnx_program = exporter.export(model, inputs, config=OnnxConfig(dynamic=True))
>>> outputs = onnx_program(**inputs)  # run in-memory
>>> exporter.export(model, inputs, config=OnnxConfig(output_path="model.onnx"))  # save to disk

export

< >

( model: PreTrainedModelsample_inputs: MutableMapping[str, Any]config: OnnxConfig | dict[str, Any] )

ExecutorchExporter

class transformers.exporters.ExecutorchExporter

< >

( )

Exporter that converts a PreTrainedModel to an ExecuTorch ExecutorchProgramManager.

Example:

>>> from transformers.exporters.exporter_executorch import ExecutorchExporter, ExecutorchConfig

>>> exporter = ExecutorchExporter()
>>> et_program = exporter.export(model, inputs, config=ExecutorchConfig(backend="xnnpack"))
>>> et_program.write_to_file("model.pte")

export

< >

( model: PreTrainedModelsample_inputs: MutableMapping[str, Any]config: ExecutorchConfig | dict[str, Any] )

Export a model to ExecuTorch, applying backend preparation and torch op patches.

DynamoConfig

class transformers.exporters.DynamoConfig

< >

( export_format: ExportFormat = <ExportFormat.DYNAMO: 'dynamo'>dynamic: bool = Falsestrict: bool = Falsedynamic_shapes: dict[str, typing.Any] | None = Noneprefer_deferred_runtime_asserts_over_guards: bool = False )

Parameters

  • dynamic (bool, optional, defaults to False) — Whether to export with dynamic (symbolic) shapes. When True and dynamic_shapes is not set, all tensor dimensions are set to Dim.AUTO automatically.
  • strict (bool, optional, defaults to False) — Whether to enable strict mode in torch.export. Runs the full symbolic trace and catches more errors, but is slower and more likely to fail on complex models.
  • dynamic_shapes (dict[str, Any], optional) — Explicit per-input dynamic shape specifications passed to torch.export. Takes precedence over dynamic.
  • prefer_deferred_runtime_asserts_over_guards (bool, optional, defaults to False) — When True, data-dependent shape guards are emitted as runtime asserts in the exported graph instead of failing the export at trace time when a guard wouldn’t hold across the full symbolic shape range. Most transformer LLMs need this set to True when using fine-grained Dim(min=, max=) bounds. Not needed with dynamic=True / Dim.AUTO, where torch.export infers shape relations instead of verifying them against the user-stated bounds.

Configuration class for exporting models via torch.export.

OnnxConfig

class transformers.exporters.OnnxConfig

< >

( export_format: ExportFormat = <ExportFormat.ONNX: 'onnx'>dynamic: bool = Falsestrict: bool = Falsedynamic_shapes: dict[str, typing.Any] | None = Noneprefer_deferred_runtime_asserts_over_guards: bool = Falseoutput_path: str | os.PathLike | None = Noneopset_version: int | None = Noneexternal_data: bool = Trueoptimize: bool = Trueexport_params: bool = Truekeep_initializers_as_inputs: bool = False )

Parameters

  • output_path (str or PathLike, optional) — Output path for the .onnx file. When None (default) the exported model is kept in memory as an ONNXProgram and not written to disk.
  • opset_version (int, optional) — ONNX opset version to target. Defaults to the latest opset supported by the installed onnxscript version.
  • external_data (bool, optional, defaults to True) — Store large weight tensors in a separate .onnx_data sidecar file instead of embedding them in the protobuf. Required for models whose weights exceed the 2 GB protobuf limit.
  • optimize (bool, optional, defaults to True) — Run onnxscript optimisation passes (constant folding, dead-code elimination, …) on the exported graph. Disable for models that hit upstream onnxscript optimiser bugs.
  • export_params (bool, optional, defaults to True) — Embed model weights in the ONNX graph. Set to False to export a weight-free graph (weights must be supplied at runtime).
  • keep_initializers_as_inputs (bool, optional, defaults to False) — Expose weight initializers as explicit graph inputs. Required by some older ONNX runtimes (opset < 9).

Configuration class for exporting models to ONNX via torch.onnx.export.

Inherits all fields from DynamoConfig (dynamic, strict, dynamic_shapes, prefer_deferred_runtime_asserts_over_guards).

ExecutorchConfig

class transformers.exporters.ExecutorchConfig

< >

( export_format: ExportFormat = <ExportFormat.EXECUTORCH: 'executorch'>dynamic: bool = Falsestrict: bool = Falsedynamic_shapes: dict[str, typing.Any] | None = Noneprefer_deferred_runtime_asserts_over_guards: bool = Falsebackend: str = 'xnnpack' )

Parameters

  • backend (str, optional, defaults to "xnnpack") — Target ExecuTorch backend. Supported values:

    • "xnnpack" — CPU inference via the XNNPACK library (default; runs anywhere).
    • "cuda" — GPU inference via the ExecuTorch CUDA backend.

Configuration class for exporting models to ExecuTorch format.

Inherits all fields from DynamoConfig (dynamic, strict, dynamic_shapes, prefer_deferred_runtime_asserts_over_guards).

Utilities

Lower-level functions that power export_for_generation, useful when you need to intervene between decomposing a model and exporting each component.

transformers.exporters.utils.get_leaf_tensors

< >

( obj: Any ) → dict[str, torch.Tensor]

Parameters

  • obj (Any) — A tensor, dataclass, dict, list, tuple, or any nesting thereof.

Returns

dict[str, torch.Tensor]

Flat mapping from dotted path strings to tensors.

Recursively retrieve all leaf tensors from a potentially nested structure.

transformers.exporters.utils.prepare_for_export

< >

( model: PreTrainedModel | torch.nn.Moduleinputs: MutableMapping[str, Any] )

Configure model and inputs for export. Mutates both model and inputs in place, returning (model, inputs, output_flags) where output_flags holds the values popped from inputs for use_cache, return_dict, etc. (to be applied reversibly onto model.config by patch_model_config during the trace).

  • Strips label inputs (labels, future_values) — loss computation is unsupported.
  • Pops output flags (use_cache, return_dict, …) from inputs so they don’t appear as traced kwargs; the values are returned for the trace block to apply onto model.config.
  • Pre-computes data-dependent vision/audio kwargs registered via @register_export_input_preparer and writes them into inputs.
  • Casts input tensors to match the model’s dtype / device.

transformers.exporters.utils.decompose_prefill_decode

< >

( model: PreTrainedModelinputs: dict[str, Any] ) → dict[str, tuple[torch.nn.Module, dict]]

Returns

dict[str, tuple[torch.nn.Module, dict]]

{"prefill": (model, prefill_inputs), "decode": (model, decode_inputs)}

Run model.generate() for 2 tokens and capture prefill and decode inputs.

Reuses the full generation machinery so every architecture (decoder-only, SSM, encoder-decoder, multi-modal, …) gets correct inputs without reimplementing the loop.

transformers.exporters.utils.decompose_multimodal

< >

( model: PreTrainedModelinputs: dict[str, Any] ) → dict[str, tuple[torch.nn.Module, dict]]

Returns

dict[str, tuple[torch.nn.Module, dict]]

One name: (module, inputs) entry per detected submodule (image/audio encoder, projector, language model, lm_head).

Raises

ValueError

  • ValueError — if no known multi-modal submodules are found on the model.

Capture inputs to each multi-modal submodule via a single forward pass.

Detects all known multi-modal submodules by attribute name (vision tower, projector, language model, lm_head, …) and captures their forward kwargs during one model(**inputs) call.

Each submodule is returned as a separate name: (module, inputs) entry for independent export. The token-merge step (e.g. masked_scatter for multi-modal models) is intentionally left outside the exported graphs — it is the caller’s responsibility to assemble inputs_embeds from the encoder outputs before running the decoder.

transformers.exporters.utils.decompose_for_generation

< >

( model: PreTrainedModelinputs: dict[str, Any] ) → `{component_name

Parameters

  • model — Generative model. Must support model.generate(**inputs).
  • inputs — Generate kwargs — what you’d pass to model.generate(**inputs).

Returns

`{component_name

(submodel, forward_inputs)}. Keys are “prefill”/“decode”for plain generative models and”_encoder”/“multi_modal_projector”/“language_model”/“lm_head”/“decode”` for multi-modal generative models.

Decompose a generative model into independently exportable (model, forward_inputs) pairs.

Runs decompose_prefill_decode to capture prefill and decode forward kwargs from a real model.generate(**inputs, max_new_tokens=2). If the prefill is multi-modal (per is_multimodal), further splits it into one entry per submodule (vision/audio encoder, projector, language model, lm_head) via decompose_multimodal.

transformers.exporters.utils.is_multimodal

< >

( model: PreTrainedModel )

Returns True if the model is multi-modal with modal encoders and a language model.

Update on GitHub

←Processors Quantization→