Buckets:
| # ExecuTorch | |
| [`ExecuTorch`](https://github.com/pytorch/executorch) is an end-to-end solution for enabling on-device inference capabilities across mobile and edge devices including wearables, embedded devices and microcontrollers. It is part of the PyTorch ecosystem and supports the deployment of PyTorch models with a focus on portability, productivity, and performance. | |
| ExecuTorch introduces well defined entry points to perform model, device, and/or use-case specific optimizations such as backend delegation, user-defined compiler transformations, memory planning, and more. The first step in preparing a PyTorch model for execution on an edge device using ExecuTorch is to export the model. This is achieved through the use of a PyTorch API called [`torch.export`](https://pytorch.org/docs/stable/export.html). | |
| ## ExecuTorch Integration[[transformers.TorchExportableModuleWithStaticCache]] | |
| An integration point is being developed to ensure that 🤗 Transformers can be exported using `torch.export`. The goal of this integration is not only to enable export but also to ensure that the exported artifact can be further lowered and optimized to run efficiently in `ExecuTorch`, particularly for mobile and edge use cases. | |
| #### transformers.TorchExportableModuleWithStaticCache[[transformers.TorchExportableModuleWithStaticCache]] | |
| ```python | |
| transformers.TorchExportableModuleWithStaticCache(model: PreTrainedModel, batch_size: int | None = None, max_cache_len: int | None = None, device: typing.Optional[torch.device] = None) | |
| ``` | |
| [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/executorch.py#L444) | |
| A recipe module designed to make a `PreTrainedModel` exportable with `torch.export`, | |
| specifically for decoder-only LM to `StaticCache`. This module ensures that the | |
| exported model is compatible with further lowering and execution in `ExecuTorch`. | |
| Note: | |
| This class is specifically designed to support export process using `torch.export` | |
| in a way that ensures the model can be further lowered and run efficiently in `ExecuTorch`. | |
| #### forward[[transformers.TorchExportableModuleWithStaticCache.forward]] | |
| ```python | |
| forward(input_ids: typing.Optional[torch.LongTensor] = None, inputs_embeds: typing.Optional[torch.Tensor] = None, cache_position: typing.Optional[torch.Tensor] = None) | |
| ``` | |
| [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/executorch.py#L536) | |
| **Parameters:** | |
| input_ids (`torch.Tensor`) : Tensor representing current input token id to the module. | |
| inputs_embeds (`torch.Tensor`) : Tensor representing current input embeddings to the module. | |
| cache_position (`torch.Tensor`) : Tensor representing current input position in the cache. | |
| **Returns:** `torch.Tensor` | |
| Logits output from the model. | |
| Forward pass of the module, which is compatible with the ExecuTorch runtime. | |
| This forward adapter serves two primary purposes: | |
| 1. **Making the Model `torch.export`-Compatible**: | |
| The adapter hides unsupported objects, such as the `Cache`, from the graph inputs and outputs, | |
| enabling the model to be exportable using `torch.export` without encountering issues. | |
| 2. **Ensuring Compatibility with `ExecuTorch` runtime**: | |
| The adapter matches the model's forward signature with that in `executorch/extension/llm/runner`, | |
| ensuring that the exported model can be executed in `ExecuTorch` out-of-the-box. | |
| #### transformers.convert_and_export_with_cache[[transformers.convert_and_export_with_cache]] | |
| ```python | |
| transformers.convert_and_export_with_cache(model: PreTrainedModel, example_input_ids: typing.Optional[torch.Tensor] = None, example_cache_position: typing.Optional[torch.Tensor] = None, dynamic_shapes: dict | None = None, strict: bool | None = None) | |
| ``` | |
| [Source](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/executorch.py#L750) | |
| **Parameters:** | |
| model (`PreTrainedModel`) : The pretrained model to be exported. | |
| example_input_ids (`Optional[torch.Tensor]`) : Example input token id used by `torch.export`. | |
| example_cache_position (`Optional[torch.Tensor]`) : Example current cache position used by `torch.export`. | |
| dynamic_shapes(`Optional[dict]`) : Dynamic shapes used by `torch.export`. | |
| strict(`Optional[bool]`) : Flag to instruct `torch.export` to use `dynamo`. | |
| **Returns:** Exported program (`torch.export.ExportedProgram`) | |
| The exported program generated via `torch.export`. | |
| Convert a `PreTrainedModel` into an exportable module and export it using `torch.export`, | |
| ensuring the exported model is compatible with `ExecuTorch`. | |
Xet Storage Details
- Size:
- 4.6 kB
- Xet hash:
- dd5eb1391e03d768da9a26af1f2ce45f52c027523c30e43a687b7c3058f23197
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.