Buckets:

hf-doc-build
/

doc

Files
xet
hf-doc-build/doc / transformers /main /ko /main_classes /processors.md
HuggingFaceDocBuilder
24 minutes ago
preview code
download
raw
23.4 kB
# 프로세서[[processors]]

## 멀티모달 프로세서[[transformers.ProcessorMixin]][[transformers.ProcessorMixin]]

모든 멀티모달 모델은 여러 모달리티(텍스트, 비전, 오디오)를 그룹화하는 데이터를 인코딩하거나 디코딩하는 객체가 필요한데, 이것은 프로세서라고 불리는 객체가 담당합니다. 프로세서는 토크나이저(텍스트 모달리티용), 이미지 프로세서(비전용), 특성 추출기(오디오용) 같이 두 개 이상의 처리 객체를 하나로 묶습니다.

이러한 프로세서는 저장 및 로딩 기능을 구현하는 다음 기본 클래스를 상속받습니다:

#### transformers.ProcessorMixin[[transformers.ProcessorMixin]]

```python
transformers.ProcessorMixin(*args, **kwargs)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L598)

This is a mixin used to provide saving/loading functionality for all processor classes.

#### apply_chat_template[[transformers.ProcessorMixin.apply_chat_template]]

```python
apply_chat_template(conversation: list[dict[str, str]] | list[list[dict[str, str]]], chat_template: str | None = None, tools: list[dict] | None = None, documents: list[dict[str, str]] | None = None, add_generation_prompt: bool = False, continue_final_message: bool | str = False, return_assistant_tokens_mask: bool = False, tokenize: bool = False, return_tensors: str | transformers.utils.generic.TensorType | None = None, return_dict: bool = False, load_audio_from_video: bool = False, processor_kwargs: dict | None = None, **kwargs)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1994)

**Parameters:**

conversation (`Union[list[Dict, [str, str]], list[list[dict[str, str]]]]`) : The conversation to format.

chat_template (`Optional[str]`, *optional*) : The Jinja template to use for formatting the conversation. If not provided, the tokenizer's chat template is used.
Similar to the `apply_chat_template` method on tokenizers, this method applies a Jinja template to input
conversations to turn them into a single tokenizable string.
The input is expected to be in the following format, where each message content is a list consisting of text and
optionally image or video inputs. One can also provide an image, video, URL or local path which will be used to form
`pixel_values` when `return_dict=True`. If not provided, one will get only the formatted text, optionally tokenized text.
conversation = [
{
"role": "user",
"content": [
{"type": "image", "url": "https://www.ilankelman.org/stopsigns/australia.jpg"},
{"type": "text", "text": "Please describe this image in detail."},
],
},
]
#### batch_decode[[transformers.ProcessorMixin.batch_decode]]
```python
batch_decode(*args, **kwargs)
```
[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1948)
This method forwards all its arguments to PreTrainedTokenizer's [batch_decode()](/docs/transformers/main/ko/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.batch_decode). Please
refer to the docstring of this method for more information.
#### check_argument_for_proper_class[[transformers.ProcessorMixin.check_argument_for_proper_class]]
```python
check_argument_for_proper_class(argument_name, argument)
```
[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L995)
Checks the passed argument's class against the expected transformers class. In case of an unexpected
mismatch between expected and actual class, an error is raise. Otherwise, the proper retrieved class
is returned.
#### create_mm_token_type_ids[[transformers.ProcessorMixin.create_mm_token_type_ids]]
```python
create_mm_token_type_ids(input_ids: list)
```
[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L921)
**Parameters:**

input_ids (*list[list[int]]*) : Batch of token ID sequences. May be unpadded (variable length), so a plain Python list of lists is expected rather than a tensor or uniformly-shaped array.
**Returns:** *list[list[int]]*
A list of the same structure as `input_ids`, where each
integer is the modality type ID for the corresponding token.

Build per-token modality type IDs for a batch of token_id sequences.
Each position is assigned an integer indicating which modality it belongs to:
`0` for regular text, `1` for image tokens, `2` for video tokens, and
`3` for audio tokens. Membership is determined by comparing against
`self.image_token_ids`, `self.video_token_ids`, and `self.audio_token_ids`.
#### decode[[transformers.ProcessorMixin.decode]]
```python
decode(*args, **kwargs)
```
[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1957)
This method forwards all its arguments to PreTrainedTokenizer's [decode()](/docs/transformers/main/ko/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.decode). Please refer to
the docstring of this method for more information.
#### from_args_and_dict[[transformers.ProcessorMixin.from_args_and_dict]]
```python
from_args_and_dict(args, processor_dict: dict, **kwargs)
```
[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1466)
**Parameters:**
processor_dict (`dict[str, Any]`) : Dictionary that will be used to instantiate the processor object. Such a dictionary can be retrieved from a pretrained checkpoint by leveraging the `~processing_utils.ProcessingMixin.to_dict` method.

kwargs (`dict[str, Any]`) : Additional parameters from which to initialize the processor object.

**Returns:** `~processing_utils.ProcessingMixin`

The processor object instantiated from those
parameters.

Instantiates a type of `~processing_utils.ProcessingMixin` from a Python dictionary of parameters.

#### from_pretrained[[transformers.ProcessorMixin.from_pretrained]]

```python
from_pretrained(pretrained_model_name_or_path: str | os.PathLike, cache_dir: str | os.PathLike | None = None, force_download: bool = False, local_files_only: bool = False, token: str | bool | None = None, revision: str = 'main', **kwargs)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1693)

**Parameters:**

pretrained_model_name_or_path (`str` or `os.PathLike`) : This can be either:  - a string, the *model id* of a pretrained feature_extractor hosted inside a model repo on huggingface.co. - a path to a *directory* containing a feature extractor file saved using the [save_pretrained()](/docs/transformers/main/ko/main_classes/feature_extractor#transformers.FeatureExtractionMixin.save_pretrained) method, e.g., `./my_model_directory/`. - a path to a saved feature extractor JSON *file*, e.g., `./my_model_directory/preprocessor_config.json`.

- ****kwargs** : Additional keyword arguments passed along to both [from_pretrained()](/docs/transformers/main/ko/main_classes/feature_extractor#transformers.FeatureExtractionMixin.from_pretrained) and `~tokenization_utils_base.PreTrainedTokenizer.from_pretrained`.

Instantiate a processor associated with a pretrained model.

This class method is simply calling the feature extractor
[from_pretrained()](/docs/transformers/main/ko/main_classes/feature_extractor#transformers.FeatureExtractionMixin.from_pretrained), image processor
[ImageProcessingMixin](/docs/transformers/main/ko/internal/image_processing_utils#transformers.ImageProcessingMixin) and the tokenizer
`~tokenization_utils_base.PreTrainedTokenizer.from_pretrained` methods. Please refer to the docstrings of the
methods above for more information.

#### get_processor_dict[[transformers.ProcessorMixin.get_processor_dict]]

```python
get_processor_dict(pretrained_model_name_or_path: str | os.PathLike, **kwargs)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1217)

**Parameters:**

pretrained_model_name_or_path (`str` or `os.PathLike`) : The identifier of the pre-trained checkpoint from which we want the dictionary of parameters.

subfolder (`str`, *optional*, defaults to `""`) : In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here.

**Returns:** `tuple[Dict, Dict]`

The dictionary(ies) that will be used to instantiate the processor object.

From a `pretrained_model_name_or_path`, resolve to a dictionary of parameters, to be used for instantiating a
processor of type `~processing_utils.ProcessingMixin` using `from_args_and_dict`.

#### get_text_with_replacements[[transformers.ProcessorMixin.get_text_with_replacements]]

```python
get_text_with_replacements(text: list, images_replacements: list = [], videos_replacements: list = [], audio_replacements: list = [])
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L816)

**Parameters:**

text (*list[str]*) : Batch of raw text strings, each potentially containing multimodal placeholder tokens. Note that it will be modified in-place and returned.

images_replacements (*list[str]*, *optional*, defaults to *[]*) : Expanded replacement strings for each image, in the order they appear across the batch. Produced by *self._process_images*.
videos_replacements (*list[str]*, *optional*, defaults to *[]*) : Expanded replacement strings for each video. Produced by *self._process_videos*.

audio_replacements (*list[str]*, *optional*, defaults to *[]*) : Expanded replacement strings for each audio input. Produced by *self._process_audio*.
**Returns:** *tuple[list[str], list[dict[str, Any]]]*
A tuple of:
- The modified *text* batch with all placeholder tokens expanded.
- *batch_replacement_offsets*: one entry per batch item, each being a
list of dicts with keys:
- *"type"* (*str*): modality name — *"image"*, *"video"*, or *"audio"*
- *"span"* (*tuple[int, int]*): original *(start, end)* char offsets of the placeholder token
- *"new_span"* (*tuple[int, int]*): *(start, end)* offsets of placeholder in the expanded string
- *"text"* (*str*): the original placeholder token string that was matched
- *"replacement"* (*str*): the string it was replaced with

Replace multimodal placeholder tokens in a batch of text strings with their
expanded representations, and return the modified texts alongside offset metadata.

This method is the core text-side preprocessing step for multimodal inputs. It
scans each text in the batch for special tokens (image, video, audio) and replaces
them in-order with the pre-computed replacement strings produced by
*self.replace_image_token* / *self.replace_video_token* / *self.replace_audio_token*.
Replacements are consumed from each modality's list sequentially, so the i-th
occurrence of e.g. `self.image_token` is replaced by `images_replacements[i]`.

To add a new multimodal processor with placeholder tokens, you need to define a correct
*self.image_token* which is the same token that is embedded in input text and also used as
placeholder and repeated many times. Then you need to override *self.replace_image_token*
to return the correct replacement string for a given image at index *i*. Same goes for all
other supported modalities.

#### parse_response[[transformers.ProcessorMixin.parse_response]]

```python
parse_response(response: str | list[int] | list[str] | list[list[int]] | np.ndarray | torch.Tensor, schema: dict | None = None, prefix: str | list[int] | list[str] | list[list[int]] | np.ndarray | torch.Tensor | None = None, tools: list[dict | collections.abc.Callable] | None = None)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L2285)

**Parameters:**

response (`str`, token ids, 1D/2D tensor, or a list of these) : The output generated by the model, either decoded text or token ids, as a single sequence or a batch.

schema (`dict`, *optional*) : A response template. If not provided, the tokenizer's `response_template` attribute is used.

prefix (`str`, token ids, 1D/2D tensor, or a list of these) : The prompt that came before generation. Many chat templates pre-write part of the message, so this is needed to parse correctly. For a batched `response`, pass either a single prefix (broadcast to every item) or one prefix per item. Only supported with new-style templates.

tools (`list[Union[Dict, Callable]]`, *optional*) : Tools available to the model, in the same format as `apply_chat_template` accepts. Tool-call arguments are cast using the calling tool's JSON schema.

Converts an output string created by generating text from a model into a parsed message dictionary.
This method is intended for use with chat models, and will read the tokenizer's `response_template`
attribute to control parsing, unless a `schema` argument is passed directly.

Accepts either a single sequence (returning a single message `dict`) or a batch (returning a `list` of
message dicts, one per item).

#### post_process_image_text_to_text[[transformers.ProcessorMixin.post_process_image_text_to_text]]

```python
post_process_image_text_to_text(generated_outputs, skip_special_tokens = True, **kwargs)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L2348)

**Parameters:**

generated_outputs (`torch.Tensor` or `np.ndarray`) : The output of the model `generate` function. The output is expected to be a tensor of shape `(batch_size, sequence_length)` or `(sequence_length,)`.

skip_special_tokens (`bool`, *optional*, defaults to `True`) : Whether or not to remove special tokens in the output. Argument passed to the tokenizer's `decode` method.

- ****kwargs** : Additional arguments to be passed to the tokenizer's `decode` method.

**Returns:** `list[str]`

The decoded text.

Post-process the output of a vlm to decode the text.

#### post_process_multimodal_output[[transformers.ProcessorMixin.post_process_multimodal_output]]

```python
post_process_multimodal_output(generated_outputs, skip_special_tokens = True, generation_mode = None, **kwargs)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L2319)

**Parameters:**

generated_outputs (`torch.Tensor` or `np.ndarray`) : The output of the model `generate` function. The output is expected to be a tensor of shape `(batch_size, sequence_length)` or `(sequence_length,)`.

skip_special_tokens (`bool`, *optional*, defaults to `True`) : Whether or not to remove special tokens in the output. Argument passed to the tokenizer's `batch_decode` method.

generation_mode (`str`, *optional*) : Generation mode indicated which modality to output and can be one of `["text", "image", "audio"]`.
- ****kwargs** : Additional arguments to be passed to the tokenizer's `batch_decode method`.

**Returns:** `list[str]`

The decoded text.

Post-process the output of a multimodal model to return the requested modality output.
If the model cannot generated the requested modality, an error will be raised.

#### prepare_inputs_layout[[transformers.ProcessorMixin.prepare_inputs_layout]]

```python
prepare_inputs_layout(images: typing.Union[ForwardRef('PIL.Image.Image'), numpy.ndarray, ForwardRef('torch.Tensor'), list['PIL.Image.Image'], list[numpy.ndarray], list['torch.Tensor'], NoneType] = None, text: str | list[str] | list[list[str]] | None = None, videos: typing.Union[list['PIL.Image.Image'], numpy.ndarray, ForwardRef('torch.Tensor'), list[numpy.ndarray], list['torch.Tensor'], list[list['PIL.Image.Image']], list[list[numpy.ndarray]], list[list['torch.Tensor']], transformers.video_utils.URL, list[transformers.video_utils.URL], list[list[transformers.video_utils.URL]], transformers.video_utils.Path, list[transformers.video_utils.Path], list[list[transformers.video_utils.Path]], NoneType] = None, audio: typing.Union[numpy.ndarray, ForwardRef('torch.Tensor'), collections.abc.Sequence[numpy.ndarray], collections.abc.Sequence['torch.Tensor'], NoneType] = None, **kwargs: *<class 'transformers.processing_utils.ProcessingKwargs'>)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L713)

Normalize and prefetch inputs before processing. Wraps text in a list for multimodal
processors, fetches remote images and audio if URLs are provided, and ensures audio
is properly batched. Returns the normalized `(images, text, videos, audio)` tuple.

#### push_to_hub[[transformers.ProcessorMixin.push_to_hub]]

```python
push_to_hub(repo_id: str, commit_message: str | None = None, commit_description: str | None = None, private: bool | None = None, token: bool | str | None = None, revision: str | None = None, create_pr: bool = False, max_shard_size: int | str | None = '50GB', tags: list[str] | None = None)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/utils/hub.py#L822)

**Parameters:**

repo_id (`str`) : The name of the repository you want to push your processor to. It should contain your organization name when pushing to a given organization.
commit_message (`str`, *optional*) : Message to commit while pushing. Will default to `"Upload processor"`.

commit_description (`str`, *optional*) : The description of the commit that will be created
private (`bool`, *optional*) : Whether to make the repo private. If `None` (default), the repo will be public unless the organization's default is private. This value is ignored if the repo already exists.
token (`bool` or `str`, *optional*) : The token to use as HTTP bearer authorization for remote files. If `True` (default), will use the token generated when running `hf auth login` (stored in `~/.huggingface`).
revision (`str`, *optional*) : Branch to push the uploaded files to.
create_pr (`bool`, *optional*, defaults to `False`) : Whether or not to create a PR with the uploaded files or directly commit.

max_shard_size (`int` or `str`, *optional*, defaults to `"50GB"`) : Only applicable for models. The maximum size for a checkpoint before being sharded. Checkpoints shard will then be each of size lower than this size. If expressed as a string, needs to be digits followed by a unit (like `"5MB"`).

tags (`list[str]`, *optional*) : List of tags to push on the Hub.

Upload the processor files to the 🤗 Model Hub.

Examples:

```python
from transformers import AutoProcessor
processor = AutoProcessor.from_pretrained("google-bert/bert-base-cased")
# Push the processor to your namespace with the name "my-finetuned-bert".
processor.push_to_hub("my-finetuned-bert")
# Push the processor to an organization with the name "my-finetuned-bert".
processor.push_to_hub("huggingface/my-finetuned-bert")
```

#### register_for_auto_class[[transformers.ProcessorMixin.register_for_auto_class]]

```python
register_for_auto_class(auto_class = 'AutoProcessor')
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1779)

**Parameters:**

auto_class (`str` or `type`, *optional*, defaults to `"AutoProcessor"`) : The auto class to register this new feature extractor with.
Register this class with a given auto class. This should only be used for custom feature extractors as the ones
in the library are already mapped with `AutoProcessor`.
#### save_pretrained[[transformers.ProcessorMixin.save_pretrained]]
```python
save_pretrained(save_directory, push_to_hub: bool = False, **kwargs)
```
[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1108)
**Parameters:**
save_directory (`str` or `os.PathLike`) : Directory where the feature extractor JSON file and the tokenizer files will be saved (directory will be created if it does not exist).

push_to_hub (`bool`, *optional*, defaults to `False`) : Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace).

kwargs (`dict[str, Any]`, *optional*) : Additional key word arguments passed along to the [push_to_hub()](/docs/transformers/main/ko/main_classes/model#transformers.utils.PushToHubMixin.push_to_hub) method.

Saves the attributes of this processor (feature extractor, tokenizer...) in the specified directory so that it
can be reloaded using the [from_pretrained()](/docs/transformers/main/ko/main_classes/processors#transformers.ProcessorMixin.from_pretrained) method.

This class method is simply calling [save_pretrained()](/docs/transformers/main/ko/main_classes/feature_extractor#transformers.FeatureExtractionMixin.save_pretrained) and
[save_pretrained()](/docs/transformers/main/ko/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.save_pretrained). Please refer to the docstrings of the
methods above for more information.

#### to_dict[[transformers.ProcessorMixin.to_dict]]

```python
to_dict()
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1018)

**Returns:** `dict[str, Any]`

Dictionary of all the attributes that make up this processor instance.

Serializes this instance to a Python dictionary.

#### to_json_file[[transformers.ProcessorMixin.to_json_file]]

```python
to_json_file(json_file_path: str | os.PathLike)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1092)

**Parameters:**

json_file_path (`str` or `os.PathLike`) : Path to the JSON file in which this processor instance's parameters will be saved.

Save this instance to a JSON file.

#### to_json_string[[transformers.ProcessorMixin.to_json_string]]

```python
to_json_string()
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L1081)

**Returns:** `str`

String containing all the attributes that make up this feature_extractor instance in JSON format.
Serializes this instance to a JSON string.
#### validate_inputs[[transformers.ProcessorMixin.validate_inputs]]
```python
validate_inputs(images: typing.Union[ForwardRef('PIL.Image.Image'), numpy.ndarray, ForwardRef('torch.Tensor'), list['PIL.Image.Image'], list[numpy.ndarray], list['torch.Tensor'], NoneType] = None, text: str | list[str] | list[list[str]] | None = None, videos: typing.Union[list['PIL.Image.Image'], numpy.ndarray, ForwardRef('torch.Tensor'), list[numpy.ndarray], list['torch.Tensor'], list[list['PIL.Image.Image']], list[list[numpy.ndarray]], list[list['torch.Tensor']], transformers.video_utils.URL, list[transformers.video_utils.URL], list[list[transformers.video_utils.URL]], transformers.video_utils.Path, list[transformers.video_utils.Path], list[list[transformers.video_utils.Path]], NoneType] = None, audio: typing.Union[numpy.ndarray, ForwardRef('torch.Tensor'), collections.abc.Sequence[numpy.ndarray], collections.abc.Sequence['torch.Tensor'], NoneType] = None, **kwargs: *<class 'transformers.processing_utils.ProcessingKwargs'>)
```
[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/processing_utils.py#L743)
Validate that at least one input is provided and that no deprecated keyword arguments
are used. Raises `ValueError` otherwise.
Override when the processor needs additional validation on the input args.
Xet Storage Details

Size:: 23.4 kB
Xet hash:: e33c779e9393c0b5922818c5c2148450930d5f4110d3994371b87d4b117eb552
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.