Kernels:

---

webgpu-kernels

/

ai.onnx.Unique

Like 0

Follow

WebGPU Kernels Community 39

---
library_name: kernels
license: apache-2.0
tags:
- kernel
- webgpu
- wgsl
---
# ai.onnx.Unique

`ai.onnx`  ·  standard ONNX operator  ·  ONNX opset ≥ 11

## Description

Finds unique values or subtensors along an optional `axis`. Without an axis, `X` is flattened; results are sorted or retain first-occurrence order. Sub-32-bit integers and booleans use lossless widened 32-bit storage. Metadata outputs remain logical int64 but use lossless uint32 storage because all values are bounded by an addressable tensor extent. Exact data-dependent output shapes must be supplied. ONNX-permitted uint16, 64-bit, string, and complex inputs are unsupported by this package.

See the [ONNX `Unique` spec](https://onnx.ai/onnx/operators/onnx__Unique.html) for the reference semantics.

## Inputs

| Name | Upstream name | Logical dtype | Rank | Shape | Description | Presence |
| --- | --- | --- | --- | --- | --- | --- |
| `x` | `X` | `T` | — | — | The N-D input tensor from which unique values or subtensors are extracted. When `axis` is omitted, tensors of any rank are flattened in row-major order. | required |

## Outputs

| Name | Upstream name | Logical dtype | WebGPU storage | Rank | Shape | Description | Presence |
| --- | --- | --- | --- | --- | --- | --- | --- |
| `y` | `Y` | `T` | runtime-selected; narrow integers and bool use 32-bit slots | derived | — | Tensor containing all unique values or subtensors of X, sorted or in first-occurrence order. | required |
| `indices` | — | `I` | `uint32` | `1` | — | Optional logical int64 indices of each `Y` value or slice's first occurrence in `X`; stored as bounded uint32 values by WebGPU. | optional |
| `inverse_indices` | — | `I` | `uint32` | `1` | — | Optional logical int64 mapping from each flattened input value, or each input-axis slice, to its corresponding index in `Y`; stored as bounded uint32 values by WebGPU. | optional |
| `counts` | — | `I` | `uint32` | `1` | — | Optional logical int64 occurrence count for each unique value or slice in `Y`; stored as bounded uint32 values by WebGPU. | optional |

## Attributes

Attributes and default values (overridable per request):

| Attribute | Default | Description |
| --- | --- | --- |
| `axis` | — | Optional axis along which unique subtensors are identified. Negative values count from the back; when omitted, the input is flattened. |
| `sorted` | `1` | Whether to sort unique elements in ascending order before output; 1 (default) sorts, 0 retains first-occurrence order. |

## Type constraints

| Variable | Allowed dtypes |
| --- | --- |
| `T` | `float32`, `float16`, `uint32`, `int32`, `int16`, `uint8`, `int8`, `bool` |
| `I` | `int64` |

## Implementation variants

One implementation is selected per call from the device capabilities, the request shapes and the dtypes; these notes say what each one covers.

- `single_class_y` — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
- `scalar_hash_parallel_y_unsorted` — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
- `scalar_hash_parallel_y_sorted` — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
- `single_class_indices` — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
- `scalar_hash_parallel_indices_unsorted` — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
- `scalar_hash_parallel_indices_sorted` — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
- `single_class_inverse` — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
- `scalar_hash_parallel_inverse_unsorted` — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
- `scalar_hash_parallel_inverse_sorted` — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
- `single_class_indices_inverse` — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
- `scalar_hash_parallel_indices_inverse_unsorted` — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
- `scalar_hash_parallel_indices_inverse_sorted` — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
- `single_class_counts` — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
- `scalar_hash_parallel_counts_unsorted` — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
- `scalar_hash_parallel_counts_sorted` — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
- `single_class_indices_counts` — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
- `scalar_hash_parallel_indices_counts_unsorted` — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
- `scalar_hash_parallel_indices_counts_sorted` — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
- `single_class_inverse_counts` — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
- `scalar_hash_parallel_inverse_counts_unsorted` — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
- `scalar_hash_parallel_inverse_counts_sorted` — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
- `single_class_indices_inverse_counts` — Copy the first representative and materialize requested metadata when the exact output contract proves there is one distinct class.
- `scalar_hash_parallel_indices_inverse_counts_unsorted` — Hash first occurrences, scan flag blocks in parallel, and scatter scalar representatives in input order with only the requested metadata.
- `scalar_hash_parallel_indices_inverse_counts_sorted` — Hash and compact scalar representatives in parallel, sort their order with shared and global bitonic stages, and materialize only the requested metadata.
- `hash_dedup` — Hashes flattened values and records each key's first input index. Long, low-cardinality f32 inputs coalesce duplicates in workgroup memory when the device's storage budget permits it.
- `axis_hash_bounded` — Hash axis slices, sort compacted representative indices in one workgroup, and scatter the exact output.
- `axis_hash_bounded_unsorted` — Hash axis slices and compact representative indices in first-occurrence order.
- `axis_hash_bounded_nan_safe` — Hash and sort finite axis slices, with an exact ordered fallback when any floating-point slice contains a NaN.
- `axis_hash_bounded_unsorted_nan_safe` — Hash finite axis slices in first-occurrence order, with an exact ordered fallback when any floating-point slice contains a NaN.
- `axis_hash_bounded_merge` — Hash axis slices, sort device-sized shared tiles, and merge by exact lexicographic ranks.
- `axis_hash_bounded_nan_safe_merge` — Hash axis slices, sort device-sized shared tiles, and merge by exact lexicographic ranks. Preserve the exact ordered fallback when a floating-point slice contains NaN.
- `flat_parallel_metadata_indices` — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_hash_metadata_indices` — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_parallel_metadata_inverse` — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_hash_metadata_inverse` — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_parallel_metadata_indices_inverse` — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_hash_metadata_indices_inverse` — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_parallel_metadata_counts` — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_hash_metadata_counts` — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_parallel_metadata_indices_counts` — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_hash_metadata_indices_counts` — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_parallel_metadata_inverse_counts` — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_hash_metadata_inverse_counts` — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_parallel_metadata_all` — Builds the unique values with the parallel dedup and compaction passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.
- `flat_hash_metadata_all` — Builds the unique values with the hash-set dedup passes, then resolves every metadata output from the finished result: one thread per input element for the bucket and the first-occurrence store, one thread per unique value for the tally.

## Files

- [`metadata.json`](build/webgpu/metadata.json) — kernel metadata (id, digests, per-variant templates, provenance)
- [`manifest.json`](build/webgpu/manifest.json) — the op contract (source of truth)
- [`test.json`](build/webgpu/test.json) — correctness cases
- [`bench.json`](build/webgpu/bench.json) — benchmark cases
- [`scan-block-prefix-u32.wgsl.jinja`](build/webgpu/scan-block-prefix-u32.wgsl.jinja)
- [`unique-axis-compact-sort.wgsl.jinja`](build/webgpu/unique-axis-compact-sort.wgsl.jinja)
- [`unique-axis-dedup.wgsl.jinja`](build/webgpu/unique-axis-dedup.wgsl.jinja)
- [`unique-axis-hash.wgsl.jinja`](build/webgpu/unique-axis-hash.wgsl.jinja)
- [`unique-axis-merge.wgsl.jinja`](build/webgpu/unique-axis-merge.wgsl.jinja)
- [`unique-axis-scalar-inverse.wgsl.jinja`](build/webgpu/unique-axis-scalar-inverse.wgsl.jinja)
- [`unique-axis-scalar-ranks.wgsl.jinja`](build/webgpu/unique-axis-scalar-ranks.wgsl.jinja)
- [`unique-axis-scatter.wgsl.jinja`](build/webgpu/unique-axis-scatter.wgsl.jinja)
- [`unique-axis.wgsl.jinja`](build/webgpu/unique-axis.wgsl.jinja)
- [`unique-compact-sort.wgsl.jinja`](build/webgpu/unique-compact-sort.wgsl.jinja)
- [`unique-dedup.wgsl.jinja`](build/webgpu/unique-dedup.wgsl.jinja)
- [`unique-flag-block-scan.wgsl.jinja`](build/webgpu/unique-flag-block-scan.wgsl.jinja)
- [`unique-flat-metadata.wgsl.jinja`](build/webgpu/unique-flat-metadata.wgsl.jinja)
- [`unique-global-sort-exchange.wgsl.jinja`](build/webgpu/unique-global-sort-exchange.wgsl.jinja)
- [`unique-global-sort-output.wgsl.jinja`](build/webgpu/unique-global-sort-output.wgsl.jinja)
- [`unique-global-sort-shared.wgsl.jinja`](build/webgpu/unique-global-sort-shared.wgsl.jinja)
- [`unique-hash-build.wgsl.jinja`](build/webgpu/unique-hash-build.wgsl.jinja)
- [`unique-hash-collect.wgsl.jinja`](build/webgpu/unique-hash-collect.wgsl.jinja)
- [`unique-hash-init.wgsl.jinja`](build/webgpu/unique-hash-init.wgsl.jinja)
- [`unique-hash-mark.wgsl.jinja`](build/webgpu/unique-hash-mark.wgsl.jinja)
- [`unique-hash-sort-collected-key-only.wgsl.jinja`](build/webgpu/unique-hash-sort-collected-key-only.wgsl.jinja)
- [`unique-scalar-compact.wgsl.jinja`](build/webgpu/unique-scalar-compact.wgsl.jinja)
- [`unique-scalar-metadata.wgsl.jinja`](build/webgpu/unique-scalar-metadata.wgsl.jinja)
- [`unique-scalar-output.wgsl.jinja`](build/webgpu/unique-scalar-output.wgsl.jinja)
- [`unique-single-class.wgsl.jinja`](build/webgpu/unique-single-class.wgsl.jinja)
- [`unique.wgsl.jinja`](build/webgpu/unique.wgsl.jinja)

## Use with `@huggingface/kernels`

```sh
npm install --save-exact @huggingface/kernels@0.0.1-preview.3
```

Outputs with inferable metadata are allocated automatically. Explicit `outputs` entries request optional results or provide metadata that cannot be inferred from the supplied inputs and attributes.

This example supplies explicit metadata for:

- `y`

The `version: 1` option selects the published kernel contract; it is independent of any operator opset, contrib `since_version`, or model version.
It follows the `v1` branch as fixes land. To pin exact artifact bytes, pass a 40-character commit `revision` instead of `version`.

Replace each `*Data` placeholder with a typed array containing the corresponding input data.

```js
import { getKernel } from "@huggingface/kernels";

const kernel = await getKernel("webgpu-kernels/ai.onnx.Unique", { version: 1 });
// Explicit destinations request optional results or supply metadata that cannot be inferred.
const { y } = await kernel({ x: { data: xData, shape: [1] } }, {
  outputs: { y: { shape: [1], dtype: "float32" } },
});
```