Instructions to use apehex/tokun with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TF-Keras
How to use apehex/tokun with TF-Keras:
# Note: 'keras<3.x' or 'tf_keras' must be installed (legacy), and from_pretrained_keras was removed in huggingface_hub 1.0. # See https://github.com/keras-team/tf-keras for more details. # !pip install "huggingface_hub<1.0" tf_keras from huggingface_hub import from_pretrained_keras model = from_pretrained_keras("apehex/tokun") - Notebooks
- Google Colab
- Kaggle
Upload folder using huggingface_hub
Browse files
.gitattributes
CHANGED
|
@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
variables/variables.data-00000-of-00001 filter=lfs diff=lfs merge=lfs -text
|
| 37 |
variants/4x16/variables/variables.data-00000-of-00001 filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
variables/variables.data-00000-of-00001 filter=lfs diff=lfs merge=lfs -text
|
| 37 |
variants/4x16/variables/variables.data-00000-of-00001 filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
variants/4x4/variables/variables.data-00000-of-00001 filter=lfs diff=lfs merge=lfs -text
|
variants/4x4/README.md
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: keras
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
## Model description
|
| 6 |
+
|
| 7 |
+
More information needed
|
| 8 |
+
|
| 9 |
+
## Intended uses & limitations
|
| 10 |
+
|
| 11 |
+
More information needed
|
| 12 |
+
|
| 13 |
+
## Training and evaluation data
|
| 14 |
+
|
| 15 |
+
More information needed
|
variants/4x4/config.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"name": "model", "trainable": true, "layers": [{"module": "keras.layers", "class_name": "InputLayer", "config": {"batch_input_shape": [null, 8192], "dtype": "int32", "sparse": false, "ragged": false, "name": "input_1"}, "registered_name": null, "name": "input_1", "inbound_nodes": []}, {"module": "tokun.model", "class_name": "Encoder", "config": {"token_dim": [4, 4], "encoding_dim": 256, "embedding_dim": 256, "hidden_dim": 1024, "latent_dim": 256, "activation": "gelu", "sequence_axis": 1, "feature_axis": -1}, "registered_name": "models>Encoder", "build_config": {"input_shape": [null, 1024]}, "name": "encoder", "inbound_nodes": [[["input_1", 0, 0, {}]]]}, {"module": "tokun.model", "class_name": "Decoder", "config": {"token_dim": [4, 4], "encoding_dim": 256, "embedding_dim": 256, "hidden_dim": 1024, "latent_dim": 256, "activation": "gelu", "sequence_axis": 1, "feature_axis": -1}, "registered_name": "models>Decoder", "build_config": {"input_shape": [null, 64, 256]}, "name": "decoder", "inbound_nodes": [[["encoder", 0, 0, {}]]]}], "input_layers": [["input_1", 0, 0]], "output_layers": [["decoder", 0, 0]]}
|
variants/4x4/fingerprint.pb
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6b1ecb945f71ef45506a59f2f8996f141705b7cf71dc390fb9a1e17976c3aa94
|
| 3 |
+
size 55
|
variants/4x4/keras_metadata.pb
ADDED
|
File without changes
|
variants/4x4/saved_model.pb
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c57abab67267ebe8c77056c49b27ae703180065803f1c234abf2496f65e53361
|
| 3 |
+
size 556949
|
variants/4x4/variables/variables.data-00000-of-00001
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b2827a1ff461d6656efb17dc084e0996c04b975818c38970b1954c3fbaea2523
|
| 3 |
+
size 4755314
|
variants/4x4/variables/variables.index
ADDED
|
Binary file (1.21 kB). View file
|
|
|