Model save

README.md

@@ -0,0 +1,79 @@
+---
+library_name: transformers
+base_model: CodeIsAbstract/HybridTimeScaleModel_conti2
+tags:
+- generated_from_trainer
+model-index:
+- name: HybridTimeScaleModel_conti3
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# HybridTimeScaleModel_conti3
+
+This model is a fine-tuned version of [CodeIsAbstract/HybridTimeScaleModel_conti2](https://huggingface.co/CodeIsAbstract/HybridTimeScaleModel_conti2) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 3.2616
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 16
+- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: constant_with_warmup
+- lr_scheduler_warmup_steps: 10
+- training_steps: 1000
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 3.1777        | 0.05  | 50   | 3.4269          |
+| 3.1601        | 0.1   | 100  | 3.3961          |
+| 3.1383        | 0.15  | 150  | 3.3838          |
+| 3.0848        | 0.2   | 200  | 3.3661          |
+| 3.0850        | 0.25  | 250  | 3.3512          |
+| 3.0674        | 0.3   | 300  | 3.3399          |
+| 3.0248        | 0.35  | 350  | 3.3282          |
+| 3.0237        | 0.4   | 400  | 3.3229          |
+| 3.0460        | 0.45  | 450  | 3.3178          |
+| 3.0554        | 0.5   | 500  | 3.3105          |
+| 3.0215        | 0.55  | 550  | 3.3045          |
+| 3.0404        | 0.6   | 600  | 3.2980          |
+| 3.0302        | 0.65  | 650  | 3.2929          |
+| 3.0227        | 0.7   | 700  | 3.2861          |
+| 2.9928        | 0.75  | 750  | 3.2805          |
+| 3.0219        | 0.8   | 800  | 3.2757          |
+| 2.9899        | 0.85  | 850  | 3.2712          |
+| 2.9925        | 0.9   | 900  | 3.2690          |
+| 2.9400        | 0.95  | 950  | 3.2654          |
+| 2.9780        | 1.0   | 1000 | 3.2616          |
+
+
+### Framework versions
+
+- Transformers 5.14.1
+- Pytorch 2.8.0+cu128
+- Datasets 5.0.1
+- Tokenizers 0.22.2

best_model_streaming_cont/config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "HybridTimeScaleLM"
+  ],
+  "bos_token_id": 1,
+  "chunk_size": 128,
+  "dropout": 0.05,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "latent_dim": 768,
+  "layer_types": [
+    "linear",
+    "linear",
+    "linear",
+    "softmax",
+    "linear",
+    "linear",
+    "linear",
+    "softmax",
+    "linear",
+    "linear",
+    "linear",
+    "softmax"
+  ],
+  "model_type": "hybrid_fourier_lm",
+  "num_layers": 12,
+  "num_modes": 64,
+  "pad_token_id": 2,
+  "tie_word_embeddings": true,
+  "time_scale": 128.0,
+  "transformers_version": "5.14.1",
+  "use_cache": false,
+  "vocab_size": 32768
+}

best_model_streaming_cont/generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 2,
+  "transformers_version": "5.14.1"
+}

best_model_streaming_cont/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98d2fe0f6c71fc451029e15d3adda76c167de5a1de26fc9f1e9f7395acc07ef0
+size 234681136

best_model_streaming_cont/tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "is_local": false,
+  "legacy": false,
+  "local_files_only": false,
+  "model_max_length": 1000000000,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

best_model_streaming_cont/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d533f720772e44e41a40b44f509f4cd172bfbbaa7e27d2a199c907bce81123b
+size 5265

tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "is_local": false,
+  "legacy": false,
+  "local_files_only": false,
+  "model_max_length": 1000000000,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}