Instructions to use hozifa1/Faqih-R1-14B-Islamic-AI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hozifa1/Faqih-R1-14B-Islamic-AI with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="hozifa1/Faqih-R1-14B-Islamic-AI")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("hozifa1/Faqih-R1-14B-Islamic-AI")
model = AutoModelForCausalLM.from_pretrained("hozifa1/Faqih-R1-14B-Islamic-AI", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use hozifa1/Faqih-R1-14B-Islamic-AI with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "hozifa1/Faqih-R1-14B-Islamic-AI"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hozifa1/Faqih-R1-14B-Islamic-AI",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/hozifa1/Faqih-R1-14B-Islamic-AI

SGLang

How to use hozifa1/Faqih-R1-14B-Islamic-AI with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "hozifa1/Faqih-R1-14B-Islamic-AI" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hozifa1/Faqih-R1-14B-Islamic-AI",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "hozifa1/Faqih-R1-14B-Islamic-AI" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hozifa1/Faqih-R1-14B-Islamic-AI",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use hozifa1/Faqih-R1-14B-Islamic-AI with Docker Model Runner:
```
docker model run hf.co/hozifa1/Faqih-R1-14B-Islamic-AI
```

hozifa1 commited on Aug 22

Commit

2819ddd

verified ·

1 Parent(s): 84b5e44

Training in progress, step 6800, checkpoint

Browse files

Files changed (8) hide show

last-checkpoint/README.md +1 -1
last-checkpoint/adapter_config.json +9 -7
last-checkpoint/adapter_model.safetensors +1 -1
last-checkpoint/optimizer.pt +1 -1
last-checkpoint/rng_state.pth +1 -1
last-checkpoint/scheduler.pt +1 -1
last-checkpoint/trainer_state.json +871 -3
last-checkpoint/training_args.bin +1 -1

last-checkpoint/README.md CHANGED Viewed

@@ -207,4 +207,4 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 [More Information Needed]
 ### Framework versions
-- PEFT 0.19.1

 [More Information Needed]
 ### Framework versions
+- PEFT 0.20.0

last-checkpoint/adapter_config.json CHANGED Viewed

@@ -27,20 +27,21 @@
   "megatron_config": null,
   "megatron_core": "megatron.core",
   "modules_to_save": null,
   "peft_type": "LORA",
-  "peft_version": "0.19.1",
   "qalora_group_size": 16,
   "r": 16,
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
-    "v_proj",
-    "q_proj",
     "down_proj",
-    "gate_proj",
     "o_proj",
-    "up_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",
@@ -48,5 +49,6 @@
   "use_bdlora": null,
   "use_dora": false,
   "use_qalora": false,
-  "use_rslora": false
 }

   "megatron_config": null,
   "megatron_core": "megatron.core",
   "modules_to_save": null,
+  "monteclora_config": null,
   "peft_type": "LORA",
+  "peft_version": "0.20.0",
   "qalora_group_size": 16,
   "r": 16,
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
     "o_proj",
+    "gate_proj",
+    "q_proj",
+    "k_proj",
+    "up_proj",
+    "v_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",
   "use_bdlora": null,
   "use_dora": false,
   "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
 }

last-checkpoint/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:266a9975e538d5172b24571c56d759b82af1f6fe5f8401c0cca8b9ee3e084ff7
 size 275341720

 version https://git-lfs.github.com/spec/v1
+oid sha256:43b0289f455c36e97dd49c1377e85ecd24060687015c0eac78ac53d8ddd9a9bc
 size 275341720

last-checkpoint/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9cc5d6bc9ec977cd43b1a8c67b55e0531dd33ba00194c86475375e0496146b0c
 size 142323235

 version https://git-lfs.github.com/spec/v1
+oid sha256:955713a7e11cd656a13a1e66eba27cb291f790b6109702f0649fa44f6ddcec63
 size 142323235

last-checkpoint/rng_state.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5c6139a235c8921523884f4fa1d6978fdf889363ebf9e3c465d4671e58280552
 size 14645

 version https://git-lfs.github.com/spec/v1
+oid sha256:43e84d7127ab0948aff7615e4734d8c9c14acaf5561f1fe30caa2c17b6b67ec6
 size 14645

last-checkpoint/scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:45b67858659b4dfc3e1470897037774241f60eeba7a480b68826613d2df0626a
 size 1465

 version https://git-lfs.github.com/spec/v1
+oid sha256:b8dccd263b248571081865ddc09411ed6488d59bba9ed4ebad561a3a0f561b60
 size 1465

last-checkpoint/trainer_state.json CHANGED Viewed

@@ -2,9 +2,9 @@
   "best_global_step": 200,
   "best_metric": 0.6362597942352295,
   "best_model_checkpoint": "/kaggle/working/faqih_checkpoints/checkpoint-200",
-  "epoch": 0.15784281486353174,
   "eval_steps": 200,
-  "global_step": 600,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
@@ -100,6 +100,874 @@
       "learning_rate": 0.00018473101265822786,
       "loss": 0.6356534576416015,
       "step": 600
     }
   ],
   "logging_steps": 50,
@@ -119,7 +987,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 7.489718726034186e+17,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null

   "best_global_step": 200,
   "best_metric": 0.6362597942352295,
   "best_model_checkpoint": "/kaggle/working/faqih_checkpoints/checkpoint-200",
+  "epoch": 1.788687931601447,
   "eval_steps": 200,
+  "global_step": 6800,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
       "learning_rate": 0.00018473101265822786,
       "loss": 0.6356534576416015,
       "step": 600
+    },
+    {
+      "epoch": 0.17099638276882603,
+      "grad_norm": 0.23116259276866913,
+      "learning_rate": 0.00018341244725738397,
+      "loss": 0.6304442214965821,
+      "step": 650
+    },
+    {
+      "epoch": 0.18414995067412035,
+      "grad_norm": 0.1896994709968567,
+      "learning_rate": 0.00018209388185654008,
+      "loss": 0.6102799224853516,
+      "step": 700
+    },
+    {
+      "epoch": 0.19730351857941467,
+      "grad_norm": 0.20064501464366913,
+      "learning_rate": 0.00018077531645569622,
+      "loss": 0.6039998626708984,
+      "step": 750
+    },
+    {
+      "epoch": 0.21045708648470898,
+      "grad_norm": 0.20514807105064392,
+      "learning_rate": 0.00017945675105485233,
+      "loss": 0.6030560302734375,
+      "step": 800
+    },
+    {
+      "epoch": 0.2236106543900033,
+      "grad_norm": 0.20657017827033997,
+      "learning_rate": 0.00017813818565400844,
+      "loss": 0.5951737976074218,
+      "step": 850
+    },
+    {
+      "epoch": 0.2367642222952976,
+      "grad_norm": 0.21234729886054993,
+      "learning_rate": 0.00017681962025316457,
+      "loss": 0.6114842224121094,
+      "step": 900
+    },
+    {
+      "epoch": 0.2499177902005919,
+      "grad_norm": 0.19885726273059845,
+      "learning_rate": 0.00017550105485232068,
+      "loss": 0.6019914245605469,
+      "step": 950
+    },
+    {
+      "epoch": 0.2630713581058862,
+      "grad_norm": 0.22743742167949677,
+      "learning_rate": 0.0001741824894514768,
+      "loss": 0.6196039962768555,
+      "step": 1000
+    },
+    {
+      "epoch": 0.27622492601118054,
+      "grad_norm": 0.2094801813364029,
+      "learning_rate": 0.00017286392405063293,
+      "loss": 0.5877687454223632,
+      "step": 1050
+    },
+    {
+      "epoch": 0.28937849391647485,
+      "grad_norm": 0.22727341949939728,
+      "learning_rate": 0.00017154535864978904,
+      "loss": 0.6195404434204101,
+      "step": 1100
+    },
+    {
+      "epoch": 0.30253206182176917,
+      "grad_norm": 0.23307493329048157,
+      "learning_rate": 0.00017022679324894515,
+      "loss": 0.6037686920166015,
+      "step": 1150
+    },
+    {
+      "epoch": 0.3156856297270635,
+      "grad_norm": 0.20066751539707184,
+      "learning_rate": 0.00016890822784810128,
+      "loss": 0.6005681228637695,
+      "step": 1200
+    },
+    {
+      "epoch": 0.3288391976323578,
+      "grad_norm": 0.2251257598400116,
+      "learning_rate": 0.0001675896624472574,
+      "loss": 0.5749369049072266,
+      "step": 1250
+    },
+    {
+      "epoch": 0.34199276553765207,
+      "grad_norm": 0.19875267148017883,
+      "learning_rate": 0.0001662710970464135,
+      "loss": 0.5884634399414063,
+      "step": 1300
+    },
+    {
+      "epoch": 0.3551463334429464,
+      "grad_norm": 0.21068838238716125,
+      "learning_rate": 0.00016495253164556964,
+      "loss": 0.5809454345703124,
+      "step": 1350
+    },
+    {
+      "epoch": 0.3682999013482407,
+      "grad_norm": 0.2040138989686966,
+      "learning_rate": 0.00016363396624472575,
+      "loss": 0.6071104431152343,
+      "step": 1400
+    },
+    {
+      "epoch": 0.381453469253535,
+      "grad_norm": 0.21657569706439972,
+      "learning_rate": 0.00016231540084388186,
+      "loss": 0.6062085723876953,
+      "step": 1450
+    },
+    {
+      "epoch": 0.39460703715882933,
+      "grad_norm": 0.21580874919891357,
+      "learning_rate": 0.000160996835443038,
+      "loss": 0.5930040740966797,
+      "step": 1500
+    },
+    {
+      "epoch": 0.40776060506412365,
+      "grad_norm": 0.20997387170791626,
+      "learning_rate": 0.0001596782700421941,
+      "loss": 0.5776944351196289,
+      "step": 1550
+    },
+    {
+      "epoch": 0.42091417296941797,
+      "grad_norm": 0.22081924974918365,
+      "learning_rate": 0.0001583597046413502,
+      "loss": 0.5901007843017578,
+      "step": 1600
+    },
+    {
+      "epoch": 0.4340677408747123,
+      "grad_norm": 0.18726694583892822,
+      "learning_rate": 0.00015704113924050635,
+      "loss": 0.583731803894043,
+      "step": 1650
+    },
+    {
+      "epoch": 0.4472213087800066,
+      "grad_norm": 0.19900982081890106,
+      "learning_rate": 0.00015572257383966246,
+      "loss": 0.5790575027465821,
+      "step": 1700
+    },
+    {
+      "epoch": 0.4603748766853009,
+      "grad_norm": 0.2211243063211441,
+      "learning_rate": 0.00015440400843881857,
+      "loss": 0.577286262512207,
+      "step": 1750
+    },
+    {
+      "epoch": 0.4735284445905952,
+      "grad_norm": 0.2111779749393463,
+      "learning_rate": 0.0001530854430379747,
+      "loss": 0.5685956954956055,
+      "step": 1800
+    },
+    {
+      "epoch": 0.4866820124958895,
+      "grad_norm": 0.21098685264587402,
+      "learning_rate": 0.0001517668776371308,
+      "loss": 0.5858037185668945,
+      "step": 1850
+    },
+    {
+      "epoch": 0.4998355804011838,
+      "grad_norm": 0.1963949352502823,
+      "learning_rate": 0.00015044831223628692,
+      "loss": 0.5649369049072266,
+      "step": 1900
+    },
+    {
+      "epoch": 0.5129891483064781,
+      "grad_norm": 0.21639689803123474,
+      "learning_rate": 0.00014912974683544306,
+      "loss": 0.5866766357421875,
+      "step": 1950
+    },
+    {
+      "epoch": 0.5261427162117724,
+      "grad_norm": 0.2360610067844391,
+      "learning_rate": 0.00014781118143459917,
+      "loss": 0.5642653274536132,
+      "step": 2000
+    },
+    {
+      "epoch": 0.5392962841170668,
+      "grad_norm": 0.17728404700756073,
+      "learning_rate": 0.00014649261603375528,
+      "loss": 0.5934088516235352,
+      "step": 2050
+    },
+    {
+      "epoch": 0.5524498520223611,
+      "grad_norm": 0.22524195909500122,
+      "learning_rate": 0.00014517405063291141,
+      "loss": 0.5913956069946289,
+      "step": 2100
+    },
+    {
+      "epoch": 0.5656034199276554,
+      "grad_norm": 0.18563304841518402,
+      "learning_rate": 0.00014385548523206752,
+      "loss": 0.5767417526245118,
+      "step": 2150
+    },
+    {
+      "epoch": 0.5787569878329497,
+      "grad_norm": 0.2004818171262741,
+      "learning_rate": 0.00014253691983122363,
+      "loss": 0.5942864227294922,
+      "step": 2200
+    },
+    {
+      "epoch": 0.591910555738244,
+      "grad_norm": 0.2049906700849533,
+      "learning_rate": 0.00014121835443037977,
+      "loss": 0.571412467956543,
+      "step": 2250
+    },
+    {
+      "epoch": 0.6050641236435383,
+      "grad_norm": 0.19811300933361053,
+      "learning_rate": 0.00013989978902953588,
+      "loss": 0.5904967880249024,
+      "step": 2300
+    },
+    {
+      "epoch": 0.6182176915488327,
+      "grad_norm": 0.2060481756925583,
+      "learning_rate": 0.000138581223628692,
+      "loss": 0.5787500381469727,
+      "step": 2350
+    },
+    {
+      "epoch": 0.631371259454127,
+      "grad_norm": 0.18603748083114624,
+      "learning_rate": 0.00013726265822784812,
+      "loss": 0.5815290832519531,
+      "step": 2400
+    },
+    {
+      "epoch": 0.6445248273594213,
+      "grad_norm": 0.21810391545295715,
+      "learning_rate": 0.00013594409282700423,
+      "loss": 0.5819861221313477,
+      "step": 2450
+    },
+    {
+      "epoch": 0.6576783952647156,
+      "grad_norm": 0.20363149046897888,
+      "learning_rate": 0.00013462552742616034,
+      "loss": 0.5840290069580079,
+      "step": 2500
+    },
+    {
+      "epoch": 0.6708319631700098,
+      "grad_norm": 0.21661260724067688,
+      "learning_rate": 0.00013330696202531648,
+      "loss": 0.5717618179321289,
+      "step": 2550
+    },
+    {
+      "epoch": 0.6839855310753041,
+      "grad_norm": 0.21132707595825195,
+      "learning_rate": 0.0001319883966244726,
+      "loss": 0.5572832489013672,
+      "step": 2600
+    },
+    {
+      "epoch": 0.6971390989805984,
+      "grad_norm": 0.1978549361228943,
+      "learning_rate": 0.0001306698312236287,
+      "loss": 0.5476178741455078,
+      "step": 2650
+    },
+    {
+      "epoch": 0.7102926668858928,
+      "grad_norm": 0.20024415850639343,
+      "learning_rate": 0.0001293512658227848,
+      "loss": 0.5517565155029297,
+      "step": 2700
+    },
+    {
+      "epoch": 0.7234462347911871,
+      "grad_norm": 0.20441296696662903,
+      "learning_rate": 0.00012803270042194094,
+      "loss": 0.5709645462036133,
+      "step": 2750
+    },
+    {
+      "epoch": 0.7365998026964814,
+      "grad_norm": 0.2133142352104187,
+      "learning_rate": 0.00012671413502109705,
+      "loss": 0.5618978881835938,
+      "step": 2800
+    },
+    {
+      "epoch": 0.7497533706017757,
+      "grad_norm": 0.2041858285665512,
+      "learning_rate": 0.00012539556962025316,
+      "loss": 0.5499007034301758,
+      "step": 2850
+    },
+    {
+      "epoch": 0.76290693850707,
+      "grad_norm": 0.23104150593280792,
+      "learning_rate": 0.0001240770042194093,
+      "loss": 0.5771116256713867,
+      "step": 2900
+    },
+    {
+      "epoch": 0.7760605064123643,
+      "grad_norm": 0.20818471908569336,
+      "learning_rate": 0.0001227584388185654,
+      "loss": 0.559295654296875,
+      "step": 2950
+    },
+    {
+      "epoch": 0.7892140743176587,
+      "grad_norm": 0.19819001853466034,
+      "learning_rate": 0.00012143987341772152,
+      "loss": 0.5422830963134766,
+      "step": 3000
+    },
+    {
+      "epoch": 0.802367642222953,
+      "grad_norm": 0.2149215042591095,
+      "learning_rate": 0.00012012130801687764,
+      "loss": 0.5710384368896484,
+      "step": 3050
+    },
+    {
+      "epoch": 0.8155212101282473,
+      "grad_norm": 0.1951475739479065,
+      "learning_rate": 0.00011880274261603376,
+      "loss": 0.5525796127319336,
+      "step": 3100
+    },
+    {
+      "epoch": 0.8286747780335416,
+      "grad_norm": 0.20850563049316406,
+      "learning_rate": 0.00011748417721518987,
+      "loss": 0.5578081512451172,
+      "step": 3150
+    },
+    {
+      "epoch": 0.8418283459388359,
+      "grad_norm": 0.21047161519527435,
+      "learning_rate": 0.000116165611814346,
+      "loss": 0.5627962112426758,
+      "step": 3200
+    },
+    {
+      "epoch": 0.8549819138441302,
+      "grad_norm": 0.20235857367515564,
+      "learning_rate": 0.00011484704641350212,
+      "loss": 0.5524932861328125,
+      "step": 3250
+    },
+    {
+      "epoch": 0.8681354817494246,
+      "grad_norm": 0.23249448835849762,
+      "learning_rate": 0.00011352848101265823,
+      "loss": 0.5648232269287109,
+      "step": 3300
+    },
+    {
+      "epoch": 0.8812890496547189,
+      "grad_norm": 0.21543475985527039,
+      "learning_rate": 0.00011220991561181435,
+      "loss": 0.542757682800293,
+      "step": 3350
+    },
+    {
+      "epoch": 0.8944426175600132,
+      "grad_norm": 0.19726432859897614,
+      "learning_rate": 0.00011089135021097049,
+      "loss": 0.5540407943725586,
+      "step": 3400
+    },
+    {
+      "epoch": 0.9075961854653075,
+      "grad_norm": 0.21633584797382355,
+      "learning_rate": 0.00010957278481012658,
+      "loss": 0.5423409271240235,
+      "step": 3450
+    },
+    {
+      "epoch": 0.9207497533706018,
+      "grad_norm": 0.1918056756258011,
+      "learning_rate": 0.0001082542194092827,
+      "loss": 0.5391794967651368,
+      "step": 3500
+    },
+    {
+      "epoch": 0.933903321275896,
+      "grad_norm": 0.20544877648353577,
+      "learning_rate": 0.00010693565400843884,
+      "loss": 0.5497972106933594,
+      "step": 3550
+    },
+    {
+      "epoch": 0.9470568891811904,
+      "grad_norm": 0.20916621387004852,
+      "learning_rate": 0.00010561708860759494,
+      "loss": 0.5546447372436524,
+      "step": 3600
+    },
+    {
+      "epoch": 0.9602104570864847,
+      "grad_norm": 0.1996736079454422,
+      "learning_rate": 0.00010429852320675106,
+      "loss": 0.5447078704833984,
+      "step": 3650
+    },
+    {
+      "epoch": 0.973364024991779,
+      "grad_norm": 0.21889854967594147,
+      "learning_rate": 0.00010297995780590717,
+      "loss": 0.5630467987060547,
+      "step": 3700
+    },
+    {
+      "epoch": 0.9865175928970733,
+      "grad_norm": 0.22059348225593567,
+      "learning_rate": 0.00010166139240506329,
+      "loss": 0.5289766311645507,
+      "step": 3750
+    },
+    {
+      "epoch": 0.9996711608023676,
+      "grad_norm": 0.20330405235290527,
+      "learning_rate": 0.00010034282700421941,
+      "loss": 0.5389650726318359,
+      "step": 3800
+    },
+    {
+      "epoch": 1.0126274251890826,
+      "grad_norm": 0.2183484435081482,
+      "learning_rate": 9.902426160337554e-05,
+      "loss": 0.5332442474365234,
+      "step": 3850
+    },
+    {
+      "epoch": 1.0257809930943769,
+      "grad_norm": 0.21353483200073242,
+      "learning_rate": 9.770569620253165e-05,
+      "loss": 0.5061777114868165,
+      "step": 3900
+    },
+    {
+      "epoch": 1.0389345609996712,
+      "grad_norm": 0.18714234232902527,
+      "learning_rate": 9.638713080168777e-05,
+      "loss": 0.5156877899169922,
+      "step": 3950
+    },
+    {
+      "epoch": 1.0520881289049655,
+      "grad_norm": 0.21101020276546478,
+      "learning_rate": 9.506856540084389e-05,
+      "loss": 0.5039350891113281,
+      "step": 4000
+    },
+    {
+      "epoch": 1.0652416968102598,
+      "grad_norm": 0.21284659206867218,
+      "learning_rate": 9.375e-05,
+      "loss": 0.5384769439697266,
+      "step": 4050
+    },
+    {
+      "epoch": 1.0783952647155541,
+      "grad_norm": 0.23729759454727173,
+      "learning_rate": 9.243143459915612e-05,
+      "loss": 0.5105631637573242,
+      "step": 4100
+    },
+    {
+      "epoch": 1.0915488326208485,
+      "grad_norm": 0.2321997582912445,
+      "learning_rate": 9.111286919831225e-05,
+      "loss": 0.512237548828125,
+      "step": 4150
+    },
+    {
+      "epoch": 1.1047024005261428,
+      "grad_norm": 0.22301840782165527,
+      "learning_rate": 8.979430379746836e-05,
+      "loss": 0.516036262512207,
+      "step": 4200
+    },
+    {
+      "epoch": 1.117855968431437,
+      "grad_norm": 0.2111876904964447,
+      "learning_rate": 8.847573839662448e-05,
+      "loss": 0.5222226715087891,
+      "step": 4250
+    },
+    {
+      "epoch": 1.1310095363367314,
+      "grad_norm": 0.22087761759757996,
+      "learning_rate": 8.715717299578059e-05,
+      "loss": 0.4861147689819336,
+      "step": 4300
+    },
+    {
+      "epoch": 1.1441631042420257,
+      "grad_norm": 0.22107750177383423,
+      "learning_rate": 8.583860759493671e-05,
+      "loss": 0.5000715255737305,
+      "step": 4350
+    },
+    {
+      "epoch": 1.15731667214732,
+      "grad_norm": 0.22825708985328674,
+      "learning_rate": 8.452004219409283e-05,
+      "loss": 0.5130445861816406,
+      "step": 4400
+    },
+    {
+      "epoch": 1.1704702400526144,
+      "grad_norm": 0.21951164305210114,
+      "learning_rate": 8.320147679324894e-05,
+      "loss": 0.5043642044067382,
+      "step": 4450
+    },
+    {
+      "epoch": 1.1836238079579087,
+      "grad_norm": 0.22362707555294037,
+      "learning_rate": 8.188291139240507e-05,
+      "loss": 0.5210093307495117,
+      "step": 4500
+    },
+    {
+      "epoch": 1.196777375863203,
+      "grad_norm": 0.21215161681175232,
+      "learning_rate": 8.056434599156119e-05,
+      "loss": 0.5018383026123047,
+      "step": 4550
+    },
+    {
+      "epoch": 1.209930943768497,
+      "grad_norm": 0.23212012648582458,
+      "learning_rate": 7.92457805907173e-05,
+      "loss": 0.5148273468017578,
+      "step": 4600
+    },
+    {
+      "epoch": 1.2230845116737914,
+      "grad_norm": 0.22493556141853333,
+      "learning_rate": 7.792721518987342e-05,
+      "loss": 0.5154372787475586,
+      "step": 4650
+    },
+    {
+      "epoch": 1.2362380795790857,
+      "grad_norm": 0.2540408968925476,
+      "learning_rate": 7.660864978902954e-05,
+      "loss": 0.5067441558837891,
+      "step": 4700
+    },
+    {
+      "epoch": 1.24939164748438,
+      "grad_norm": 0.2226845622062683,
+      "learning_rate": 7.529008438818565e-05,
+      "loss": 0.4940685272216797,
+      "step": 4750
+    },
+    {
+      "epoch": 1.2625452153896743,
+      "grad_norm": 0.24263730645179749,
+      "learning_rate": 7.397151898734176e-05,
+      "loss": 0.4980718994140625,
+      "step": 4800
+    },
+    {
+      "epoch": 1.2756987832949687,
+      "grad_norm": 0.20515185594558716,
+      "learning_rate": 7.26529535864979e-05,
+      "loss": 0.4952202606201172,
+      "step": 4850
+    },
+    {
+      "epoch": 1.288852351200263,
+      "grad_norm": 0.2140342742204666,
+      "learning_rate": 7.133438818565401e-05,
+      "loss": 0.47644241333007814,
+      "step": 4900
+    },
+    {
+      "epoch": 1.3020059191055573,
+      "grad_norm": 0.23612593114376068,
+      "learning_rate": 7.001582278481013e-05,
+      "loss": 0.49656723022460936,
+      "step": 4950
+    },
+    {
+      "epoch": 1.3151594870108516,
+      "grad_norm": 0.21949142217636108,
+      "learning_rate": 6.869725738396626e-05,
+      "loss": 0.5016007232666015,
+      "step": 5000
+    },
+    {
+      "epoch": 1.328313054916146,
+      "grad_norm": 0.2435092031955719,
+      "learning_rate": 6.737869198312236e-05,
+      "loss": 0.47929866790771486,
+      "step": 5050
+    },
+    {
+      "epoch": 1.3414666228214402,
+      "grad_norm": 0.2446790486574173,
+      "learning_rate": 6.606012658227849e-05,
+      "loss": 0.5098638153076172,
+      "step": 5100
+    },
+    {
+      "epoch": 1.3546201907267346,
+      "grad_norm": 0.22514788806438446,
+      "learning_rate": 6.474156118143461e-05,
+      "loss": 0.522101058959961,
+      "step": 5150
+    },
+    {
+      "epoch": 1.3677737586320289,
+      "grad_norm": 0.21301281452178955,
+      "learning_rate": 6.342299578059072e-05,
+      "loss": 0.4935792922973633,
+      "step": 5200
+    },
+    {
+      "epoch": 1.3809273265373232,
+      "grad_norm": 0.24475009739398956,
+      "learning_rate": 6.210443037974684e-05,
+      "loss": 0.4931691741943359,
+      "step": 5250
+    },
+    {
+      "epoch": 1.3940808944426175,
+      "grad_norm": 0.215195894241333,
+      "learning_rate": 6.078586497890295e-05,
+      "loss": 0.5058505630493164,
+      "step": 5300
+    },
+    {
+      "epoch": 1.4072344623479118,
+      "grad_norm": 0.22771233320236206,
+      "learning_rate": 5.9467299578059074e-05,
+      "loss": 0.5185404586791992,
+      "step": 5350
+    },
+    {
+      "epoch": 1.4203880302532061,
+      "grad_norm": 0.23890231549739838,
+      "learning_rate": 5.814873417721519e-05,
+      "loss": 0.5046481323242188,
+      "step": 5400
+    },
+    {
+      "epoch": 1.4335415981585005,
+      "grad_norm": 0.22931832075119019,
+      "learning_rate": 5.6830168776371307e-05,
+      "loss": 0.49437793731689456,
+      "step": 5450
+    },
+    {
+      "epoch": 1.4466951660637948,
+      "grad_norm": 0.23015224933624268,
+      "learning_rate": 5.551160337552743e-05,
+      "loss": 0.5133797073364258,
+      "step": 5500
+    },
+    {
+      "epoch": 1.459848733969089,
+      "grad_norm": 0.22849251329898834,
+      "learning_rate": 5.4193037974683546e-05,
+      "loss": 0.4944322204589844,
+      "step": 5550
+    },
+    {
+      "epoch": 1.4730023018743834,
+      "grad_norm": 0.24185696244239807,
+      "learning_rate": 5.287447257383966e-05,
+      "loss": 0.5075090408325196,
+      "step": 5600
+    },
+    {
+      "epoch": 1.4861558697796777,
+      "grad_norm": 0.23637419939041138,
+      "learning_rate": 5.1555907172995784e-05,
+      "loss": 0.5133906936645508,
+      "step": 5650
+    },
+    {
+      "epoch": 1.499309437684972,
+      "grad_norm": 0.24460475146770477,
+      "learning_rate": 5.02373417721519e-05,
+      "loss": 0.5059809494018555,
+      "step": 5700
+    },
+    {
+      "epoch": 1.5124630055902664,
+      "grad_norm": 0.2371147722005844,
+      "learning_rate": 4.891877637130802e-05,
+      "loss": 0.5034146499633789,
+      "step": 5750
+    },
+    {
+      "epoch": 1.5256165734955607,
+      "grad_norm": 0.23984476923942566,
+      "learning_rate": 4.760021097046414e-05,
+      "loss": 0.4908566665649414,
+      "step": 5800
+    },
+    {
+      "epoch": 1.538770141400855,
+      "grad_norm": 0.21663543581962585,
+      "learning_rate": 4.6281645569620256e-05,
+      "loss": 0.5003865814208984,
+      "step": 5850
+    },
+    {
+      "epoch": 1.5519237093061493,
+      "grad_norm": 0.22912858426570892,
+      "learning_rate": 4.496308016877637e-05,
+      "loss": 0.5095011138916016,
+      "step": 5900
+    },
+    {
+      "epoch": 1.5650772772114436,
+      "grad_norm": 0.22724448144435883,
+      "learning_rate": 4.3644514767932495e-05,
+      "loss": 0.495461540222168,
+      "step": 5950
+    },
+    {
+      "epoch": 1.578230845116738,
+      "grad_norm": 0.2396576702594757,
+      "learning_rate": 4.232594936708861e-05,
+      "loss": 0.4898215103149414,
+      "step": 6000
+    },
+    {
+      "epoch": 1.5913844130220323,
+      "grad_norm": 0.2576962113380432,
+      "learning_rate": 4.100738396624473e-05,
+      "loss": 0.48984790802001954,
+      "step": 6050
+    },
+    {
+      "epoch": 1.6045379809273266,
+      "grad_norm": 0.23540741205215454,
+      "learning_rate": 3.968881856540085e-05,
+      "loss": 0.4990139389038086,
+      "step": 6100
+    },
+    {
+      "epoch": 1.617691548832621,
+      "grad_norm": 0.21285173296928406,
+      "learning_rate": 3.8370253164556966e-05,
+      "loss": 0.5028389739990234,
+      "step": 6150
+    },
+    {
+      "epoch": 1.6308451167379152,
+      "grad_norm": 0.21637603640556335,
+      "learning_rate": 3.705168776371308e-05,
+      "loss": 0.5122966384887695,
+      "step": 6200
+    },
+    {
+      "epoch": 1.6439986846432095,
+      "grad_norm": 0.23689775168895721,
+      "learning_rate": 3.57331223628692e-05,
+      "loss": 0.4887590026855469,
+      "step": 6250
+    },
+    {
+      "epoch": 1.6571522525485038,
+      "grad_norm": 0.23213249444961548,
+      "learning_rate": 3.441455696202532e-05,
+      "loss": 0.5122234725952148,
+      "step": 6300
+    },
+    {
+      "epoch": 1.6703058204537982,
+      "grad_norm": 0.24195370078086853,
+      "learning_rate": 3.309599156118144e-05,
+      "loss": 0.501855354309082,
+      "step": 6350
+    },
+    {
+      "epoch": 1.6834593883590925,
+      "grad_norm": 0.21080398559570312,
+      "learning_rate": 3.177742616033755e-05,
+      "loss": 0.4720833969116211,
+      "step": 6400
+    },
+    {
+      "epoch": 1.6966129562643868,
+      "grad_norm": 0.23730884492397308,
+      "learning_rate": 3.0458860759493672e-05,
+      "loss": 0.5111521911621094,
+      "step": 6450
+    },
+    {
+      "epoch": 1.7097665241696811,
+      "grad_norm": 0.2520897388458252,
+      "learning_rate": 2.914029535864979e-05,
+      "loss": 0.5011271667480469,
+      "step": 6500
+    },
+    {
+      "epoch": 1.7229200920749754,
+      "grad_norm": 0.2265089452266693,
+      "learning_rate": 2.7821729957805908e-05,
+      "loss": 0.4725305938720703,
+      "step": 6550
+    },
+    {
+      "epoch": 1.7360736599802697,
+      "grad_norm": 0.23663493990898132,
+      "learning_rate": 2.6503164556962027e-05,
+      "loss": 0.4987929153442383,
+      "step": 6600
+    },
+    {
+      "epoch": 1.749227227885564,
+      "grad_norm": 0.22110675275325775,
+      "learning_rate": 2.5184599156118143e-05,
+      "loss": 0.47567306518554686,
+      "step": 6650
+    },
+    {
+      "epoch": 1.7623807957908584,
+      "grad_norm": 0.2357766330242157,
+      "learning_rate": 2.3866033755274263e-05,
+      "loss": 0.504273910522461,
+      "step": 6700
+    },
+    {
+      "epoch": 1.7755343636961527,
+      "grad_norm": 0.23699399828910828,
+      "learning_rate": 2.2547468354430382e-05,
+      "loss": 0.509364128112793,
+      "step": 6750
+    },
+    {
+      "epoch": 1.788687931601447,
+      "grad_norm": 0.22617210447788239,
+      "learning_rate": 2.12289029535865e-05,
+      "loss": 0.48645030975341796,
+      "step": 6800
     }
   ],
   "logging_steps": 50,
       "attributes": {}
     }
   },
+  "total_flos": 8.47088159254766e+18,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null

last-checkpoint/training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2dcb9fd1efebc7c487e99696e7655a5dce1fb979ef93fc72ec06ba1e079f6ebf
 size 5777

 version https://git-lfs.github.com/spec/v1
+oid sha256:34f353c448b484cfebf8c7e25bfc7699a62f45447bbb965aeca60fbba78dbc3f
 size 5777