Add files using upload-large-folder tool

lm-quant-toolkit/src/lm_quant_toolkit.egg-info/PKG-INFO

@@ -0,0 +1,496 @@
+Metadata-Version: 2.4
+Name: lm_quant_toolkit
+Version: 0.0.1.dev0
+Summary: LLM Quantization Evaluation Harness
+Author-email: Justin Zhang <schnell18@gmail.com>
+License: Copyright 2024 Justin Zhang
+        
+        Permission is hereby granted, free of charge, to any person obtaining a
+        copy of this software and associated documentation files (the
+        “Software”), to deal in the Software without restriction, including
+        without limitation the rights to use, copy, modify, merge, publish,
+        distribute, sublicense, and/or sell copies of the Software, and to
+        permit persons to whom the Software is furnished to do so, subject to
+        the following conditions:
+        
+        THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS
+        OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+        MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+        IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+        CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+        TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+        SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+        
+Project-URL: Homepage, https://github.com/schnell18/lm-quant-toolkit
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT No Attribution License (MIT-0)
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.7
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: Implementation :: CPython
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: accelerate>=0.30.1
+Requires-Dist: bitsandbytes>0.37.0
+Requires-Dist: antlr4-python3-runtime==4.11.0
+Requires-Dist: datasets==2.20.0
+Requires-Dist: Jinja2==3.1.4
+Requires-Dist: langdetect==1.0.9
+Requires-Dist: nltk==3.9.1
+Requires-Dist: numpy==1.26.4
+Requires-Dist: optimum>=1.21.4
+Requires-Dist: pandas==2.2.2
+Requires-Dist: safetensors==0.4.3
+Requires-Dist: scikit-learn==1.4.2
+Requires-Dist: scipy==1.13.0
+Requires-Dist: sentencepiece==0.2.0
+Requires-Dist: tokenizers>=0.19.1
+Requires-Dist: torch>=2.1.0
+Requires-Dist: tqdm==4.66.4
+Requires-Dist: transformers>=4.41.2
+Provides-Extra: dev
+Requires-Dist: bumpver; extra == "dev"
+Requires-Dist: pip-tools; extra == "dev"
+Provides-Extra: test
+Requires-Dist: tox; extra == "test"
+Provides-Extra: doc
+Requires-Dist: sphinx; extra == "doc"
+Dynamic: license-file
+
+# Overview
+
+The **lm-quant-toolkit** is a suite of tools to facilitate large neural network
+quantization research. It includes a quantization harness tool to drive
+quantization experiments on large language models and vision models, to collect
+and summarize experiment data for further analysis. It also includes tool to
+prepare experiment meta data and visualization tools to interpret experiment
+results. Specifically, lm-quant-toolkit consists of:
+
+- LLM quantization harness tool
+- ViT quantization harness tool
+- FNorm Metadata Preparation Tool
+- Kurtosis Metrics Measuring Tool
+- Sensitivity Score Measuring Tool
+- Calibration Dataset Generation Tool
+- Visualization Tools
+
+## Citation
+
+~~~~
+@inproceedings{zhang2025mxq,
+  title = {A Mixed Quantization Approach for Data-Free Quantization of LLMs},
+  author = {Feng Zhang and Yanbin Liu and Weihua Li and Xiaodan Wang and Quan Bai},
+  year = {2025},
+  url = {https://openreview.net/forum?id=M3Y74vmsMcY},
+}
+~~~~
+
+## Setup test harness
+
+Most tools are implemented in Python and are extensively tested under the
+Python 3.11.9. The visualization tools are implemented in R. The usages of
+these tools are elaborated in the following sections. This section describes
+how to setup the lm-quant-toolkit and the companion visualization tools.
+
+The Python tools dependend on Python libraries such as transformers, datasets,
+numpy, PyTorch etc. A few Python libraries are patched to support MXQ.
+Specifically, required patched dependencies include AutoGPTQ (for CUDA 12.5
+compatibility), HQQ (support MXQ extension), lm_eval (for end-to-end LLM
+performance evaluation), clip_benchmark (for vision model evaluation). These
+dependencies are installed automatically as part of setup process. To setup the
+Python tools, follow this procedure:
+
+- Ensure Python and miniconda are installed
+- Create a Python virtual enivonrment using Python 3.11.9 and activate this enivonrment
+- Clone the lm-quant-toolkit project from [the lm-quant-toolkit project][2]
+- Run the script setup-harness.sh under the root directory of the lm-quant-toolkit project
+
+Or simply use the convenient script `setup-harness.sh` included this project.
+
+## Setup visualization tools
+
+The visualization tools are R scripts to transform, aggregate and visualize
+experiment results. They are wrapped in bash scripts to automate the whole
+experiment loop, which consists of model quantization, perplex evaluation,
+memory consumption test and experiment report generation. The R visualization
+scripts can also be used separately. To setup the visualization tools, please
+follow this procedure:
+
+- Ensure a recent version of R, for instance R 4.4.1, is installed.
+- Optionally, RStudio could be installed to extend and trouble shoot the
+visualization tools in an intuitive enivonrment.
+- Install the third-party packages required by the visualization tools by
+running the script `setup-visualization.sh` under the root directory of the
+lm-quant-toolkit project.
+
+# Quantization tool usage
+
+## LLM Quantization Harness Tool
+
+This tool executes various quantization tasks and runs diverse evaluation
+benchmarks such as perplexity, GPU memory usage, quantized model storage. It
+also supports end-to-end LLM performance evaluation through the integration
+with the `lm-eval` tool. This harness tool works with various state-of-art
+quantization methods such as GPTQ, AWQ, BitsAndBytes and HQQ, which enables a
+fair comparison between the proposed methods and the state-of-art baselines.
+Furthermore, it facilitates the complex and time-consuming benchmarking tasks
+by offering resumption from failed subtasks, aggregate subtask's evaluation
+results. Lastly, this tool provides declarative CLI interface to ease complex
+experiment automation through shell scripting.
+
+## FNorm Metadata Preparation Tool
+
+This tool calculates the Frobenius norms, a.k.a FNorm, of the quantization
+errors of all weight matricies inside a particular large language model. The
+FNorm meta-data are crucial to the MXQ quantization scheme as it guides MXQ to
+allocate optimal quantization configurations. This tool accepts a list of
+Hugging Face-compliant model identifiers. The output of this tool is a series
+of .csv files under specified directory. Each file contains the Frobenius
+norms for the 12 quantization configurations.
+
+The tool is implemented in Python and provides a convenient CLI interface to
+enable shell scripting. It is located separately in the `dump.py` file
+under the `src` folder in the `lm-quant-toolkit` project, which helps
+to reduce unnecessary dependencies. A typical usage is demonstrated in the code
+snippet as follows:
+
+~~~~bash
+#!/bin/bash
+
+TOOLKIT_DIR="$HOME/work/lm-quant-toolkit"
+MODELS="meta-llama/Llama-2-7b-hf meta-llama/Llama-2-13b-hf meta-llama/Llama-3.1-8B"
+mkdir -p /tmp/fnorm-dump
+python $TOOLKIT_DIR/src/dump.py fnorm \
+    --model $MODELS \
+    --output-dir /tmp/fnorm-dump
+~~~~
+
+
+## Kurtosis Metrics Measuring Tool
+
+This tool calculates the Kurtosis metrics of weight matricies layer-by-layer
+inside a particular large language model. The Kurtosis metrcis are crucial to
+identify sensitive layers to improve the accuracy of MXQ quantization. This
+tool accepts a list of Hugging Face-compliant model identifiers. The output of
+this tool is a series of .csv files under specified directory. Each file
+contains the Kurtosis metrics for corresponding models.
+
+The tool is implemented in Python and provides a convenient CLI interface to
+enable shell scripting. It is included in the `dump.py` file under the
+`src` folder in the `lm-quant-toolkit` project. A typical usage is
+demonstrated in the code snippet as follows:
+
+~~~~bash
+#!/bin/bash
+
+MODELS="meta-llama/Llama-2-7b-hf meta-llama/Llama-2-13b-hf meta-llama/Meta-Llama-3-8B"
+mkdir -p /tmp/kurtosis-dump
+python ../src/dump.py kurtosis \
+    --model $MODELS \
+    --output-dir /tmp/kurtosis-dump
+~~~~
+
+This code snippet demonstrates dumping the kurtosis metrics for the three Llama
+models into the `/tmp/kurtosis-dump` directory.
+
+## Sensitivity Score Measuring Tool
+
+This tool calculates the sensitivity score of each layer of a particular large
+language model. The sensitivity score are crucial to identify sensitive layers
+to improve the accuracy of MXQ quantization. This tool accepts a list of
+Hugging Face-compliant model identifiers. The output of this tool is a series
+of .csv files, each contains the sensitivity score for corresponding model.
+These files are crucial inputs to guide the SensiBoost and Sensitivity-based
+MiLP.
+
+The tool is implemented in Python and provides a convenient CLI interface to
+enable shell scripting. It is compatible with any transformer-based LLMs with
+an implementation of the popular Hugging Face transformers library. It is
+located separately in the `dump.py` file under the `src` folder in
+the `lm-quant-toolkit` project, which helps to reduce unnecessary
+dependencies. A typical usage is demonstrated in the code snippet as follows:
+
+
+~~~~bash
+#!/bin/bash
+
+TOOLKIT_DIR="$HOME/work/lm-quant-toolkit"
+RESULT_BASE_DIR="/data/llm/mxq/results"
+CALIB_DATASETS="bos pileval wikitext c4"
+CONFIGS="b2g128 b2g64 b2g32 b3g128 b3g64 b3g32 b4g128 b4g64 b4g32 b8g128 b8g64 b8g32"
+MODELS="Qwen/Qwen2.5-7B Qwen/Qwen2.5-Coder-7B Qwen/Qwen2.5-Coder-7B-Instruct Qwen/Qwen2.5-Math-7B"
+
+EXP_NAME=sensi_qwen25
+RESULT_DIR=$RESULT_BASE_DIR/$EXP_NAME
+mkdir -p $RESULT_DIR/data
+
+for DS in $CALIB_DATASETS; do
+    for CFG in $CONFIGS; do
+        for MODEL in $MODELS; do
+            SHORT_ID=$(echo $MODEL | cut -d/ -f2)
+            OUT_FILE="${RESULT_DIR}/data/qwen25-sensi-${SHORT_ID}-${CFG}-${DS}.csv"
+            python $TOOLKIT_DIR/src/dump.py sensi \
+                --model $MODEL \
+                --config $CFG \
+                --calib-dataset $DS \
+                --output-file $OUT_FILE
+        done
+    done
+done
+~~~~
+
+The code snippet demonstrates how to calculate the sensitivity scores for a
+series of Qwen2.5 models using 4 calibration datasets under 12 bit budgets.
+
+## Calibration Dataset Generation Tool
+
+This tool generates a small synthensized dataset named branch of science
+(denoted as BoS, published on Hugging Face), which includes a few hundred of
+textual defintions for science, art and business topics such as Mathematics,
+Physics, Chemstry, Law, Music and Journalism etc. The dataset is intended to
+validate if the sensitivity property generalize to diverse datasets.
+
+The tool generates an initial dataset in .csv format which requires further
+processing. The output of this tool is random due to the generative nature of
+LLM. This tool requires a Llama-2-7B model being served with an OpenAI
+compatible RESTful API endpoint. User can either use a hosted API endpoint or
+deploy a local instance by following the instruction at the end of this section.
+
+Once the API endpoint is secured, run the following script to generate the BoS dataset:
+
+~~~~bash
+#!/bin/bash
+
+TOOLKIT_DIR="$HOME/work/lm-quant-toolkit"
+
+$TOOLKIT_DIR/utils/generate.py \
+  --model="meta-llama/Llama-2-7b-chat-hf" \
+  --variant="vLLM" \
+  --topic-file=topics-l1.txt \
+  --trace
+~~~~
+Lastly, find the result in the csv files under current directory.
+
+### Local API endpoint
+To deploy a local API endpoint using vLLM, create a virtual environment using
+`conda` as follows:
+
+~~~~bash
+conda create -n vllm python=3.11 -y
+conda activate vllm
+pip install vllm==0.6.4.post1
+~~~~
+Then configure and launch the API server
+~~~~bash
+#!/bin/bash
+
+vllm serve meta-llama/Llama-2-7b-chat-hf --dtype auto --api-key token-abc123
+~~~~
+Watch the output vLLm to make sure it starts up successfully.
+
+# Visualization Tool usage
+
+The visualization tools facilitate visualizing the experiment results and the
+weight distribution, and generating insights of the latent features to quantize
+LLMs more efficiently. Most visualization tools are implemented in R and
+leverages the open-source plot libraries such as ggplot2, circlize, ggbreak,
+ggmagnify. They provide CLI interface to simplify integaration with the
+quantization harness tool.
+
+These CLI tools support diverse options to allow user specify input dataset,
+select particular model or approach to plot. To get help on these specific CLI
+options, type `./plot_xxx.R --help` on command line prompt. For instance,
+to get help on the MXQ allocation visualization tool, you may run command as
+follows:
+
+~~~~bash
+./plot-mxq-allocation.R --help
+Usage: ./plot-mxq-allocation.R [options]
+
+Options:
+        -h, --help
+                Show this help message and exit
+
+        -m CHARACTER, --model=CHARACTER
+                Model ID
+
+        -b DOUBLE, --budget=DOUBLE
+                Bit Budget
+
+        -d CHARACTER, --baseline_data_dir=CHARACTER
+                Data directory of baseline results
+
+        -q CHARACTER, --quant_cfg_allot_file=CHARACTER
+                The combined quant config allocation csv file
+
+        --attempt1=CHARACTER
+                The first attempt to plot
+
+        --attempt2=CHARACTER
+                The second attempt to plot
+
+        --fnorm
+                Display FNorm value in the bar chart
+~~~~
+
+## Weight Distribution Visualization Tool
+
+This tool enables visualizing layer-wised weight distribution of large language
+models. It is implemented as an R script, which provides a convenient CLI
+interface to enable shell scripting. Given a weight distribution metrics csv
+file, it produces a pdf file under the `pdfs` with 3x3 sub-plots of column
+digrams for the 9 modules in the Llama family models.
+
+The tool is named `plot-wdist-llm.R` and located under the
+`data-vis` folder in the `lm-quant-toolkit` project. A typical usage
+is demonstrated in the code snippet as follows:
+
+~~~~bash
+#!/bin/bash
+
+TOOLKIT_DIR="$HOME/work/lm-quant-toolkit"
+
+$TOOLKIT_DIR/data-vis/plot-wdist-llm.R -m Llama-2-7b-hf
+~~~~
+
+## Perplexity vs Bit Budget Visualization Tool
+
+This tool enables visualizing the relationship between perplexity and bit
+budget for diverse MXQ experiments against their baselines. The generated
+diagram shows how memory reduction affects perplexity, which facilitates
+memory-accuracy trade-off.
+
+The tool is named `plot-ppl-mem.R` and located under the `data-vis`
+folder in the `lm-quant-toolkit` project. It accepts a csv file containing
+the perplexity metrics of MXQ and its baselines. The output are series of PDF
+files corresponding to the models defined in the input file, which are placed
+under the `pdfs` subfolder. A typical usage is demonstrated in the code
+snippet as follows:
+
+~~~~bash
+#!/bin/bash
+
+TOOLKIT_DIR="$HOME/work/lm-quant-toolkit"
+
+$TOOLKIT_DIR/data-vis/plot-ppl-mem.R -d data/combined.csv
+~~~~
+
+## Quantization Speed Comparison Visualization Tool
+
+This tool generates column digrams to explore the quantization speed among
+various approaches. The tool is also implemented as an R script, which provides a
+convenient CLI interface to enable shell scripting. The tool is named
+`plot-quant-speed.R` and located under the `data-vis` folder in the
+`lm-quant-toolkit` project. Given a combined perplexity metrics csv file,
+it produces a column digrams with x-axis in log-scale. Similar to other tools,
+the PDF file is placed under the `pdfs` subfolder. A typical usage is
+demonstrated in the code snippet as follows:
+
+~~~~bash
+#!/bin/bash
+
+TOOLKIT_DIR="$HOME/work/lm-quant-toolkit"
+
+$TOOLKIT_DIR/data-vis/plot-quant-speed.R -d data/combined.csv
+~~~~
+
+## GPU Memory Usage Visualization Tool
+
+This tool generates column digrams to present the actual GPU memory consumption
+of LLMs quantized by diverse methods. The tool is also implemented as an R script,
+which provides a convenient CLI interface to enable shell scripting. The tool
+is named `plot-mem-consumption.R` and located under the `data-vis`
+folder in the `lm-quant-toolkit` project. Given a combined perplexity
+metrics csv file, it produces a column digrams of GPU memory usage in
+Giga-byte. Similar to other tools, the PDF file is placed under the `pdfs`
+subfolder. A typical usage is demonstrated in the code snippet as follows:
+
+~~~~bash
+#!/bin/bash
+
+TOOLKIT_DIR="$HOME/work/lm-quant-toolkit"
+
+$TOOLKIT_DIR/data-vis/plot-mem-consumption.R -d data/combined.csv
+~~~~
+
+## Quantization Configuration Allocation Visualization Tool
+
+This tool offers insights into the way MXQ and its variants allocate bit budget
+to modules and layers. The variants, a.k.a. attempt, to include in the plot are
+configurable. A maximium of 4 variants can be plotted in a circular layout
+thanks to plot library circlize \citep{zuguang_2014}.
+The first input expected by the tool is a combined quantization configuration
+allocation csv file which should include experiment outcome for diverse methods
+such as HQQ and MXQ. The second parameter is the directory where Frobenius
+norms csv files are located. The third parameter is the perplexity score csv
+file. The tool produces circos digram in PDF format.
+
+The tool is named `plot-circos-allot.R` and located under the
+`data-vis` folder in the `lm-quant-toolkit` project. A typical usage
+is demonstrated in the code snippet as follows:
+
+~~~~bash
+#!/bin/bash
+
+TOOLKIT_DIR="../../.."
+
+MODELS="
+Llama-3-7b-hf
+Llama-3-13b-hf
+Meta-Llama-3-8B
+"
+BUDGETS="4.25 3.51"
+
+STOP=2
+TOPM=2
+for MODEL in $MODELS; do
+    for BUDGET in $BUDGETS; do
+        $TOOLKIT_DIR/data-vis/plot-circos-allot.R \
+          --model $MODEL \
+          --budget $BUDGET \
+          --fnorm_data_dir $TOOLKIT_DIR/src/data/ \
+          --ppl_csv_file data/combined.csv \
+          --quant_cfg_allot_file data/quant-cfg-allocation.csv \
+          --attempt1 sensi-boost-${STOP}-${TOPM} \
+          --attempt2 kurt-boost-${STOP}-${TOPM} \
+          --attempt3 hqq\
+          --attempt4 mxq1
+    done
+done
+~~~~
+
+This code snippet demonstrates how to generate a quant config allocation
+comparison diagram to examine the nuanced difference between the SensiBoost and
+kurtBoost approaches, with a stop of 2 and top-{m} 2, as well as the HQQ and
+MXQ baselines.
+
+## SensiBoost/KurtBoost Win-Tie-Loss Visualization Tool
+
+This tool enables qualitative analysis of effectiveness of the proposed
+SensiBoost and KurtBoost methods. It is implemented as an R script, which
+provides a conventional CLI interface to ease automation.
+Given a combined perplexity metrics csv file, it produces a series of column
+digrams in PDF format. The csv file should include experiment outcome for
+SensiBoost, KurtBoost, the ablation tests or baseline such as HQQ and MXQ. The
+name experiment, a.k.a. attempt, should follow the pattern
+`<method>-<stop>-<top m>`.
+
+This tool is included in the `lm-quant-toolkit` under `data-vis`
+folder. A typical usage is demonstrated in the code snippet as follows:
+
+~~~~bash
+#!/bin/bash
+
+TOOLKIT_DIR="$HOME/work/lm-quant-toolkit"
+
+$TOOLKIT_DIR/data-vis/plot-win-tie-loss.R -f data/combined.csv
+~~~~
+
+[1]: https://huggingface.co/docs/leaderboards/leaderboards/intro
+[2]: https://github.com/schnell18/lm-quant-toolkit.git

lm-quant-toolkit/src/lm_quant_toolkit.egg-info/top_level.txt

@@ -0,0 +1,4 @@
+cli
+data
+dump
+lm_quant_toolkit

lm-quant-toolkit/tmp/kurtosis-dump/Llama-3.1-70B-Instruct/kurtosis-models.csv

@@ -0,0 +1,561 @@
+module,layer,param_count,kurtosis
+self_attn.q_proj,0,67108864,50077.07020777786
+self_attn.k_proj,0,8388608,39460.79179477061
+self_attn.v_proj,0,8388608,9714.34062603725
+self_attn.o_proj,0,67108864,10.904280975829113
+mlp.gate_proj,0,234881024,152197.48465199952
+mlp.down_proj,0,234881024,32.99856251662286
+mlp.up_proj,0,234881024,119950.58806774796
+self_attn.q_proj,1,67108864,7634.308020776232
+self_attn.k_proj,1,8388608,4045.9008739798405
+self_attn.v_proj,1,8388608,9760.690767305987
+self_attn.o_proj,1,67108864,5.247417587827986
+mlp.gate_proj,1,234881024,1677.4783417349631
+mlp.down_proj,1,234881024,3.7038335284892256
+mlp.up_proj,1,234881024,267.9390012342344
+self_attn.q_proj,2,67108864,1373.5682851432819
+self_attn.k_proj,2,8388608,1111.6096014243033
+self_attn.v_proj,2,8388608,10306.363888292855
+self_attn.o_proj,2,67108864,3.6552611309880483
+mlp.gate_proj,2,234881024,851.6969454109125
+mlp.down_proj,2,234881024,3.3072159525903313
+mlp.up_proj,2,234881024,922.1415404404745
+self_attn.q_proj,3,67108864,17541.425851319673
+self_attn.k_proj,3,8388608,20784.26100605871
+self_attn.v_proj,3,8388608,25673.00689440138
+self_attn.o_proj,3,67108864,3.644268034206665
+mlp.gate_proj,3,234881024,15691.560458318761
+mlp.down_proj,3,234881024,40.45401999822738
+mlp.up_proj,3,234881024,2539.000471338992
+self_attn.q_proj,4,67108864,7.01199062448764
+self_attn.k_proj,4,8388608,9.370548461185491
+self_attn.v_proj,4,8388608,3.422610565498518
+self_attn.o_proj,4,67108864,6.290687861578241
+mlp.gate_proj,4,234881024,1590.719586515775
+mlp.down_proj,4,234881024,3.816093831675206
+mlp.up_proj,4,234881024,1277.63775095725
+self_attn.q_proj,5,67108864,23.47911895266793
+self_attn.k_proj,5,8388608,677.3984071986448
+self_attn.v_proj,5,8388608,894.4273846075629
+self_attn.o_proj,5,67108864,3.296181507520106
+mlp.gate_proj,5,234881024,18141.760474590814
+mlp.down_proj,5,234881024,5.569542037487064
+mlp.up_proj,5,234881024,19741.460966568164
+self_attn.q_proj,6,67108864,4.737472510077891
+self_attn.k_proj,6,8388608,6.28482435030699
+self_attn.v_proj,6,8388608,3.1949319558664526
+self_attn.o_proj,6,67108864,3.1876794093984895
+mlp.gate_proj,6,234881024,16.97415898400329
+mlp.down_proj,6,234881024,3.107273236401777
+mlp.up_proj,6,234881024,17.278935340937498
+self_attn.q_proj,7,67108864,5.3062298822771
+self_attn.k_proj,7,8388608,6.3989575820638755
+self_attn.v_proj,7,8388608,3.319259655615111
+self_attn.o_proj,7,67108864,3.2268536713778437
+mlp.gate_proj,7,234881024,4.412707019425456
+mlp.down_proj,7,234881024,3.2622001389028434
+mlp.up_proj,7,234881024,4.453036431734782
+self_attn.q_proj,8,67108864,4.225744503191156
+self_attn.k_proj,8,8388608,5.538586485202242
+self_attn.v_proj,8,8388608,3.145357959348748
+self_attn.o_proj,8,67108864,3.3075502773385597
+mlp.gate_proj,8,234881024,605.4065863153668
+mlp.down_proj,8,234881024,3.148875135642212
+mlp.up_proj,8,234881024,545.2094768954765
+self_attn.q_proj,9,67108864,4.626318521871889
+self_attn.k_proj,9,8388608,18.110182144188688
+self_attn.v_proj,9,8388608,3.4127233382476083
+self_attn.o_proj,9,67108864,3.343338909146038
+mlp.gate_proj,9,234881024,41175.84235461787
+mlp.down_proj,9,234881024,3.38088918027413
+mlp.up_proj,9,234881024,43223.504083877364
+self_attn.q_proj,10,67108864,4.661802680249256
+self_attn.k_proj,10,8388608,5.933341926126116
+self_attn.v_proj,10,8388608,3.287763426474098
+self_attn.o_proj,10,67108864,3.198276928060741
+mlp.gate_proj,10,234881024,4.2201623631799094
+mlp.down_proj,10,234881024,3.1242473850967833
+mlp.up_proj,10,234881024,4.260781494912422
+self_attn.q_proj,11,67108864,4.326732839920102
+self_attn.k_proj,11,8388608,96.2013334283494
+self_attn.v_proj,11,8388608,11.112308182238634
+self_attn.o_proj,11,67108864,3.1117830457099296
+mlp.gate_proj,11,234881024,90.8949857257992
+mlp.down_proj,11,234881024,3.063019991017979
+mlp.up_proj,11,234881024,90.09881557668906
+self_attn.q_proj,12,67108864,4.654632209401089
+self_attn.k_proj,12,8388608,6.768848948834895
+self_attn.v_proj,12,8388608,3.134953599249177
+self_attn.o_proj,12,67108864,3.1950263977919664
+mlp.gate_proj,12,234881024,3.455442706225709
+mlp.down_proj,12,234881024,3.1220996391185625
+mlp.up_proj,12,234881024,3.41490427342156
+self_attn.q_proj,13,67108864,3.921981518844444
+self_attn.k_proj,13,8388608,5.73077593166267
+self_attn.v_proj,13,8388608,3.1558259054853184
+self_attn.o_proj,13,67108864,3.209088296657055
+mlp.gate_proj,13,234881024,3.26340141239501
+mlp.down_proj,13,234881024,3.1174169514091696
+mlp.up_proj,13,234881024,3.182833914362146
+self_attn.q_proj,14,67108864,4.493911728019046
+self_attn.k_proj,14,8388608,5.603381075339063
+self_attn.v_proj,14,8388608,3.3681821467740556
+self_attn.o_proj,14,67108864,3.1366976135822657
+mlp.gate_proj,14,234881024,3.24841837827099
+mlp.down_proj,14,234881024,3.1350513421324404
+mlp.up_proj,14,234881024,3.1245283054662205
+self_attn.q_proj,15,67108864,4.202784172149532
+self_attn.k_proj,15,8388608,5.303467145281563
+self_attn.v_proj,15,8388608,3.3026273414789435
+self_attn.o_proj,15,67108864,3.094438221411981
+mlp.gate_proj,15,234881024,3.2920384953785344
+mlp.down_proj,15,234881024,3.144731145462784
+mlp.up_proj,15,234881024,3.1136725779062746
+self_attn.q_proj,16,67108864,4.127041513677288
+self_attn.k_proj,16,8388608,4.4027329851517285
+self_attn.v_proj,16,8388608,3.1021911892141
+self_attn.o_proj,16,67108864,3.1486319735274955
+mlp.gate_proj,16,234881024,3.651930324792214
+mlp.down_proj,16,234881024,3.201778629319989
+mlp.up_proj,16,234881024,3.130698984716068
+self_attn.q_proj,17,67108864,3.9219029798622382
+self_attn.k_proj,17,8388608,4.590671372379875
+self_attn.v_proj,17,8388608,3.1846241749351685
+self_attn.o_proj,17,67108864,3.165195106970997
+mlp.gate_proj,17,234881024,3.2894933585832495
+mlp.down_proj,17,234881024,3.2290879248460866
+mlp.up_proj,17,234881024,3.0879444621515773
+self_attn.q_proj,18,67108864,4.290286031152325
+self_attn.k_proj,18,8388608,4.276753877997762
+self_attn.v_proj,18,8388608,3.2636621659918172
+self_attn.o_proj,18,67108864,3.126792161775731
+mlp.gate_proj,18,234881024,3.2806371332708464
+mlp.down_proj,18,234881024,3.6343741321062284
+mlp.up_proj,18,234881024,3.078617792985047
+self_attn.q_proj,19,67108864,4.333680489415027
+self_attn.k_proj,19,8388608,4.312923523593489
+self_attn.v_proj,19,8388608,3.343346934453593
+self_attn.o_proj,19,67108864,3.1445077883279864
+mlp.gate_proj,19,234881024,3.3251958588835886
+mlp.down_proj,19,234881024,3.286983003187836
+mlp.up_proj,19,234881024,3.058443864229761
+self_attn.q_proj,20,67108864,3.6023438221862927
+self_attn.k_proj,20,8388608,4.305761150060791
+self_attn.v_proj,20,8388608,3.303898774945715
+self_attn.o_proj,20,67108864,3.179063247417259
+mlp.gate_proj,20,234881024,3.3034934581311526
+mlp.down_proj,20,234881024,3.193835605356857
+mlp.up_proj,20,234881024,3.0634256403657854
+self_attn.q_proj,21,67108864,3.833524025878901
+self_attn.k_proj,21,8388608,4.283864774362631
+self_attn.v_proj,21,8388608,3.4179391228727534
+self_attn.o_proj,21,67108864,3.2527775780334185
+mlp.gate_proj,21,234881024,3.295837951674078
+mlp.down_proj,21,234881024,3.7189961017716255
+mlp.up_proj,21,234881024,3.0539597013693713
+self_attn.q_proj,22,67108864,3.992485829720125
+self_attn.k_proj,22,8388608,4.2401274149792165
+self_attn.v_proj,22,8388608,3.276660802302844
+self_attn.o_proj,22,67108864,3.4019412971533076
+mlp.gate_proj,22,234881024,3.2881559146208823
+mlp.down_proj,22,234881024,3.2575832089250785
+mlp.up_proj,22,234881024,3.0809675905531013
+self_attn.q_proj,23,67108864,3.7834554205574036
+self_attn.k_proj,23,8388608,4.3626993857914895
+self_attn.v_proj,23,8388608,3.3862458002334996
+self_attn.o_proj,23,67108864,3.471488491687408
+mlp.gate_proj,23,234881024,3.2490968555754813
+mlp.down_proj,23,234881024,3.2110399954331075
+mlp.up_proj,23,234881024,3.063835950784802
+self_attn.q_proj,24,67108864,4.19494418979753
+self_attn.k_proj,24,8388608,5.4925369681879035
+self_attn.v_proj,24,8388608,3.405025286198556
+self_attn.o_proj,24,67108864,4.404597607247822
+mlp.gate_proj,24,234881024,3.3143801477577526
+mlp.down_proj,24,234881024,3.476402186670993
+mlp.up_proj,24,234881024,3.111574499777276
+self_attn.q_proj,25,67108864,3.905361339628942
+self_attn.k_proj,25,8388608,4.396242809412252
+self_attn.v_proj,25,8388608,3.4801073221211665
+self_attn.o_proj,25,67108864,4.346419880637028
+mlp.gate_proj,25,234881024,3.2959757912710255
+mlp.down_proj,25,234881024,3.582946617930683
+mlp.up_proj,25,234881024,3.184299854260005
+self_attn.q_proj,26,67108864,3.827512021367978
+self_attn.k_proj,26,8388608,4.742099995628205
+self_attn.v_proj,26,8388608,3.2743583002216603
+self_attn.o_proj,26,67108864,3.7991034370737884
+mlp.gate_proj,26,234881024,3.3295786660387647
+mlp.down_proj,26,234881024,3.6742888947812924
+mlp.up_proj,26,234881024,3.097896273559521
+self_attn.q_proj,27,67108864,4.000722310123793
+self_attn.k_proj,27,8388608,4.761179373465575
+self_attn.v_proj,27,8388608,3.3805942543508567
+self_attn.o_proj,27,67108864,3.9302469130836553
+mlp.gate_proj,27,234881024,3.3290525422149178
+mlp.down_proj,27,234881024,3.7291236574128996
+mlp.up_proj,27,234881024,3.079679387144104
+self_attn.q_proj,28,67108864,3.902287580872323
+self_attn.k_proj,28,8388608,4.5608528245784274
+self_attn.v_proj,28,8388608,3.739223336201326
+self_attn.o_proj,28,67108864,7.24242637946067
+mlp.gate_proj,28,234881024,3.8712063376828327
+mlp.down_proj,28,234881024,3.7624383998724733
+mlp.up_proj,28,234881024,3.076614660829312
+self_attn.q_proj,29,67108864,3.9688353916017136
+self_attn.k_proj,29,8388608,4.591889844946916
+self_attn.v_proj,29,8388608,3.469078618129888
+self_attn.o_proj,29,67108864,6.240643470112109
+mlp.gate_proj,29,234881024,3.5654406909275527
+mlp.down_proj,29,234881024,4.309764414846158
+mlp.up_proj,29,234881024,3.0865097028227115
+self_attn.q_proj,30,67108864,4.056995089736469
+self_attn.k_proj,30,8388608,4.556484167103538
+self_attn.v_proj,30,8388608,3.4272413888611424
+self_attn.o_proj,30,67108864,3.96354451055712
+mlp.gate_proj,30,234881024,3.4539773814036923
+mlp.down_proj,30,234881024,4.588225623591116
+mlp.up_proj,30,234881024,3.0906250142792953
+self_attn.q_proj,31,67108864,4.185270022245394
+self_attn.k_proj,31,8388608,4.687624633602759
+self_attn.v_proj,31,8388608,3.4727076268711397
+self_attn.o_proj,31,67108864,4.348714489183227
+mlp.gate_proj,31,234881024,3.817640211361434
+mlp.down_proj,31,234881024,3.335366609118617
+mlp.up_proj,31,234881024,3.051595821169206
+self_attn.q_proj,32,67108864,4.387007660043509
+self_attn.k_proj,32,8388608,4.179708473434365
+self_attn.v_proj,32,8388608,3.178540419936189
+self_attn.o_proj,32,67108864,4.914118061236895
+mlp.gate_proj,32,234881024,3.4238079623887097
+mlp.down_proj,32,234881024,5.273825804418633
+mlp.up_proj,32,234881024,3.0798650830344285
+self_attn.q_proj,33,67108864,4.022721461531028
+self_attn.k_proj,33,8388608,4.408855825740268
+self_attn.v_proj,33,8388608,3.170997521692167
+self_attn.o_proj,33,67108864,4.953800662765951
+mlp.gate_proj,33,234881024,1709077.427270148
+mlp.down_proj,33,234881024,4.052988965242056
+mlp.up_proj,33,234881024,187748.3601954899
+self_attn.q_proj,34,67108864,3.950555186643392
+self_attn.k_proj,34,8388608,4.194768658417521
+self_attn.v_proj,34,8388608,3.373397566556314
+self_attn.o_proj,34,67108864,8.461143044426993
+mlp.gate_proj,34,234881024,3.5646992436150136
+mlp.down_proj,34,234881024,4.1950956436748355
+mlp.up_proj,34,234881024,3.0744769333144277
+self_attn.q_proj,35,67108864,4.2799173607864915
+self_attn.k_proj,35,8388608,4.742608118838655
+self_attn.v_proj,35,8388608,3.3094053936935097
+self_attn.o_proj,35,67108864,13.490826093463431
+mlp.gate_proj,35,234881024,3.541410837621408
+mlp.down_proj,35,234881024,3.8364806950362302
+mlp.up_proj,35,234881024,3.051901104742366
+self_attn.q_proj,36,67108864,4.298105408048522
+self_attn.k_proj,36,8388608,4.968571163775364
+self_attn.v_proj,36,8388608,3.3535770164237726
+self_attn.o_proj,36,67108864,3.8021700550970867
+mlp.gate_proj,36,234881024,3.7106374972232437
+mlp.down_proj,36,234881024,4.568099119113231
+mlp.up_proj,36,234881024,3.056284167397647
+self_attn.q_proj,37,67108864,3.8942040473468973
+self_attn.k_proj,37,8388608,4.266784990165495
+self_attn.v_proj,37,8388608,3.1941692267400326
+self_attn.o_proj,37,67108864,8.401760689615529
+mlp.gate_proj,37,234881024,3.4724631025690837
+mlp.down_proj,37,234881024,3.961450262879031
+mlp.up_proj,37,234881024,3.0788392989087625
+self_attn.q_proj,38,67108864,4.598862785282816
+self_attn.k_proj,38,8388608,4.734314479741277
+self_attn.v_proj,38,8388608,3.1909430361447195
+self_attn.o_proj,38,67108864,3.736047554500255
+mlp.gate_proj,38,234881024,3.766540270435748
+mlp.down_proj,38,234881024,3.865464336298293
+mlp.up_proj,38,234881024,3.076181770653106
+self_attn.q_proj,39,67108864,4.483846534083388
+self_attn.k_proj,39,8388608,5.093267262151783
+self_attn.v_proj,39,8388608,3.210445757926791
+self_attn.o_proj,39,67108864,14.452447009013284
+mlp.gate_proj,39,234881024,3.5009372454866106
+mlp.down_proj,39,234881024,4.892350334132089
+mlp.up_proj,39,234881024,3.05297996222971
+self_attn.q_proj,40,67108864,4.622020522551382
+self_attn.k_proj,40,8388608,4.498165230655478
+self_attn.v_proj,40,8388608,3.122335529947702
+self_attn.o_proj,40,67108864,3.8993528623696445
+mlp.gate_proj,40,234881024,3.4732290329037
+mlp.down_proj,40,234881024,3.883320989331787
+mlp.up_proj,40,234881024,3.0684329438006115
+self_attn.q_proj,41,67108864,4.244571736174325
+self_attn.k_proj,41,8388608,4.635358166841929
+self_attn.v_proj,41,8388608,3.1138257909092064
+self_attn.o_proj,41,67108864,3.274012576878743
+mlp.gate_proj,41,234881024,3.374611709234885
+mlp.down_proj,41,234881024,4.488186967654265
+mlp.up_proj,41,234881024,3.0309117779519803
+self_attn.q_proj,42,67108864,4.726926148746448
+self_attn.k_proj,42,8388608,6.010075160800793
+self_attn.v_proj,42,8388608,3.0684547763307255
+self_attn.o_proj,42,67108864,3.279936019958201
+mlp.gate_proj,42,234881024,3.4450623262713465
+mlp.down_proj,42,234881024,4.202059625630225
+mlp.up_proj,42,234881024,3.05184351683434
+self_attn.q_proj,43,67108864,4.125479699336057
+self_attn.k_proj,43,8388608,5.369637968082942
+self_attn.v_proj,43,8388608,3.4064244984327043
+self_attn.o_proj,43,67108864,3.6058125061650426
+mlp.gate_proj,43,234881024,3.5682210372555363
+mlp.down_proj,43,234881024,3.2460352594265953
+mlp.up_proj,43,234881024,3.038703860082813
+self_attn.q_proj,44,67108864,4.176082221707317
+self_attn.k_proj,44,8388608,4.363785175387674
+self_attn.v_proj,44,8388608,3.0770005553092883
+self_attn.o_proj,44,67108864,3.297724171807459
+mlp.gate_proj,44,234881024,3.5897218527986863
+mlp.down_proj,44,234881024,3.4194285596203797
+mlp.up_proj,44,234881024,3.047096795584518
+self_attn.q_proj,45,67108864,4.432655005186118
+self_attn.k_proj,45,8388608,5.078206241519742
+self_attn.v_proj,45,8388608,3.0677416622398845
+self_attn.o_proj,45,67108864,3.2831425178044626
+mlp.gate_proj,45,234881024,3.649682851293374
+mlp.down_proj,45,234881024,4.7543099382219705
+mlp.up_proj,45,234881024,3.0674277406832613
+self_attn.q_proj,46,67108864,5.99656653988929
+self_attn.k_proj,46,8388608,31.058143058069565
+self_attn.v_proj,46,8388608,3.0408918772260134
+self_attn.o_proj,46,67108864,3.0415489026581675
+mlp.gate_proj,46,234881024,3.5893660109123195
+mlp.down_proj,46,234881024,3.05018937370663
+mlp.up_proj,46,234881024,3.0364695516089975
+self_attn.q_proj,47,67108864,4.905736579202569
+self_attn.k_proj,47,8388608,6.539744947472659
+self_attn.v_proj,47,8388608,3.2770110697392623
+self_attn.o_proj,47,67108864,3.137199309593585
+mlp.gate_proj,47,234881024,3.4913316993096557
+mlp.down_proj,47,234881024,3.8823738651574917
+mlp.up_proj,47,234881024,3.0240264114017097
+self_attn.q_proj,48,67108864,4.46026281260636
+self_attn.k_proj,48,8388608,5.787822270549902
+self_attn.v_proj,48,8388608,3.286362977188494
+self_attn.o_proj,48,67108864,3.3029833193139453
+mlp.gate_proj,48,234881024,3.4690269877178457
+mlp.down_proj,48,234881024,3.6783860196094733
+mlp.up_proj,48,234881024,3.073570245953952
+self_attn.q_proj,49,67108864,4.524349055252506
+self_attn.k_proj,49,8388608,6.3695274938647435
+self_attn.v_proj,49,8388608,3.070153932347442
+self_attn.o_proj,49,67108864,3.2548583831411495
+mlp.gate_proj,49,234881024,3.656322733803583
+mlp.down_proj,49,234881024,4.717758912511075
+mlp.up_proj,49,234881024,3.0496516247916743
+self_attn.q_proj,50,67108864,5.070338999644826
+self_attn.k_proj,50,8388608,10.19492547000164
+self_attn.v_proj,50,8388608,3.138502590021607
+self_attn.o_proj,50,67108864,3.06424334578906
+mlp.gate_proj,50,234881024,3.646602235980731
+mlp.down_proj,50,234881024,3.047180645244621
+mlp.up_proj,50,234881024,3.0367408822667423
+self_attn.q_proj,51,67108864,4.343256312325087
+self_attn.k_proj,51,8388608,6.295231834932254
+self_attn.v_proj,51,8388608,3.193476408501602
+self_attn.o_proj,51,67108864,3.1558681069681698
+mlp.gate_proj,51,234881024,3.746455252203498
+mlp.down_proj,51,234881024,3.4817187511320946
+mlp.up_proj,51,234881024,3.067487548857431
+self_attn.q_proj,52,67108864,4.498443647568412
+self_attn.k_proj,52,8388608,4.938394036084573
+self_attn.v_proj,52,8388608,3.1660326437522897
+self_attn.o_proj,52,67108864,3.5510439570758194
+mlp.gate_proj,52,234881024,3.747397272723642
+mlp.down_proj,52,234881024,3.9160843911905205
+mlp.up_proj,52,234881024,3.0648548432717035
+self_attn.q_proj,53,67108864,4.393817959076866
+self_attn.k_proj,53,8388608,6.62766077092751
+self_attn.v_proj,53,8388608,3.0539048230361434
+self_attn.o_proj,53,67108864,3.096761749260707
+mlp.gate_proj,53,234881024,3.66567034364309
+mlp.down_proj,53,234881024,4.518880757840602
+mlp.up_proj,53,234881024,3.027552535681796
+self_attn.q_proj,54,67108864,5.400473073114253
+self_attn.k_proj,54,8388608,11.060153767457736
+self_attn.v_proj,54,8388608,3.07829409594642
+self_attn.o_proj,54,67108864,3.4115773929746798
+mlp.gate_proj,54,234881024,3.52124370212324
+mlp.down_proj,54,234881024,3.575758768321153
+mlp.up_proj,54,234881024,3.0325189403721526
+self_attn.q_proj,55,67108864,4.994680191668898
+self_attn.k_proj,55,8388608,7.3837185959020735
+self_attn.v_proj,55,8388608,3.086709558248576
+self_attn.o_proj,55,67108864,90.53231647202891
+mlp.gate_proj,55,234881024,3.4273110260425197
+mlp.down_proj,55,234881024,3.961558537643844
+mlp.up_proj,55,234881024,3.0890577011062823
+self_attn.q_proj,56,67108864,4.324196410109104
+self_attn.k_proj,56,8388608,4.706665437163564
+self_attn.v_proj,56,8388608,3.050993345764472
+self_attn.o_proj,56,67108864,3.0504901058852294
+mlp.gate_proj,56,234881024,3.5275215216868103
+mlp.down_proj,56,234881024,3.955941864143964
+mlp.up_proj,56,234881024,3.019893513845698
+self_attn.q_proj,57,67108864,4.307683360664238
+self_attn.k_proj,57,8388608,5.560412734044873
+self_attn.v_proj,57,8388608,3.0987851304174736
+self_attn.o_proj,57,67108864,3.1697988530826398
+mlp.gate_proj,57,234881024,3.470637731829874
+mlp.down_proj,57,234881024,3.1128679811855187
+mlp.up_proj,57,234881024,3.021086697598766
+self_attn.q_proj,58,67108864,4.7894476880776296
+self_attn.k_proj,58,8388608,9.437933048072432
+self_attn.v_proj,58,8388608,3.2306052509485883
+self_attn.o_proj,58,67108864,3.2156058516704014
+mlp.gate_proj,58,234881024,3.416402664948724
+mlp.down_proj,58,234881024,3.1383673234855394
+mlp.up_proj,58,234881024,3.0185079241503905
+self_attn.q_proj,59,67108864,4.715433296382645
+self_attn.k_proj,59,8388608,9.393806371068832
+self_attn.v_proj,59,8388608,3.095290937847528
+self_attn.o_proj,59,67108864,3.084638447270029
+mlp.gate_proj,59,234881024,3.6690508888540765
+mlp.down_proj,59,234881024,4.874151687295695
+mlp.up_proj,59,234881024,3.0390412649343186
+self_attn.q_proj,60,67108864,4.211894416395964
+self_attn.k_proj,60,8388608,4.65510899862313
+self_attn.v_proj,60,8388608,3.111446340840253
+self_attn.o_proj,60,67108864,3.18285063196728
+mlp.gate_proj,60,234881024,3.6324542808580946
+mlp.down_proj,60,234881024,4.531685923069815
+mlp.up_proj,60,234881024,3.0298242714603276
+self_attn.q_proj,61,67108864,4.6911353199058405
+self_attn.k_proj,61,8388608,8.890719778371801
+self_attn.v_proj,61,8388608,3.391452955548722
+self_attn.o_proj,61,67108864,3.0654822825819186
+mlp.gate_proj,61,234881024,3.602893710174133
+mlp.down_proj,61,234881024,3.2612659050441506
+mlp.up_proj,61,234881024,3.0291412673325193
+self_attn.q_proj,62,67108864,4.549192298517021
+self_attn.k_proj,62,8388608,39.9753454512312
+self_attn.v_proj,62,8388608,3.034675355787572
+self_attn.o_proj,62,67108864,3.6423645746340827
+mlp.gate_proj,62,234881024,3.603287074188739
+mlp.down_proj,62,234881024,3.5266177990751117
+mlp.up_proj,62,234881024,3.0249922044627953
+self_attn.q_proj,63,67108864,4.724229588349163
+self_attn.k_proj,63,8388608,12.077296250277273
+self_attn.v_proj,63,8388608,3.05817433799786
+self_attn.o_proj,63,67108864,3.2556174768015476
+mlp.gate_proj,63,234881024,3.82723181952847
+mlp.down_proj,63,234881024,3.399928234775146
+mlp.up_proj,63,234881024,3.030111089719925
+self_attn.q_proj,64,67108864,4.210742167110019
+self_attn.k_proj,64,8388608,5.29448965985361
+self_attn.v_proj,64,8388608,3.2604123463846566
+self_attn.o_proj,64,67108864,3.304177628388613
+mlp.gate_proj,64,234881024,3.974164664573265
+mlp.down_proj,64,234881024,3.6201428240974267
+mlp.up_proj,64,234881024,3.0313052726711063
+self_attn.q_proj,65,67108864,8.607857134028613
+self_attn.k_proj,65,8388608,34.1169055496606
+self_attn.v_proj,65,8388608,3.143150059496031
+self_attn.o_proj,65,67108864,55.59685397426782
+mlp.gate_proj,65,234881024,4.08091896544732
+mlp.down_proj,65,234881024,3.121620660082446
+mlp.up_proj,65,234881024,3.0258424681636478
+self_attn.q_proj,66,67108864,4.600143362206491
+self_attn.k_proj,66,8388608,8.127798110760418
+self_attn.v_proj,66,8388608,3.1610484219042885
+self_attn.o_proj,66,67108864,4.860538825374716
+mlp.gate_proj,66,234881024,3.821214756225454
+mlp.down_proj,66,234881024,3.5875013346109856
+mlp.up_proj,66,234881024,3.049574210117053
+self_attn.q_proj,67,67108864,3.983329944567504
+self_attn.k_proj,67,8388608,5.0761267719431835
+self_attn.v_proj,67,8388608,3.121141492325547
+self_attn.o_proj,67,67108864,3.25909461616682
+mlp.gate_proj,67,234881024,3.8222401068275937
+mlp.down_proj,67,234881024,3.045947451910267
+mlp.up_proj,67,234881024,3.0565406179976864
+self_attn.q_proj,68,67108864,3.8775783174454426
+self_attn.k_proj,68,8388608,4.796456531308156
+self_attn.v_proj,68,8388608,3.169145217899688
+self_attn.o_proj,68,67108864,3.217944117904559
+mlp.gate_proj,68,234881024,4.064794241629476
+mlp.down_proj,68,234881024,3.7151417873688874
+mlp.up_proj,68,234881024,3.051466449237794
+self_attn.q_proj,69,67108864,4.074999092781845
+self_attn.k_proj,69,8388608,5.127720120462794
+self_attn.v_proj,69,8388608,3.1494028993327174
+self_attn.o_proj,69,67108864,3.301148152077264
+mlp.gate_proj,69,234881024,3.9610090897671877
+mlp.down_proj,69,234881024,3.04624678957998
+mlp.up_proj,69,234881024,3.052386517879511
+self_attn.q_proj,70,67108864,3.91244528428478
+self_attn.k_proj,70,8388608,4.5118943795740885
+self_attn.v_proj,70,8388608,3.1786881385379093
+self_attn.o_proj,70,67108864,6.147876204047078
+mlp.gate_proj,70,234881024,3.8265790067693755
+mlp.down_proj,70,234881024,3.4396640282834654
+mlp.up_proj,70,234881024,3.092815359682455
+self_attn.q_proj,71,67108864,3.829866648667652
+self_attn.k_proj,71,8388608,4.027548962050551
+self_attn.v_proj,71,8388608,3.140610291024372
+self_attn.o_proj,71,67108864,3.5074519103252393
+mlp.gate_proj,71,234881024,3.7711421572107255
+mlp.down_proj,71,234881024,3.3155966944159667
+mlp.up_proj,71,234881024,3.061763057397585
+self_attn.q_proj,72,67108864,3.719855955091989
+self_attn.k_proj,72,8388608,4.06502380658356
+self_attn.v_proj,72,8388608,3.1855552482569744
+self_attn.o_proj,72,67108864,19.952586074634674
+mlp.gate_proj,72,234881024,3.942518592095566
+mlp.down_proj,72,234881024,3.7487232277977727
+mlp.up_proj,72,234881024,3.047633679976642
+self_attn.q_proj,73,67108864,3.938507796120421
+self_attn.k_proj,73,8388608,4.179817568349999
+self_attn.v_proj,73,8388608,3.3628651133660714
+self_attn.o_proj,73,67108864,8.17662953296392
+mlp.gate_proj,73,234881024,3.9621509706727105
+mlp.down_proj,73,234881024,3.755906408039283
+mlp.up_proj,73,234881024,3.228858993772048
+self_attn.q_proj,74,67108864,3.9921439830289445
+self_attn.k_proj,74,8388608,4.141923696641275
+self_attn.v_proj,74,8388608,3.590678549435015
+self_attn.o_proj,74,67108864,8.769365482236061
+mlp.gate_proj,74,234881024,3.818930983219163
+mlp.down_proj,74,234881024,3.697890443418807
+mlp.up_proj,74,234881024,3.0629015923001126
+self_attn.q_proj,75,67108864,3.986977966197573
+self_attn.k_proj,75,8388608,4.123669340903573
+self_attn.v_proj,75,8388608,3.2146650360681344
+self_attn.o_proj,75,67108864,3.810305394636643
+mlp.gate_proj,75,234881024,3.964281372296313
+mlp.down_proj,75,234881024,4.247000472256862
+mlp.up_proj,75,234881024,3.08758109753974
+self_attn.q_proj,76,67108864,4.050899599179429
+self_attn.k_proj,76,8388608,4.0306338419286964
+self_attn.v_proj,76,8388608,3.310504680339803
+self_attn.o_proj,76,67108864,5.739300027579159
+mlp.gate_proj,76,234881024,4.746034199738044
+mlp.down_proj,76,234881024,3.5501438271527164
+mlp.up_proj,76,234881024,3.2709687186476244
+self_attn.q_proj,77,67108864,3.9874622122309176
+self_attn.k_proj,77,8388608,4.031801527311769
+self_attn.v_proj,77,8388608,3.457236004057245
+self_attn.o_proj,77,67108864,3.571794947537351
+mlp.gate_proj,77,234881024,3.8937679813950767
+mlp.down_proj,77,234881024,3.459139629811048
+mlp.up_proj,77,234881024,3.2758882964046974
+self_attn.q_proj,78,67108864,3.8334683779046177
+self_attn.k_proj,78,8388608,4.014139527279989
+self_attn.v_proj,78,8388608,3.397777071587226
+self_attn.o_proj,78,67108864,3.4207603555992474
+mlp.gate_proj,78,234881024,4.167646287901939
+mlp.down_proj,78,234881024,3.8754421655805174
+mlp.up_proj,78,234881024,3.617700435066121
+self_attn.q_proj,79,67108864,4.275777252444301
+self_attn.k_proj,79,8388608,5.0387224964198225
+self_attn.v_proj,79,8388608,3.669491235550119
+self_attn.o_proj,79,67108864,6.140289634775126
+mlp.gate_proj,79,234881024,3.7830182224968967
+mlp.down_proj,79,234881024,4.411826982786458
+mlp.up_proj,79,234881024,3.551016961161939

lm-quant-toolkit/tmp/kurtosis-dump/Mistral-7B-Instruct-v0.3/kurtosis-models.csv

@@ -0,0 +1,225 @@
+module,layer,param_count,kurtosis
+self_attn.q_proj,0,16777216,60.17712647624026
+self_attn.k_proj,0,4194304,28.653533803838418
+self_attn.v_proj,0,4194304,8.059729712213453
+self_attn.o_proj,0,16777216,7.374105781769602
+mlp.gate_proj,0,58720256,4.0323653851586405
+mlp.down_proj,0,58720256,3.707233609392377
+mlp.up_proj,0,58720256,3.219435372098439
+self_attn.q_proj,1,16777216,9.882409318565285
+self_attn.k_proj,1,4194304,14.004439993846377
+self_attn.v_proj,1,4194304,4.5242809141333105
+self_attn.o_proj,1,16777216,5.883120015358403
+mlp.gate_proj,1,58720256,6.47026818737708
+mlp.down_proj,1,58720256,6.182068263853448
+mlp.up_proj,1,58720256,3.3626831543529336
+self_attn.q_proj,2,16777216,4.276679902027134
+self_attn.k_proj,2,4194304,5.368743892037284
+self_attn.v_proj,2,4194304,3.5271998190041502
+self_attn.o_proj,2,16777216,5.694789583263856
+mlp.gate_proj,2,58720256,3.3180077157362007
+mlp.down_proj,2,58720256,3.4346853003610067
+mlp.up_proj,2,58720256,3.1333503454620053
+self_attn.q_proj,3,16777216,4.03470825398327
+self_attn.k_proj,3,4194304,4.6939005125164694
+self_attn.v_proj,3,4194304,3.2383937793350674
+self_attn.o_proj,3,16777216,4.474048807385678
+mlp.gate_proj,3,58720256,3.2683794510757465
+mlp.down_proj,3,58720256,3.4870122305345976
+mlp.up_proj,3,58720256,3.1619121034939885
+self_attn.q_proj,4,16777216,4.513734556040291
+self_attn.k_proj,4,4194304,5.218521983663884
+self_attn.v_proj,4,4194304,3.2569268412693986
+self_attn.o_proj,4,16777216,4.969638007437325
+mlp.gate_proj,4,58720256,3.4401548297972213
+mlp.down_proj,4,58720256,3.748635340149656
+mlp.up_proj,4,58720256,3.2669290715700083
+self_attn.q_proj,5,16777216,4.198458974702791
+self_attn.k_proj,5,4194304,4.4313382483658215
+self_attn.v_proj,5,4194304,3.1918263456913927
+self_attn.o_proj,5,16777216,3.806947751504118
+mlp.gate_proj,5,58720256,3.4465898185171104
+mlp.down_proj,5,58720256,3.6304150103911246
+mlp.up_proj,5,58720256,3.211178099274968
+self_attn.q_proj,6,16777216,4.251021111217926
+self_attn.k_proj,6,4194304,4.816587815629525
+self_attn.v_proj,6,4194304,3.243378018226967
+self_attn.o_proj,6,16777216,3.848549854772985
+mlp.gate_proj,6,58720256,3.4439861956064015
+mlp.down_proj,6,58720256,3.678341214148525
+mlp.up_proj,6,58720256,3.27799880503302
+self_attn.q_proj,7,16777216,4.535839111569815
+self_attn.k_proj,7,4194304,5.411159083240771
+self_attn.v_proj,7,4194304,3.3226809081885245
+self_attn.o_proj,7,16777216,4.94906256800735
+mlp.gate_proj,7,58720256,3.447043851785688
+mlp.down_proj,7,58720256,3.4510594762783047
+mlp.up_proj,7,58720256,3.3080114350162426
+self_attn.q_proj,8,16777216,4.079127148250755
+self_attn.k_proj,8,4194304,4.759867303124605
+self_attn.v_proj,8,4194304,3.835431638020718
+self_attn.o_proj,8,16777216,3.6581144114227935
+mlp.gate_proj,8,58720256,3.4683366960354944
+mlp.down_proj,8,58720256,3.911754138699287
+mlp.up_proj,8,58720256,3.2793075409199934
+self_attn.q_proj,9,16777216,4.900776106591824
+self_attn.k_proj,9,4194304,5.073599191983385
+self_attn.v_proj,9,4194304,3.3616271718785518
+self_attn.o_proj,9,16777216,3.5757431668604696
+mlp.gate_proj,9,58720256,3.5220311615172815
+mlp.down_proj,9,58720256,3.7145987045082647
+mlp.up_proj,9,58720256,3.289410238555988
+self_attn.q_proj,10,16777216,4.169503684933775
+self_attn.k_proj,10,4194304,4.541564090996692
+self_attn.v_proj,10,4194304,3.547738011233978
+self_attn.o_proj,10,16777216,3.6995600492122764
+mlp.gate_proj,10,58720256,3.6246815841079987
+mlp.down_proj,10,58720256,4.0127317494008325
+mlp.up_proj,10,58720256,3.3159073681090967
+self_attn.q_proj,11,16777216,4.771900939822997
+self_attn.k_proj,11,4194304,5.415683478268487
+self_attn.v_proj,11,4194304,3.627483928427408
+self_attn.o_proj,11,16777216,4.175675158410296
+mlp.gate_proj,11,58720256,3.9691322930659307
+mlp.down_proj,11,58720256,3.93018466676117
+mlp.up_proj,11,58720256,3.431293501946332
+self_attn.q_proj,12,16777216,5.812020104416476
+self_attn.k_proj,12,4194304,5.996968360094484
+self_attn.v_proj,12,4194304,3.475396999006639
+self_attn.o_proj,12,16777216,3.953616260693548
+mlp.gate_proj,12,58720256,3.889852553385862
+mlp.down_proj,12,58720256,3.981337453439595
+mlp.up_proj,12,58720256,3.3095079463372357
+self_attn.q_proj,13,16777216,3.970451098063313
+self_attn.k_proj,13,4194304,4.525413577625811
+self_attn.v_proj,13,4194304,3.3434455055859127
+self_attn.o_proj,13,16777216,3.729239537538608
+mlp.gate_proj,13,58720256,3.804264132874413
+mlp.down_proj,13,58720256,3.757553704003033
+mlp.up_proj,13,58720256,3.353261828432498
+self_attn.q_proj,14,16777216,4.725925950968306
+self_attn.k_proj,14,4194304,5.2898149911833565
+self_attn.v_proj,14,4194304,3.8068967261016975
+self_attn.o_proj,14,16777216,3.9087033981669164
+mlp.gate_proj,14,58720256,4.380295790290444
+mlp.down_proj,14,58720256,5.33515392318582
+mlp.up_proj,14,58720256,3.3916770467263624
+self_attn.q_proj,15,16777216,5.4236265828868655
+self_attn.k_proj,15,4194304,6.318360165493945
+self_attn.v_proj,15,4194304,3.45949692634637
+self_attn.o_proj,15,16777216,3.5106811328238794
+mlp.gate_proj,15,58720256,4.46589628190928
+mlp.down_proj,15,58720256,3.7827117206447993
+mlp.up_proj,15,58720256,3.2658295763662157
+self_attn.q_proj,16,16777216,4.190674288983912
+self_attn.k_proj,16,4194304,4.850896119465776
+self_attn.v_proj,16,4194304,3.350058695126207
+self_attn.o_proj,16,16777216,5.037098198802513
+mlp.gate_proj,16,58720256,4.629009051235029
+mlp.down_proj,16,58720256,3.772777737127814
+mlp.up_proj,16,58720256,3.288646760222826
+self_attn.q_proj,17,16777216,4.127390327215259
+self_attn.k_proj,17,4194304,4.609981303767143
+self_attn.v_proj,17,4194304,3.5698898452068053
+self_attn.o_proj,17,16777216,4.132327502527257
+mlp.gate_proj,17,58720256,4.3842945428817615
+mlp.down_proj,17,58720256,3.710237566987906
+mlp.up_proj,17,58720256,3.418545829183981
+self_attn.q_proj,18,16777216,5.014363779733479
+self_attn.k_proj,18,4194304,5.135466741338119
+self_attn.v_proj,18,4194304,3.5523022015748507
+self_attn.o_proj,18,16777216,4.497196521770899
+mlp.gate_proj,18,58720256,4.080996913699577
+mlp.down_proj,18,58720256,3.80944331912374
+mlp.up_proj,18,58720256,3.295935054119426
+self_attn.q_proj,19,16777216,3.9243888134385934
+self_attn.k_proj,19,4194304,5.228034930265339
+self_attn.v_proj,19,4194304,3.395882522382695
+self_attn.o_proj,19,16777216,4.7825991341801934
+mlp.gate_proj,19,58720256,3.8641538760194427
+mlp.down_proj,19,58720256,4.388008896722242
+mlp.up_proj,19,58720256,3.2739660011781786
+self_attn.q_proj,20,16777216,4.202794177115714
+self_attn.k_proj,20,4194304,4.781963571190247
+self_attn.v_proj,20,4194304,3.4064181401644564
+self_attn.o_proj,20,16777216,3.892511953366737
+mlp.gate_proj,20,58720256,3.589645680695326
+mlp.down_proj,20,58720256,3.610683254828349
+mlp.up_proj,20,58720256,3.188388027731601
+self_attn.q_proj,21,16777216,3.8414908458362187
+self_attn.k_proj,21,4194304,4.623803245725223
+self_attn.v_proj,21,4194304,3.3354089818914803
+self_attn.o_proj,21,16777216,3.697911689259188
+mlp.gate_proj,21,58720256,3.435389048772403
+mlp.down_proj,21,58720256,3.7469601601057922
+mlp.up_proj,21,58720256,3.136036153308386
+self_attn.q_proj,22,16777216,3.803641990506112
+self_attn.k_proj,22,4194304,4.6249222005318735
+self_attn.v_proj,22,4194304,3.2267355962758204
+self_attn.o_proj,22,16777216,3.6973209171674437
+mlp.gate_proj,22,58720256,3.355598476760606
+mlp.down_proj,22,58720256,3.4441818203540002
+mlp.up_proj,22,58720256,3.105396859848281
+self_attn.q_proj,23,16777216,3.675871811841553
+self_attn.k_proj,23,4194304,4.384261778221278
+self_attn.v_proj,23,4194304,3.2893450334994423
+self_attn.o_proj,23,16777216,3.919391587027105
+mlp.gate_proj,23,58720256,3.327770818728332
+mlp.down_proj,23,58720256,3.418971574528143
+mlp.up_proj,23,58720256,3.1032117047512218
+self_attn.q_proj,24,16777216,3.780481324903357
+self_attn.k_proj,24,4194304,4.428028106253863
+self_attn.v_proj,24,4194304,3.194827020122189
+self_attn.o_proj,24,16777216,3.3858916444710165
+mlp.gate_proj,24,58720256,3.276686717358284
+mlp.down_proj,24,58720256,3.634553753362112
+mlp.up_proj,24,58720256,3.171395002503821
+self_attn.q_proj,25,16777216,3.7762605172519748
+self_attn.k_proj,25,4194304,4.422413178568468
+self_attn.v_proj,25,4194304,3.2829051239415348
+self_attn.o_proj,25,16777216,3.6938734655960284
+mlp.gate_proj,25,58720256,3.326865940929996
+mlp.down_proj,25,58720256,3.257935991706683
+mlp.up_proj,25,58720256,3.099798073883842
+self_attn.q_proj,26,16777216,3.7865802348312863
+self_attn.k_proj,26,4194304,4.650732505606878
+self_attn.v_proj,26,4194304,3.3119866865698815
+self_attn.o_proj,26,16777216,3.2181015371129145
+mlp.gate_proj,26,58720256,3.2981586538497916
+mlp.down_proj,26,58720256,3.555098249599667
+mlp.up_proj,26,58720256,3.1281239438504946
+self_attn.q_proj,27,16777216,3.686869796177915
+self_attn.k_proj,27,4194304,4.285555376185663
+self_attn.v_proj,27,4194304,3.2078787280798933
+self_attn.o_proj,27,16777216,4.131692891942931
+mlp.gate_proj,27,58720256,3.4092379039074054
+mlp.down_proj,27,58720256,3.3630155578332794
+mlp.up_proj,27,58720256,3.1344947969590486
+self_attn.q_proj,28,16777216,4.017032677725968
+self_attn.k_proj,28,4194304,5.370324622412694
+self_attn.v_proj,28,4194304,3.827258127186466
+self_attn.o_proj,28,16777216,3.364390436831042
+mlp.gate_proj,28,58720256,3.3968383459630993
+mlp.down_proj,28,58720256,3.6362066892761105
+mlp.up_proj,28,58720256,3.1597216570106452
+self_attn.q_proj,29,16777216,4.228814863585314
+self_attn.k_proj,29,4194304,5.4703920704327516
+self_attn.v_proj,29,4194304,3.8404078722991195
+self_attn.o_proj,29,16777216,3.742304360279831
+mlp.gate_proj,29,58720256,3.4101824243225765
+mlp.down_proj,29,58720256,3.363471975124289
+mlp.up_proj,29,58720256,3.612222910754434
+self_attn.q_proj,30,16777216,4.243461395208247
+self_attn.k_proj,30,4194304,5.410600557378085
+self_attn.v_proj,30,4194304,4.158051845594555
+self_attn.o_proj,30,16777216,4.394985673004208
+mlp.gate_proj,30,58720256,4.144386546993606
+mlp.down_proj,30,58720256,5.818492316592101
+mlp.up_proj,30,58720256,3.5630530199057135
+self_attn.q_proj,31,16777216,4.140692966383174
+self_attn.k_proj,31,4194304,5.782846387395266
+self_attn.v_proj,31,4194304,4.100183326805308
+self_attn.o_proj,31,16777216,62.117407530546046
+mlp.gate_proj,31,58720256,3.5150252878856003
+mlp.down_proj,31,58720256,7.974207532905209
+mlp.up_proj,31,58720256,3.355771026158226

lm-quant-toolkit/tmp/kurtosis-dump/kurtosis-Llama-2-7b-hf.csv

@@ -0,0 +1,225 @@
+module,layer,param_count,kurtosis
+self_attn.q_proj,0,16777216,70.89063407817052
+self_attn.k_proj,0,16777216,38.24344781783628
+self_attn.v_proj,0,16777216,6.17988231863005
+self_attn.o_proj,0,16777216,39.606632106657415
+mlp.gate_proj,0,45088768,4.269190067451742
+mlp.down_proj,0,45088768,3.9410243132303155
+mlp.up_proj,0,45088768,3.1257809370322978
+self_attn.q_proj,1,16777216,9.506592860422103
+self_attn.k_proj,1,16777216,9.935371374026081
+self_attn.v_proj,1,16777216,4.8384166345295005
+self_attn.o_proj,1,16777216,26.69787713965271
+mlp.gate_proj,1,45088768,6.280002848563622
+mlp.down_proj,1,45088768,7.5802935382124135
+mlp.up_proj,1,45088768,3.4447561314119977
+self_attn.q_proj,2,16777216,5.845498006156931
+self_attn.k_proj,2,16777216,5.422568329046552
+self_attn.v_proj,2,16777216,3.1704948733015765
+self_attn.o_proj,2,16777216,7.172113174858918
+mlp.gate_proj,2,45088768,3.252795019767423
+mlp.down_proj,2,45088768,3.5000946352030273
+mlp.up_proj,2,45088768,3.0643926831566968
+self_attn.q_proj,3,16777216,5.747449112728132
+self_attn.k_proj,3,16777216,4.552550541532545
+self_attn.v_proj,3,16777216,3.2101376661454877
+self_attn.o_proj,3,16777216,6.738655132097817
+mlp.gate_proj,3,45088768,3.2690103679987637
+mlp.down_proj,3,45088768,3.522186223965613
+mlp.up_proj,3,45088768,3.11530955415524
+self_attn.q_proj,4,16777216,5.076595574419752
+self_attn.k_proj,4,16777216,4.330934358734191
+self_attn.v_proj,4,16777216,3.12290612284217
+self_attn.o_proj,4,16777216,5.993817704977201
+mlp.gate_proj,4,45088768,3.3662009159295945
+mlp.down_proj,4,45088768,3.9507138760715965
+mlp.up_proj,4,45088768,3.103154995797977
+self_attn.q_proj,5,16777216,4.239130739572246
+self_attn.k_proj,5,16777216,4.1880352601972435
+self_attn.v_proj,5,16777216,3.166712534307685
+self_attn.o_proj,5,16777216,5.158751141507659
+mlp.gate_proj,5,45088768,3.3132728389187305
+mlp.down_proj,5,45088768,3.5031316078707673
+mlp.up_proj,5,45088768,3.1074866507472088
+self_attn.q_proj,6,16777216,4.859155894110683
+self_attn.k_proj,6,16777216,4.478034883058803
+self_attn.v_proj,6,16777216,3.301832949774581
+self_attn.o_proj,6,16777216,7.088338485259551
+mlp.gate_proj,6,45088768,3.3562287806004885
+mlp.down_proj,6,45088768,3.6766909225129805
+mlp.up_proj,6,45088768,3.1103515391488963
+self_attn.q_proj,7,16777216,4.7117156949789765
+self_attn.k_proj,7,16777216,4.368184768466018
+self_attn.v_proj,7,16777216,3.35458205981544
+self_attn.o_proj,7,16777216,5.347610561393708
+mlp.gate_proj,7,45088768,3.3854765386900025
+mlp.down_proj,7,45088768,3.6398775576866766
+mlp.up_proj,7,45088768,3.0859186463660886
+self_attn.q_proj,8,16777216,4.825607138911687
+self_attn.k_proj,8,16777216,4.302162251441214
+self_attn.v_proj,8,16777216,3.224550114501631
+self_attn.o_proj,8,16777216,6.007940412405506
+mlp.gate_proj,8,45088768,3.3917166195373265
+mlp.down_proj,8,45088768,3.6487149038625772
+mlp.up_proj,8,45088768,3.143100821324133
+self_attn.q_proj,9,16777216,4.141120940492358
+self_attn.k_proj,9,16777216,4.097917798059499
+self_attn.v_proj,9,16777216,3.254566641438937
+self_attn.o_proj,9,16777216,4.903297282365117
+mlp.gate_proj,9,45088768,3.4635676641917645
+mlp.down_proj,9,45088768,3.6072720146683652
+mlp.up_proj,9,45088768,3.1384390669911246
+self_attn.q_proj,10,16777216,4.133870396866107
+self_attn.k_proj,10,16777216,4.262353252802395
+self_attn.v_proj,10,16777216,3.221364068191206
+self_attn.o_proj,10,16777216,4.063452398003821
+mlp.gate_proj,10,45088768,3.659082896762817
+mlp.down_proj,10,45088768,3.6335260007037236
+mlp.up_proj,10,45088768,3.094164330042007
+self_attn.q_proj,11,16777216,5.116813149153919
+self_attn.k_proj,11,16777216,4.630848587525195
+self_attn.v_proj,11,16777216,3.3207023327508245
+self_attn.o_proj,11,16777216,6.1512512579371075
+mlp.gate_proj,11,45088768,3.5786700386051513
+mlp.down_proj,11,45088768,3.5530041249449695
+mlp.up_proj,11,45088768,3.1151256150814888
+self_attn.q_proj,12,16777216,4.227513682256377
+self_attn.k_proj,12,16777216,4.210345039001077
+self_attn.v_proj,12,16777216,3.2917196179921357
+self_attn.o_proj,12,16777216,4.434055632045808
+mlp.gate_proj,12,45088768,3.5231071621054237
+mlp.down_proj,12,45088768,3.291699935375262
+mlp.up_proj,12,45088768,3.0757519740562933
+self_attn.q_proj,13,16777216,4.246715043246877
+self_attn.k_proj,13,16777216,4.160297130333501
+self_attn.v_proj,13,16777216,3.240945054885195
+self_attn.o_proj,13,16777216,4.47524457880676
+mlp.gate_proj,13,45088768,3.492987813719813
+mlp.down_proj,13,45088768,3.647359138499682
+mlp.up_proj,13,45088768,3.0997099821864476
+self_attn.q_proj,14,16777216,4.172159714823189
+self_attn.k_proj,14,16777216,4.20591811396014
+self_attn.v_proj,14,16777216,3.2207892668840437
+self_attn.o_proj,14,16777216,4.3913518343431885
+mlp.gate_proj,14,45088768,3.5157555990507983
+mlp.down_proj,14,45088768,3.240919165543396
+mlp.up_proj,14,45088768,3.088386720975056
+self_attn.q_proj,15,16777216,4.180196395053585
+self_attn.k_proj,15,16777216,4.031791099881734
+self_attn.v_proj,15,16777216,3.1698077407619936
+self_attn.o_proj,15,16777216,3.962050750665648
+mlp.gate_proj,15,45088768,3.495786797250684
+mlp.down_proj,15,45088768,3.5596160792985456
+mlp.up_proj,15,45088768,3.1036080580162118
+self_attn.q_proj,16,16777216,4.9387756507742715
+self_attn.k_proj,16,16777216,4.181085379327221
+self_attn.v_proj,16,16777216,3.1732431739777383
+self_attn.o_proj,16,16777216,3.9618297565651868
+mlp.gate_proj,16,45088768,3.4721134762605304
+mlp.down_proj,16,45088768,3.307626413448843
+mlp.up_proj,16,45088768,3.0796057455877506
+self_attn.q_proj,17,16777216,4.41058356494222
+self_attn.k_proj,17,16777216,4.011172975365525
+self_attn.v_proj,17,16777216,3.1317380318450874
+self_attn.o_proj,17,16777216,4.40230065747996
+mlp.gate_proj,17,45088768,3.379672209074253
+mlp.down_proj,17,45088768,3.4676421560877833
+mlp.up_proj,17,45088768,3.0539835798489796
+self_attn.q_proj,18,16777216,4.331741585407725
+self_attn.k_proj,18,16777216,3.852686131364871
+self_attn.v_proj,18,16777216,3.1396861815281687
+self_attn.o_proj,18,16777216,4.205083278063674
+mlp.gate_proj,18,45088768,3.363501779614211
+mlp.down_proj,18,45088768,3.4708772288943393
+mlp.up_proj,18,45088768,3.0475381520216764
+self_attn.q_proj,19,16777216,4.501974087132231
+self_attn.k_proj,19,16777216,3.9534432721176604
+self_attn.v_proj,19,16777216,3.1479206835009523
+self_attn.o_proj,19,16777216,4.21699893959325
+mlp.gate_proj,19,45088768,3.3059414349857956
+mlp.down_proj,19,45088768,3.3501680690597713
+mlp.up_proj,19,45088768,3.0769263170951198
+self_attn.q_proj,20,16777216,5.089448547787887
+self_attn.k_proj,20,16777216,4.017856732818786
+self_attn.v_proj,20,16777216,3.077329271332318
+self_attn.o_proj,20,16777216,6.20978041938933
+mlp.gate_proj,20,45088768,3.24058842202935
+mlp.down_proj,20,45088768,3.5542677087714925
+mlp.up_proj,20,45088768,3.037644738477705
+self_attn.q_proj,21,16777216,5.023500600377066
+self_attn.k_proj,21,16777216,4.043552781047404
+self_attn.v_proj,21,16777216,3.0914145141503155
+self_attn.o_proj,21,16777216,4.176771420961363
+mlp.gate_proj,21,45088768,3.2252613589641217
+mlp.down_proj,21,45088768,3.2126358273768942
+mlp.up_proj,21,45088768,3.0317359434824303
+self_attn.q_proj,22,16777216,4.650421015682584
+self_attn.k_proj,22,16777216,3.8562861685873253
+self_attn.v_proj,22,16777216,3.1028997163538112
+self_attn.o_proj,22,16777216,12.72973753381282
+mlp.gate_proj,22,45088768,3.1997543042302996
+mlp.down_proj,22,45088768,3.045221148790187
+mlp.up_proj,22,45088768,3.026070663498454
+self_attn.q_proj,23,16777216,4.622973048321524
+self_attn.k_proj,23,16777216,3.751007505803854
+self_attn.v_proj,23,16777216,3.1018698150322948
+self_attn.o_proj,23,16777216,3.976658721156604
+mlp.gate_proj,23,45088768,3.1522250758763932
+mlp.down_proj,23,45088768,3.661347930223575
+mlp.up_proj,23,45088768,3.029466900689184
+self_attn.q_proj,24,16777216,4.810796636988315
+self_attn.k_proj,24,16777216,4.161086490091737
+self_attn.v_proj,24,16777216,3.08752802550685
+self_attn.o_proj,24,16777216,6.397808422117449
+mlp.gate_proj,24,45088768,3.159957907684576
+mlp.down_proj,24,45088768,3.1393332037439903
+mlp.up_proj,24,45088768,3.0209667213622073
+self_attn.q_proj,25,16777216,4.259720660607395
+self_attn.k_proj,25,16777216,3.828115065648799
+self_attn.v_proj,25,16777216,3.0825314450000945
+self_attn.o_proj,25,16777216,3.4139961077564593
+mlp.gate_proj,25,45088768,3.2140774706633204
+mlp.down_proj,25,45088768,3.2576513111327277
+mlp.up_proj,25,45088768,3.02615172988415
+self_attn.q_proj,26,16777216,4.746416094050959
+self_attn.k_proj,26,16777216,3.950877539540366
+self_attn.v_proj,26,16777216,3.174601908030728
+self_attn.o_proj,26,16777216,5.404761630435047
+mlp.gate_proj,26,45088768,3.258833850594769
+mlp.down_proj,26,45088768,3.0934423947000096
+mlp.up_proj,26,45088768,3.0306968402468804
+self_attn.q_proj,27,16777216,4.456513964473084
+self_attn.k_proj,27,16777216,3.5343563483836373
+self_attn.v_proj,27,16777216,3.0942072739706443
+self_attn.o_proj,27,16777216,3.855701436089548
+mlp.gate_proj,27,45088768,3.3571662046824033
+mlp.down_proj,27,45088768,3.149947949147703
+mlp.up_proj,27,45088768,3.032937516667663
+self_attn.q_proj,28,16777216,4.9206986159336
+self_attn.k_proj,28,16777216,3.7013362450844207
+self_attn.v_proj,28,16777216,3.15623756263924
+self_attn.o_proj,28,16777216,3.815716480743954
+mlp.gate_proj,28,45088768,3.412267812021293
+mlp.down_proj,28,45088768,3.3256110936488867
+mlp.up_proj,28,45088768,3.045774527398766
+self_attn.q_proj,29,16777216,5.75206972772167
+self_attn.k_proj,29,16777216,3.9551189681736383
+self_attn.v_proj,29,16777216,3.113633082966048
+self_attn.o_proj,29,16777216,4.135413539616198
+mlp.gate_proj,29,45088768,3.605206129991087
+mlp.down_proj,29,45088768,3.547261426279374
+mlp.up_proj,29,45088768,3.3965450154438708
+self_attn.q_proj,30,16777216,5.794882486073459
+self_attn.k_proj,30,16777216,3.65192595962886
+self_attn.v_proj,30,16777216,3.1510945883178296
+self_attn.o_proj,30,16777216,3.989114407498837
+mlp.gate_proj,30,45088768,8.489964918198787
+mlp.down_proj,30,45088768,5.107840285620544
+mlp.up_proj,30,45088768,9.524281481179829
+self_attn.q_proj,31,16777216,4.2604406701808495
+self_attn.k_proj,31,16777216,4.044775676830005
+self_attn.v_proj,31,16777216,3.423163795614051
+self_attn.o_proj,31,16777216,38.917877019289406
+mlp.gate_proj,31,45088768,3.5969066369466294
+mlp.down_proj,31,45088768,8.550558646527769
+mlp.up_proj,31,45088768,3.714970630664952

lm-quant-toolkit/utils/combine-wdist-llama.R

@@ -0,0 +1,45 @@
+library(dplyr)
+library(readr)
+
+strip_name <- function(name) {
+  start <- nchar("wdist-") + 1
+  stop <- nchar(name) - 4
+  return(substr(name,
+ start,
+ stop))
+}
+
+wdist_data_dir <- "../data-vis/data/wdist/tmp"
+wdist_dir <- normalizePath(wdist_data_dir)
+wdist_fps <- dir(
+  path = wdist_dir,
+
+  pattern = paste0("wdist-.*",
+ "\\.csv$"),
+
+  full.names = TRUE
+)
+names(wdist_fps) <- sapply((basename(wdist_fps)),
+ strip_name)
+df_wdist <- plyr::ldply(
+  wdist_fps,
+  read.csv,
+  stringsAsFactors = FALSE,
+  .id = "model"
+)
+
+k_cols <- c(
+  "model",
+  "module",
+  "param_count",
+  "layer",
+  "percentile_0",
+  "percentile_99",
+  "percentile_999",
+  "percentile_9999",
+  "percentile_100",
+  "kurtosis"
+)
+df_wdist <- df_wdist |>
+  select(all_of(k_cols))
+write_csv(df_wdist, "llama-wdist.csv")

lm-quant-toolkit/utils/gen-ds-bos/.gitignore

@@ -0,0 +1 @@
+settings.ini

lm-quant-toolkit/utils/gen-ds-bos/llama2-7b.sh

@@ -0,0 +1,5 @@
+./generate.py \
+  --model="meta-llama/Llama-2-7b-chat-hf" \
+  --variant="vLLM" \
+  --topic-file=topics-l1.txt \
+  --trace

lm-quant-toolkit/utils/gen-ds-bos/topics-l1.txt

@@ -0,0 +1,25 @@
+"Math":15
+"Engineering":15
+"Computer Science":15
+"Chemstry":15
+"Physics":15
+"Geography":10
+"History":7
+"English":9
+"Psychology":8
+"Politics":8
+"Law":10
+"Art":10
+"Music":10
+"Photography":5
+"Medical":10
+"Biology":10
+"Business":10
+"Agiculture":10
+"Architecture":10
+"Journalism":10
+"Oceanography":5
+"Social Science":5
+"Ecology":5
+"Astronomy":5
+"Life Sciences":5

models/Llama-3.1-8B-quantization-baselines/generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": 128001,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.57.3"
+}

models/Qwen/Qwen2.5-14B/config.json

@@ -0,0 +1,27 @@
+{
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "eos_token_id": 151643,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 13824,
+  "max_position_embeddings": 131072,
+  "max_window_layers": 48,
+  "model_type": "qwen2",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 48,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 1000000.0,
+  "sliding_window": 131072,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.43.1",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 152064
+}

models/Qwen3-8B/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2024 Alibaba Cloud
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

models/Qwen3-8B/generation_config.json

@@ -0,0 +1,13 @@
+{
+    "bos_token_id": 151643,
+    "do_sample": true,
+    "eos_token_id": [
+        151645,
+        151643
+    ],
+    "pad_token_id": 151643,
+    "temperature": 0.6,
+    "top_k": 20,
+    "top_p": 0.95,
+    "transformers_version": "4.51.0"
+}

models/Qwen3-8B/model.safetensors.index.json

@@ -0,0 +1,406 @@
+{
+  "metadata": {
+    "total_size": 16381470720
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00005-of-00005.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.10.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.10.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.17.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.17.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.18.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.18.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.2.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.2.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.20.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.20.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.27.self_attn.k_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.27.self_attn.q_norm.weight": "model-00003-of-00005.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00003-of-00005.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.28.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.28.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.3.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.3.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.30.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.30.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.32.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.33.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.34.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.mlp.down_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.mlp.gate_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.mlp.up_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.self_attn.k_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.self_attn.k_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.self_attn.q_norm.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.self_attn.q_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.35.self_attn.v_proj.weight": "model-00004-of-00005.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.4.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.4.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.self_attn.k_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.self_attn.q_norm.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.7.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.7.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00005.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.8.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.8.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.self_attn.k_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.self_attn.q_norm.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00005.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00005.safetensors",
+    "model.norm.weight": "model-00004-of-00005.safetensors"
+  }
+}

models/Qwen3-8B/tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n    {%- set index = (messages|length - 1) - loop.index0 %}\n    {%- if ns.multi_step_tool and message.role == \"user\" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n        {%- set ns.multi_step_tool = false %}\n        {%- set ns.last_query_index = index %}\n    {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n    {%- if message.content is string %}\n        {%- set content = message.content %}\n    {%- else %}\n        {%- set content = '' %}\n    {%- endif %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is string %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in content %}\n                {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n                {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- if loop.index0 > ns.last_query_index %}\n            {%- if loop.last or (not loop.last and reasoning_content) %}\n                {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n            {%- else %}\n                {{- '<|im_start|>' + message.role + '\\n' + content }}\n            {%- endif %}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- endif %}\n{%- endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

models/meta-llama/Llama-2-7b-hf/.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Responsible-Use-Guide.pdf filter=lfs diff=lfs merge=lfs -text

models/meta-llama/Llama-2-7b-hf/LICENSE.txt

@@ -0,0 +1,126 @@
+LLAMA 2 COMMUNITY LICENSE AGREEMENT	
+Llama 2 Version Release Date: July 18, 2023
+
+"Agreement" means the terms and conditions for use, reproduction, distribution and 
+modification of the Llama Materials set forth herein.
+
+"Documentation" means the specifications, manuals and documentation 
+accompanying Llama 2 distributed by Meta at ai.meta.com/resources/models-and-
+libraries/llama-downloads/.
+
+"Licensee" or "you" means you, or your employer or any other person or entity (if 
+you are entering into this Agreement on such person or entity's behalf), of the age 
+required under applicable laws, rules or regulations to provide legal consent and that 
+has legal authority to bind your employer or such other person or entity if you are 
+entering in this Agreement on their behalf.
+
+"Llama 2" means the foundational large language models and software and 
+algorithms, including machine-learning model code, trained model weights, 
+inference-enabling code, training-enabling code, fine-tuning enabling code and other 
+elements of the foregoing distributed by Meta at ai.meta.com/resources/models-and-
+libraries/llama-downloads/.
+
+"Llama Materials" means, collectively, Meta's proprietary Llama 2 and 
+Documentation (and any portion thereof) made available under this Agreement.
+
+"Meta" or "we" means Meta Platforms Ireland Limited (if you are located in or, if you 
+are an entity, your principal place of business is in the EEA or Switzerland) and Meta 
+Platforms, Inc. (if you are located outside of the EEA or Switzerland). 
+
+By clicking "I Accept" below or by using or distributing any portion or element of the 
+Llama Materials, you agree to be bound by this Agreement.
+
+1. License Rights and Redistribution. 
+
+      a. Grant of Rights. You are granted a non-exclusive, worldwide, non-
+transferable and royalty-free limited license under Meta's intellectual property or 
+other rights owned by Meta embodied in the Llama Materials to use, reproduce, 
+distribute, copy, create derivative works of, and make modifications to the Llama 
+Materials.  
+      
+      b. Redistribution and Use.  
+
+            i. If you distribute or make the Llama Materials, or any derivative works 
+thereof, available to a third party, you shall provide a copy of this Agreement to such 
+third party. 
+            ii.  If you receive Llama Materials, or any derivative works thereof, from 
+a Licensee as part of an integrated end user product, then Section 2 of this 
+Agreement will not apply to you. 
+
+            iii. You must retain in all copies of the Llama Materials that you 
+distribute the following attribution notice within a "Notice" text file distributed as a 
+part of such copies: "Llama 2 is licensed under the LLAMA 2 Community License, 
+Copyright (c) Meta Platforms, Inc. All Rights Reserved."
+
+            iv. Your use of the Llama Materials must comply with applicable laws 
+and regulations (including trade compliance laws and regulations) and adhere to the 
+Acceptable Use Policy for the Llama Materials (available at 
+https://ai.meta.com/llama/use-policy), which is hereby incorporated by reference into 
+this Agreement.
+
+            v. You will not use the Llama Materials or any output or results of the 
+Llama Materials to improve any other large language model (excluding Llama 2 or 
+derivative works thereof).  
+
+2. Additional Commercial Terms. If, on the Llama 2 version release date, the 
+monthly active users of the products or services made available by or for Licensee, 
+or Licensee's affiliates, is greater than 700 million monthly active users in the 
+preceding calendar month, you must request a license from Meta, which Meta may 
+grant to you in its sole discretion, and you are not authorized to exercise any of the 
+rights under this Agreement unless or until Meta otherwise expressly grants you 
+such rights.
+            
+3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE 
+LLAMA MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE 
+PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 
+EITHER EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY 
+WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR 
+FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE 
+FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING 
+THE LLAMA MATERIALS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR 
+USE OF THE LLAMA MATERIALS AND ANY OUTPUT AND RESULTS.
+
+4. Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES BE 
+LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, 
+NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS 
+AGREEMENT, FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, 
+CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN 
+IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF 
+ANY OF THE FOREGOING.
+ 
+5. Intellectual Property.
+
+      a. No trademark licenses are granted under this Agreement, and in 
+connection with the Llama Materials, neither Meta nor Licensee may use any name 
+or mark owned by or associated with the other or any of its affiliates, except as 
+required for reasonable and customary use in describing and redistributing the 
+Llama Materials.
+
+      b. Subject to Meta's ownership of Llama Materials and derivatives made by or 
+for Meta, with respect to any derivative works and modifications of the Llama 
+Materials that are made by you, as between you and Meta, you are and will be the 
+owner of such derivative works and modifications.
+
+      c. If you institute litigation or other proceedings against Meta or any entity 
+(including a cross-claim or counterclaim in a lawsuit) alleging that the Llama 
+Materials or Llama 2 outputs or results, or any portion of any of the foregoing, 
+constitutes infringement of intellectual property or other rights owned or licensable 
+by you, then any licenses granted to you under this Agreement shall terminate as of 
+the date such litigation or claim is filed or instituted. You will indemnify and hold 
+harmless Meta from and against any claim by any third party arising out of or related 
+to your use or distribution of the Llama Materials.
+
+6. Term and Termination. The term of this Agreement will commence upon your 
+acceptance of this Agreement or access to the Llama Materials and will continue in 
+full force and effect until terminated in accordance with the terms and conditions 
+herein. Meta may terminate this Agreement if you are in breach of any term or 
+condition of this Agreement. Upon termination of this Agreement, you shall delete 
+and cease use of the Llama Materials. Sections 3, 4 and 7 shall survive the 
+termination of this Agreement. 
+
+7. Governing Law and Jurisdiction. This Agreement will be governed and 
+construed under the laws of the State of California without regard to choice of law 
+principles, and the UN Convention on Contracts for the International Sale of Goods 
+does not apply to this Agreement. The courts of California shall have exclusive 
+jurisdiction of any dispute arising out of this Agreement. 
+

models/meta-llama/Llama-2-7b-hf/README.md

@@ -0,0 +1,134 @@
+---
+extra_gated_heading: Access Llama 2 on Hugging Face
+extra_gated_description: >-
+  This is a form to enable access to Llama 2 on Hugging Face after you have been
+  granted access from Meta. Please visit the [Meta website](https://ai.meta.com/resources/models-and-libraries/llama-downloads) and accept our
+  license terms and acceptable use policy before submitting this form. Requests
+  will be processed in 1-2 days.
+extra_gated_button_content: Submit
+extra_gated_fields:
+  I agree to share my name, email address and username with Meta and confirm that I have already been granted download access on the Meta website: checkbox
+language:
+- en
+pipeline_tag: text-generation
+inference: false
+tags:
+- facebook
+- meta
+- pytorch
+- llama
+- llama-2
+---
+# **Llama 2**
+Llama 2 is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. This is the repository for the 7B pretrained model, converted for the Hugging Face Transformers format. Links to other models can be found in the index at the bottom.
+
+## Model Details
+*Note: Use of this model is governed by the Meta license. In order to download the model weights and tokenizer, please visit the [website](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) and accept our License before requesting access here.*
+
+Meta developed and publicly released the Llama 2 family of large language models (LLMs), a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama-2-Chat, are optimized for dialogue use cases. Llama-2-Chat models outperform open-source chat models on most benchmarks we tested, and in our human evaluations for helpfulness and safety, are on par with some popular closed-source models like ChatGPT and PaLM.
+
+**Model Developers** Meta
+
+**Variations** Llama 2 comes in a range of parameter sizes — 7B, 13B, and 70B — as well as pretrained and fine-tuned variations.
+
+**Input** Models input text only.
+
+**Output** Models generate text only.
+
+**Model Architecture** Llama 2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align to human preferences for helpfulness and safety.
+
+
+||Training Data|Params|Content Length|GQA|Tokens|LR|
+|---|---|---|---|---|---|---|
+|Llama 2|*A new mix of publicly available online data*|7B|4k|&#10007;|2.0T|3.0 x 10<sup>-4</sup>|
+|Llama 2|*A new mix of publicly available online data*|13B|4k|&#10007;|2.0T|3.0 x 10<sup>-4</sup>|
+|Llama 2|*A new mix of publicly available online data*|70B|4k|&#10004;|2.0T|1.5 x 10<sup>-4</sup>|
+
+*Llama 2 family of models.* Token counts refer to pretraining data only. All models are trained with a global batch-size of 4M tokens. Bigger models -  70B -- use Grouped-Query Attention (GQA) for improved inference scalability.
+
+**Model Dates** Llama 2 was trained between January 2023 and July 2023.
+
+**Status** This is a static model trained on an offline dataset. Future versions of the tuned models will be released as we improve model safety with community feedback.
+
+**License** A custom commercial license is available at: [https://ai.meta.com/resources/models-and-libraries/llama-downloads/](https://ai.meta.com/resources/models-and-libraries/llama-downloads/)
+
+## Intended Use
+**Intended Use Cases** Llama 2 is intended for commercial and research use in English. Tuned models are intended for assistant-like chat, whereas pretrained models can be adapted for a variety of natural language generation tasks.
+
+To get the expected features and performance for the chat versions, a specific formatting needs to be followed, including the `INST` and `<<SYS>>` tags, `BOS` and `EOS` tokens, and the whitespaces and breaklines in between (we recommend calling `strip()` on inputs to avoid double-spaces). See our reference code in github for details: [`chat_completion`](https://github.com/facebookresearch/llama/blob/main/llama/generation.py#L212).
+
+**Out-of-scope Uses** Use in any manner that violates applicable laws or regulations (including trade compliance laws).Use in languages other than English. Use in any other way that is prohibited by the Acceptable Use Policy and Licensing Agreement for Llama 2.
+
+## Hardware and Software
+**Training Factors** We used custom training libraries, Meta's Research Super Cluster, and production clusters for pretraining. Fine-tuning, annotation, and evaluation were also performed on third-party cloud compute.
+
+**Carbon Footprint** Pretraining utilized a cumulative 3.3M GPU hours of computation on hardware of type A100-80GB (TDP of 350-400W). Estimated total emissions were 539 tCO2eq, 100% of which were offset by Meta’s sustainability program.
+
+||Time (GPU hours)|Power Consumption (W)|Carbon Emitted(tCO<sub>2</sub>eq)|
+|---|---|---|---|
+|Llama 2 7B|184320|400|31.22|
+|Llama 2 13B|368640|400|62.44|
+|Llama 2 70B|1720320|400|291.42|
+|Total|3311616||539.00|
+
+**CO<sub>2</sub> emissions during pretraining.** Time: total GPU time required for training each model. Power Consumption: peak power capacity per GPU device for the GPUs used adjusted for power usage efficiency. 100% of the emissions are directly offset by Meta's sustainability program, and because we are openly releasing these models, the pretraining costs do not need to be incurred by others.
+
+## Training Data
+**Overview** Llama 2 was pretrained on 2 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction datasets, as well as over one million new human-annotated examples. Neither the pretraining nor the fine-tuning datasets include Meta user data.
+
+**Data Freshness** The pretraining data has a cutoff of September 2022, but some tuning data is more recent, up to July 2023.
+
+## Evaluation Results
+
+In this section, we report the results for the Llama 1 and Llama 2 models on standard academic benchmarks.For all the evaluations, we use our internal evaluations library.
+
+|Model|Size|Code|Commonsense Reasoning|World Knowledge|Reading Comprehension|Math|MMLU|BBH|AGI Eval|
+|---|---|---|---|---|---|---|---|---|---|
+|Llama 1|7B|14.1|60.8|46.2|58.5|6.95|35.1|30.3|23.9|
+|Llama 1|13B|18.9|66.1|52.6|62.3|10.9|46.9|37.0|33.9|
+|Llama 1|33B|26.0|70.0|58.4|67.6|21.4|57.8|39.8|41.7|
+|Llama 1|65B|30.7|70.7|60.5|68.6|30.8|63.4|43.5|47.6|
+|Llama 2|7B|16.8|63.9|48.9|61.3|14.6|45.3|32.6|29.3|
+|Llama 2|13B|24.5|66.9|55.4|65.8|28.7|54.8|39.4|39.1|
+|Llama 2|70B|**37.5**|**71.9**|**63.6**|**69.4**|**35.2**|**68.9**|**51.2**|**54.2**|
+
+**Overall performance on grouped academic benchmarks.** *Code:* We report the average pass@1 scores of our models on HumanEval and MBPP. *Commonsense Reasoning:* We report the average of PIQA, SIQA, HellaSwag, WinoGrande, ARC easy and challenge, OpenBookQA, and CommonsenseQA. We report 7-shot results for CommonSenseQA and 0-shot results for all other benchmarks. *World Knowledge:* We evaluate the 5-shot performance on NaturalQuestions and TriviaQA and report the average. *Reading Comprehension:* For reading comprehension, we report the 0-shot average on SQuAD, QuAC, and BoolQ. *MATH:* We report the average of the GSM8K (8 shot) and MATH (4 shot) benchmarks at top 1.
+
+|||TruthfulQA|Toxigen|
+|---|---|---|---|
+|Llama 1|7B|27.42|23.00|
+|Llama 1|13B|41.74|23.08|
+|Llama 1|33B|44.19|22.57|
+|Llama 1|65B|48.71|21.77|
+|Llama 2|7B|33.29|**21.25**|
+|Llama 2|13B|41.86|26.10|
+|Llama 2|70B|**50.18**|24.60|
+
+**Evaluation of pretrained LLMs on automatic safety benchmarks.** For TruthfulQA, we present the percentage of generations that are both truthful and informative (the higher the better). For ToxiGen, we present the percentage of toxic generations (the smaller the better).
+
+
+|||TruthfulQA|Toxigen|
+|---|---|---|---|
+|Llama-2-Chat|7B|57.04|**0.00**|
+|Llama-2-Chat|13B|62.18|**0.00**|
+|Llama-2-Chat|70B|**64.14**|0.01|
+
+**Evaluation of fine-tuned LLMs on different safety datasets.** Same metric definitions as above.
+
+## Ethical Considerations and Limitations
+Llama 2 is a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Llama 2’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of Llama 2, developers should perform safety testing and tuning tailored to their specific applications of the model.
+
+Please see the Responsible Use Guide available at [https://ai.meta.com/llama/responsible-use-guide/](https://ai.meta.com/llama/responsible-use-guide)
+
+## Reporting Issues
+Please report any software “bug,” or other problems with the models through one of the following means:
+- Reporting issues with the model: [github.com/facebookresearch/llama](http://github.com/facebookresearch/llama)
+- Reporting problematic content generated by the model: [developers.facebook.com/llama_output_feedback](http://developers.facebook.com/llama_output_feedback)
+- Reporting bugs and security concerns: [facebook.com/whitehat/info](http://facebook.com/whitehat/info)
+
+## Llama Model Index
+|Model|Llama2|Llama2-hf|Llama2-chat|Llama2-chat-hf|
+|---|---|---|---|---|
+|7B| [Link](https://huggingface.co/llamaste/Llama-2-7b) | [Link](https://huggingface.co/llamaste/Llama-2-7b-hf) | [Link](https://huggingface.co/llamaste/Llama-2-7b-chat) | [Link](https://huggingface.co/llamaste/Llama-2-7b-chat-hf)|
+|13B| [Link](https://huggingface.co/llamaste/Llama-2-13b) | [Link](https://huggingface.co/llamaste/Llama-2-13b-hf) | [Link](https://huggingface.co/llamaste/Llama-2-13b-chat) | [Link](https://huggingface.co/llamaste/Llama-2-13b-hf)|
+|70B| [Link](https://huggingface.co/llamaste/Llama-2-70b) | [Link](https://huggingface.co/llamaste/Llama-2-70b-hf) | [Link](https://huggingface.co/llamaste/Llama-2-70b-chat) | [Link](https://huggingface.co/llamaste/Llama-2-70b-hf)|

models/meta-llama/Llama-2-7b-hf/USE_POLICY.md

@@ -0,0 +1,50 @@
+# Llama 2 Acceptable Use Policy
+
+Meta is committed to promoting safe and fair use of its tools and features, including Llama 2. If you access or use Llama 2, you agree to this Acceptable Use Policy (“Policy”). The most recent copy of this policy can be found at [ai.meta.com/llama/use-policy](http://ai.meta.com/llama/use-policy).
+
+## Prohibited Uses
+We want everyone to use Llama 2 safely and responsibly. You agree you will not use, or allow others to use, Llama 2 to: 
+
+1. Violate the law or others’ rights, including to:
+    1. Engage in, promote, generate, contribute to, encourage, plan, incite, or further illegal or unlawful activity or content, such as: 
+        1. Violence or terrorism 
+        2. Exploitation or harm to children, including the solicitation, creation, acquisition, or dissemination of child exploitative content or failure to report Child Sexual Abuse Material
+        3. Human trafficking, exploitation, and sexual violence
+        4. The illegal distribution of information or materials to minors, including obscene materials, or failure to employ legally required age-gating in connection with such information or materials.
+        5. Sexual solicitation
+        6. Any other criminal activity
+    2. Engage in, promote, incite, or facilitate the harassment, abuse, threatening, or bullying of individuals or groups of individuals
+    3. Engage in, promote, incite, or facilitate discrimination or other unlawful or harmful conduct in the provision of employment, employment benefits, credit, housing, other economic benefits, or other essential goods and services
+    4. Engage in the unauthorized or unlicensed practice of any profession including, but not limited to, financial, legal, medical/health, or related professional practices 
+    5. Collect, process, disclose, generate, or infer health, demographic, or other sensitive personal or private information about individuals without rights and consents required by applicable laws
+    6. Engage in or facilitate any action or generate any content that infringes, misappropriates, or otherwise violates any third-party rights, including the outputs or results of any products or services using the Llama 2 Materials
+    7. Create, generate, or facilitate the creation of malicious code, malware, computer viruses or do anything else that could disable, overburden, interfere with or impair the proper working, integrity, operation or appearance of a website or computer system 
+
+
+
+2. Engage in, promote, incite, facilitate, or assist in the planning or development of activities that present a risk of death or bodily harm to individuals, including use of Llama 2 related to the following:
+    1. Military, warfare, nuclear industries or applications, espionage, use for materials or activities that are subject to the International Traffic Arms Regulations (ITAR) maintained by the United States Department of State
+    2. Guns and illegal weapons (including weapon development)
+    3. Illegal drugs and regulated/controlled substances
+    4. Operation of critical infrastructure, transportation technologies, or heavy machinery
+    5. Self-harm or harm to others, including suicide, cutting, and eating disorders
+    6. Any content intended to incite or promote violence, abuse, or any infliction of bodily harm to an individual
+
+
+
+3. Intentionally deceive or mislead others, including use of Llama 2 related to the following:
+    1. Generating, promoting, or furthering fraud or the creation or promotion of disinformation
+    2. Generating, promoting, or furthering defamatory content, including the creation of defamatory statements, images, or other content
+    3. Generating, promoting, or further distributing spam
+    4. Impersonating another individual without consent, authorization, or legal right
+    5. Representing that the use of Llama 2 or outputs are human-generated
+    6. Generating or facilitating false online engagement, including fake reviews and other means of fake online engagement 
+4. Fail to appropriately disclose to end users any known dangers of your AI system 
+
+Please report any violation of this Policy, software “bug,” or other problems that could lead to a violation of this Policy through one of the following means:
+
+* Reporting issues with the model: [github.com/facebookresearch/llama](http://github.com/facebookresearch/llama)
+* Reporting risky content generated by the model: [developers.facebook.com/llama_output_feedback](http://developers.facebook.com/llama_output_feedback)
+* Reporting bugs and security concerns: [facebook.com/whitehat/info](http://facebook.com/whitehat/info)
+* Reporting violations of the Acceptable Use Policy or unlicensed uses of Llama: [LlamaUseReport@meta.com](mailto:LlamaUseReport@meta.com)
+

models/meta-llama/Llama-2-7b-hf/config.json

@@ -0,0 +1,25 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 32,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.31.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

models/meta-llama/Llama-2-7b-hf/generation_config.json

@@ -0,0 +1,10 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 32000,
+  "do_sample": true,
+  "temperature": 0.9,
+  "top_p": 0.6,
+  "transformers_version": "4.31.0.dev0"
+}

models/meta-llama/Llama-2-7b-hf/model.safetensors.index.json

@@ -0,0 +1,330 @@
+{
+  "metadata": {
+    "total_size": 13476839424
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00002-of-00002.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.rotary_emb.inv_freq": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.rotary_emb.inv_freq": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.rotary_emb.inv_freq": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.rotary_emb.inv_freq": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.rotary_emb.inv_freq": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.rotary_emb.inv_freq": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.rotary_emb.inv_freq": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.rotary_emb.inv_freq": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.rotary_emb.inv_freq": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.norm.weight": "model-00002-of-00002.safetensors"
+  }
+}

models/meta-llama/Llama-2-7b-hf/pytorch_model.bin.index.json

@@ -0,0 +1,330 @@
+{
+  "metadata": {
+    "total_size": 26953670656
+  },
+  "weight_map": {
+    "lm_head.weight": "pytorch_model-00003-of-00003.bin",
+    "model.embed_tokens.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.11.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.12.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.20.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.23.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
+    "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
+    "model.layers.24.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
+    "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
+    "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
+    "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
+    "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
+    "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
+    "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.3.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.30.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
+    "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
+    "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
+    "model.layers.4.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
+    "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
+    "model.norm.weight": "pytorch_model-00003-of-00003.bin"
+  }
+}

models/meta-llama/Llama-2-7b-hf/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

models/meta-llama/Llama-3.1-8B/.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

models/meta-llama/Llama-3.1-8B/LICENSE

@@ -0,0 +1,114 @@
+LLAMA 3.1 COMMUNITY LICENSE AGREEMENT
+Llama 3.1 Version Release Date: July 23, 2024
+
+“Agreement” means the terms and conditions for use, reproduction, distribution and modification of the
+Llama Materials set forth herein.
+
+“Documentation” means the specifications, manuals and documentation accompanying Llama 3.1
+distributed by Meta at https://llama.meta.com/doc/overview.
+
+“Licensee” or “you” means you, or your employer or any other person or entity (if you are entering into
+this Agreement on such person or entity’s behalf), of the age required under applicable laws, rules or
+regulations to provide legal consent and that has legal authority to bind your employer or such other
+person or entity if you are entering in this Agreement on their behalf.
+
+“Llama 3.1” means the foundational large language models and software and algorithms, including
+machine-learning model code, trained model weights, inference-enabling code, training-enabling code,
+fine-tuning enabling code and other elements of the foregoing distributed by Meta at
+https://llama.meta.com/llama-downloads.
+
+“Llama Materials” means, collectively, Meta’s proprietary Llama 3.1 and Documentation (and any
+portion thereof) made available under this Agreement.
+
+“Meta” or “we” means Meta Platforms Ireland Limited (if you are located in or, if you are an entity, your
+principal place of business is in the EEA or Switzerland) and Meta Platforms, Inc. (if you are located
+outside of the EEA or Switzerland).
+
+By clicking “I Accept” below or by using or distributing any portion or element of the Llama Materials,
+you agree to be bound by this Agreement.
+
+1. License Rights and Redistribution.
+
+  a. Grant of Rights. You are granted a non-exclusive, worldwide, non-transferable and royalty-free
+limited license under Meta’s intellectual property or other rights owned by Meta embodied in the Llama
+Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the
+Llama Materials.
+
+  b. Redistribution and Use.
+
+      i. If you distribute or make available the Llama Materials (or any derivative works
+thereof), or a product or service (including another AI model) that contains any of them, you shall (A)
+provide a copy of this Agreement with any such Llama Materials; and (B) prominently display “Built with
+Llama” on a related website, user interface, blogpost, about page, or product documentation. If you use
+the Llama Materials or any outputs or results of the Llama Materials to create, train, fine tune, or
+otherwise improve an AI model, which is distributed or made available, you shall also include “Llama” at
+the beginning of any such AI model name.
+
+      ii. If you receive Llama Materials, or any derivative works thereof, from a Licensee as part 
+of an integrated end user product, then Section 2 of this Agreement will not apply to you.
+
+      iii. You must retain in all copies of the Llama Materials that you distribute the following
+attribution notice within a “Notice” text file distributed as a part of such copies: “Llama 3.1 is
+licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights
+Reserved.”
+
+      iv. Your use of the Llama Materials must comply with applicable laws and regulations
+(including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for the Llama
+Materials (available at https://llama.meta.com/llama3_1/use-policy), which is hereby incorporated by
+reference into this Agreement.
+
+2. Additional Commercial Terms. If, on the Llama 3.1 version release date, the monthly active users
+of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700
+million monthly active users in the preceding calendar month, you must request a license from Meta,
+which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the
+rights under this Agreement unless or until Meta otherwise expressly grants you such rights.
+
+3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE LLAMA MATERIALS AND ANY
+OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF
+ANY KIND, AND META DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED,
+INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT,
+MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR
+DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE LLAMA MATERIALS AND
+ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE LLAMA MATERIALS AND ANY OUTPUT AND
+RESULTS.
+
+4. Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF
+LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING
+OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL,
+INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE BEEN ADVISED
+OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+
+5. Intellectual Property.
+
+  a. No trademark licenses are granted under this Agreement, and in connection with the Llama
+Materials, neither Meta nor Licensee may use any name or mark owned by or associated with the other
+or any of its affiliates, except as required for reasonable and customary use in describing and
+redistributing the Llama Materials or as set forth in this Section 5(a). Meta hereby grants you a license to
+use “Llama” (the “Mark”) solely as required to comply with the last sentence of Section 1.b.i. You will
+comply with Meta’s brand guidelines (currently accessible at
+https://about.meta.com/brand/resources/meta/company-brand/ ). All goodwill arising out of your use
+of the Mark will inure to the benefit of Meta.
+
+  b. Subject to Meta’s ownership of Llama Materials and derivatives made by or for Meta, with
+respect to any derivative works and modifications of the Llama Materials that are made by you, as
+between you and Meta, you are and will be the owner of such derivative works and modifications.
+
+  c. If you institute litigation or other proceedings against Meta or any entity (including a
+cross-claim or counterclaim in a lawsuit) alleging that the Llama Materials or Llama 3.1 outputs or
+results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other
+rights owned or licensable by you, then any licenses granted to you under this Agreement shall
+terminate as of the date such litigation or claim is filed or instituted. You will indemnify and hold
+harmless Meta from and against any claim by any third party arising out of or related to your use or
+distribution of the Llama Materials.
+
+6. Term and Termination. The term of this Agreement will commence upon your acceptance of this
+Agreement or access to the Llama Materials and will continue in full force and effect until terminated in
+accordance with the terms and conditions herein. Meta may terminate this Agreement if you are in
+breach of any term or condition of this Agreement. Upon termination of this Agreement, you shall delete
+and cease use of the Llama Materials. Sections 3, 4 and 7 shall survive the termination of this
+Agreement.
+
+7. Governing Law and Jurisdiction. This Agreement will be governed and construed under the laws of
+the State of California without regard to choice of law principles, and the UN Convention on Contracts
+for the International Sale of Goods does not apply to this Agreement. The courts of California shall have
+exclusive jurisdiction of any dispute arising out of this Agreement.

models/meta-llama/Llama-3.1-8B/config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": 128001,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "low_freq_factor": 1.0,
+    "high_freq_factor": 4.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.43.0.dev0",
+  "use_cache": true,
+  "vocab_size": 128256
+}

models/meta-llama/Llama-3.1-8B/generation_config.json

@@ -0,0 +1,9 @@
+{
+  "do_sample": true,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "eos_token_id": 128001,
+  "transformers_version": "4.43.0.dev0"
+}

quantization_metric/bit_layers/singletask_arc_easy/bits_1/configure_5.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_1/configure_6.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_1/configure_7.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_1/configure_8.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_1/configure_9.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_2/configure_17.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_2/configure_19.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_2/configure_20.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_2/configure_21.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_2/configure_22.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_2/configure_23.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_2/configure_24.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4
+]

quantization_metric/bit_layers/singletask_arc_easy/bits_2/configure_25.json

@@ -0,0 +1,34 @@
+[
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  4,
+  2,
+  4,
+  2,
+  4,
+  4,
+  4,
+  4
+]