bmah-dmx commited on
Commit
653dc0c
·
verified ·
1 Parent(s): ec82278

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md CHANGED
@@ -13,3 +13,39 @@ model-index:
13
  - type: perplexity (BASIC)
14
  value: 32.05778110592746
15
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  - type: perplexity (BASIC)
14
  value: 32.05778110592746
15
  ---
16
+ This is a d-Matrix functional reference of the OPT-125M model.
17
+ The reference provides the following functional *configurations*:
18
+ Configuration | Explanation
19
+ :-- | :--
20
+ **`BASELINE`** | a reference functionally equivalent to the original model
21
+ **`BASIC`** | all linear algebraic operands quantized to `BFP16-64`, and all other operations transformed to approximated kernel simulations
22
+
23
+
24
+ ### Usage
25
+
26
+ Install d-Matrix [Dmx_Compressor](https://github.com/d-matrix-ai/dmx-compressor) first.
27
+ ```sh
28
+ pip install dmx_compressor
29
+ pip install lm-eval
30
+ ```
31
+
32
+ The following is an example model and its evaluation.
33
+
34
+ ```python
35
+ from dmx.compressor import config_rules
36
+ from dmx.compressor.modeling import DmxModel
37
+ import lm_eval
38
+
39
+ model_args = f"pretrained="d-matrix/d-matrix/opt-125m",revision="main",trust_remote_code=True"
40
+
41
+ lm = lm_eval.api.registry.get_model("hf").create_from_arg_string(model_args, {"batch_size": 1})
42
+
43
+ # Transform the model with DMX
44
+ model = DmxModel.from_torch(lm._model)
45
+ model.transform(model.dmx_config, *eval(f"config_rules.{dmx_config}")) # Assign desired configuration to dmx_config, i.e. "BASELINE"
46
+ lm._model = model # Set the transformed model back
47
+
48
+ task_dict = lm_eval.tasks.get_task_dict([task_name]) # Assign desired task to task_name, i.e. "wikitext"
49
+
50
+ evaluation_results = lm_eval.evaluate(lm, task_dict)
51
+ ```