qaihm-bot commited on
Commit
8718a49
·
verified ·
1 Parent(s): 08671f9

See https://github.com/qualcomm/ai-hub-models/releases/v0.63.0 for changelog.

Files changed (1) hide show
  1. README.md +6 -8
README.md CHANGED
@@ -16,7 +16,7 @@ pipeline_tag: text-generation
16
  Falcon3 family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B.
17
 
18
  This is based on the implementation of Falcon3-7B-Instruct found [here](https://huggingface.co/tiiuae/Falcon3-7B-Instruct).
19
- This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.62.2/src/qai_hub_models/models/falcon_v3_7b_instruct) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
20
 
21
  Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
22
 
@@ -30,12 +30,12 @@ See the [LLM-on-Genie](https://github.com/qualcomm/ai-hub-apps/tree/main/tutoria
30
 
31
  ## Getting Started
32
  Due to licensing restrictions, we cannot distribute pre-exported model assets for this model.
33
- Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.62.2/src/qai_hub_models/models/falcon_v3_7b_instruct) Python library to compile and export the model with your own:
34
  - Custom weights (e.g., fine-tuned checkpoints)
35
  - Custom input shapes
36
  - Target device and runtime configurations
37
 
38
- See our repository for [Falcon3-7B-Instruct on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.62.2/src/qai_hub_models/models/falcon_v3_7b_instruct) for usage instructions.
39
 
40
 
41
  ## Model Details
@@ -50,21 +50,19 @@ See our repository for [Falcon3-7B-Instruct on GitHub](https://github.com/qualco
50
  ## Performance Summary
51
  | Model | Runtime | Precision | Chipset | Context Length | Response Rate (tokens per second) | Time To First Token (range, seconds)
52
  |---|---|---|---|---|---|---
53
- | Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® 8 Elite Gen 5 Mobile | 4096 | 17.845033645629883 | 0.098628 - 3.156096
54
- | Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® 8 Elite Mobile | 4096 | 15.377281188964844 | 0.121521 - 3.888672
55
  | Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® X2 Elite | 4096 | 24.13884735107422 | 0.125103 - 4.003296
56
  | Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® X Elite | 4096 | 5.068192481994629 | 0.1874 - 5.9968
57
  | Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ IQ-9075 | 4096 | 10.775397300720215 | 0.162771 - 5.208672
58
  | Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ IQ-X7181 | 4096 | 5.068192481994629 | 0.1874 - 5.9968
59
  | Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ Q-8750 | 4096 | 15.377281188964844 | 0.121521 - 3.888672
60
- | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® 8 Elite Gen 5 Mobile | 4096 | 14.864458 | 0.14304516129032258 - 4.577445161290322
61
- | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® 8 Elite Mobile | 4096 | 15.741016 | 0.14337 - 4.58784
62
  | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® X2 Elite | 4096 | 19.845977 | 0.12204629032258064 - 3.9054812903225806
63
  | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® X Elite | 4096 | 11.534744 | 0.19841499999999998 - 6.349279999999999
64
  | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-8275 | 4096 | 9.483059 | 0.219881 - 7.036192
65
  | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-9075 | 4096 | 10.134524 | 0.17698829032258065 - 5.663625290322581
66
  | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-X7181 | 4096 | 11.534744 | 0.19841499999999998 - 6.349279999999999
67
- | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ Q-8750 | 4096 | 15.741016 | 0.14337 - 4.58784
68
 
69
  ## License
70
  * The license for the original implementation of Falcon3-7B-Instruct can be found
 
16
  Falcon3 family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B.
17
 
18
  This is based on the implementation of Falcon3-7B-Instruct found [here](https://huggingface.co/tiiuae/Falcon3-7B-Instruct).
19
+ This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/falcon_v3_7b_instruct) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
20
 
21
  Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
22
 
 
30
 
31
  ## Getting Started
32
  Due to licensing restrictions, we cannot distribute pre-exported model assets for this model.
33
+ Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/falcon_v3_7b_instruct) Python library to compile and export the model with your own:
34
  - Custom weights (e.g., fine-tuned checkpoints)
35
  - Custom input shapes
36
  - Target device and runtime configurations
37
 
38
+ See our repository for [Falcon3-7B-Instruct on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/falcon_v3_7b_instruct) for usage instructions.
39
 
40
 
41
  ## Model Details
 
50
  ## Performance Summary
51
  | Model | Runtime | Precision | Chipset | Context Length | Response Rate (tokens per second) | Time To First Token (range, seconds)
52
  |---|---|---|---|---|---|---
 
 
53
  | Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® X2 Elite | 4096 | 24.13884735107422 | 0.125103 - 4.003296
54
  | Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® X Elite | 4096 | 5.068192481994629 | 0.1874 - 5.9968
55
  | Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ IQ-9075 | 4096 | 10.775397300720215 | 0.162771 - 5.208672
56
  | Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ IQ-X7181 | 4096 | 5.068192481994629 | 0.1874 - 5.9968
57
  | Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ Q-8750 | 4096 | 15.377281188964844 | 0.121521 - 3.888672
58
+ | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 4096 | 11.064772 | 0.18091438709677415 - 5.789260387096773
59
+ | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® 8 Elite For Galaxy Mobile | 4096 | 10.326481 | 0.1718072580645161 - 5.497832258064515
60
  | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® X2 Elite | 4096 | 19.845977 | 0.12204629032258064 - 3.9054812903225806
61
  | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® X Elite | 4096 | 11.534744 | 0.19841499999999998 - 6.349279999999999
62
  | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-8275 | 4096 | 9.483059 | 0.219881 - 7.036192
63
  | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-9075 | 4096 | 10.134524 | 0.17698829032258065 - 5.663625290322581
64
  | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-X7181 | 4096 | 11.534744 | 0.19841499999999998 - 6.349279999999999
65
+ | Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ Q-8750 | 4096 | 10.326481 | 0.1718072580645161 - 5.497832258064515
66
 
67
  ## License
68
  * The license for the original implementation of Falcon3-7B-Instruct can be found