v0.63.0
Browse filesSee https://github.com/qualcomm/ai-hub-models/releases/v0.63.0 for changelog.
README.md
CHANGED
|
@@ -16,7 +16,7 @@ pipeline_tag: text-generation
|
|
| 16 |
Falcon3 family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B.
|
| 17 |
|
| 18 |
This is based on the implementation of Falcon3-7B-Instruct found [here](https://huggingface.co/tiiuae/Falcon3-7B-Instruct).
|
| 19 |
-
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.
|
| 20 |
|
| 21 |
Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
|
| 22 |
|
|
@@ -30,12 +30,12 @@ See the [LLM-on-Genie](https://github.com/qualcomm/ai-hub-apps/tree/main/tutoria
|
|
| 30 |
|
| 31 |
## Getting Started
|
| 32 |
Due to licensing restrictions, we cannot distribute pre-exported model assets for this model.
|
| 33 |
-
Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.
|
| 34 |
- Custom weights (e.g., fine-tuned checkpoints)
|
| 35 |
- Custom input shapes
|
| 36 |
- Target device and runtime configurations
|
| 37 |
|
| 38 |
-
See our repository for [Falcon3-7B-Instruct on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.
|
| 39 |
|
| 40 |
|
| 41 |
## Model Details
|
|
@@ -50,21 +50,19 @@ See our repository for [Falcon3-7B-Instruct on GitHub](https://github.com/qualco
|
|
| 50 |
## Performance Summary
|
| 51 |
| Model | Runtime | Precision | Chipset | Context Length | Response Rate (tokens per second) | Time To First Token (range, seconds)
|
| 52 |
|---|---|---|---|---|---|---
|
| 53 |
-
| Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® 8 Elite Gen 5 Mobile | 4096 | 17.845033645629883 | 0.098628 - 3.156096
|
| 54 |
-
| Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® 8 Elite Mobile | 4096 | 15.377281188964844 | 0.121521 - 3.888672
|
| 55 |
| Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® X2 Elite | 4096 | 24.13884735107422 | 0.125103 - 4.003296
|
| 56 |
| Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® X Elite | 4096 | 5.068192481994629 | 0.1874 - 5.9968
|
| 57 |
| Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ IQ-9075 | 4096 | 10.775397300720215 | 0.162771 - 5.208672
|
| 58 |
| Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ IQ-X7181 | 4096 | 5.068192481994629 | 0.1874 - 5.9968
|
| 59 |
| Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ Q-8750 | 4096 | 15.377281188964844 | 0.121521 - 3.888672
|
| 60 |
-
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® 8 Elite Gen 5 Mobile | 4096 |
|
| 61 |
-
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® 8 Elite Mobile | 4096 |
|
| 62 |
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® X2 Elite | 4096 | 19.845977 | 0.12204629032258064 - 3.9054812903225806
|
| 63 |
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® X Elite | 4096 | 11.534744 | 0.19841499999999998 - 6.349279999999999
|
| 64 |
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-8275 | 4096 | 9.483059 | 0.219881 - 7.036192
|
| 65 |
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-9075 | 4096 | 10.134524 | 0.17698829032258065 - 5.663625290322581
|
| 66 |
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-X7181 | 4096 | 11.534744 | 0.19841499999999998 - 6.349279999999999
|
| 67 |
-
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ Q-8750 | 4096 |
|
| 68 |
|
| 69 |
## License
|
| 70 |
* The license for the original implementation of Falcon3-7B-Instruct can be found
|
|
|
|
| 16 |
Falcon3 family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B.
|
| 17 |
|
| 18 |
This is based on the implementation of Falcon3-7B-Instruct found [here](https://huggingface.co/tiiuae/Falcon3-7B-Instruct).
|
| 19 |
+
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/falcon_v3_7b_instruct) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
|
| 20 |
|
| 21 |
Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
|
| 22 |
|
|
|
|
| 30 |
|
| 31 |
## Getting Started
|
| 32 |
Due to licensing restrictions, we cannot distribute pre-exported model assets for this model.
|
| 33 |
+
Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/falcon_v3_7b_instruct) Python library to compile and export the model with your own:
|
| 34 |
- Custom weights (e.g., fine-tuned checkpoints)
|
| 35 |
- Custom input shapes
|
| 36 |
- Target device and runtime configurations
|
| 37 |
|
| 38 |
+
See our repository for [Falcon3-7B-Instruct on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/falcon_v3_7b_instruct) for usage instructions.
|
| 39 |
|
| 40 |
|
| 41 |
## Model Details
|
|
|
|
| 50 |
## Performance Summary
|
| 51 |
| Model | Runtime | Precision | Chipset | Context Length | Response Rate (tokens per second) | Time To First Token (range, seconds)
|
| 52 |
|---|---|---|---|---|---|---
|
|
|
|
|
|
|
| 53 |
| Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® X2 Elite | 4096 | 24.13884735107422 | 0.125103 - 4.003296
|
| 54 |
| Falcon3-7B-Instruct | GENIE | w4a16 | Snapdragon® X Elite | 4096 | 5.068192481994629 | 0.1874 - 5.9968
|
| 55 |
| Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ IQ-9075 | 4096 | 10.775397300720215 | 0.162771 - 5.208672
|
| 56 |
| Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ IQ-X7181 | 4096 | 5.068192481994629 | 0.1874 - 5.9968
|
| 57 |
| Falcon3-7B-Instruct | GENIE | w4a16 | Qualcomm® Dragonwing™ Q-8750 | 4096 | 15.377281188964844 | 0.121521 - 3.888672
|
| 58 |
+
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 4096 | 11.064772 | 0.18091438709677415 - 5.789260387096773
|
| 59 |
+
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® 8 Elite For Galaxy Mobile | 4096 | 10.326481 | 0.1718072580645161 - 5.497832258064515
|
| 60 |
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® X2 Elite | 4096 | 19.845977 | 0.12204629032258064 - 3.9054812903225806
|
| 61 |
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Snapdragon® X Elite | 4096 | 11.534744 | 0.19841499999999998 - 6.349279999999999
|
| 62 |
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-8275 | 4096 | 9.483059 | 0.219881 - 7.036192
|
| 63 |
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-9075 | 4096 | 10.134524 | 0.17698829032258065 - 5.663625290322581
|
| 64 |
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ IQ-X7181 | 4096 | 11.534744 | 0.19841499999999998 - 6.349279999999999
|
| 65 |
+
| Falcon3-7B-Instruct | GENIEX_QAIRT | w4a16 | Qualcomm® Dragonwing™ Q-8750 | 4096 | 10.326481 | 0.1718072580645161 - 5.497832258064515
|
| 66 |
|
| 67 |
## License
|
| 68 |
* The license for the original implementation of Falcon3-7B-Instruct can be found
|