| --- |
| library_name: None |
| tags: |
| - pruna-ai |
| --- |
| |
| # Model Card for hnpinq/Quantized_CL |
| |
| This model was created using the [pruna](https://github.com/PrunaAI/pruna) library. Pruna is a model optimization framework built for developers, enabling you to deliver more efficient models with minimal implementation overhead. |
| |
| ## Usage |
| |
| First things first, you need to install the pruna library: |
| |
| ```bash |
| pip install pruna |
| ``` |
| |
| You can [use the None library to load the model](https://huggingface.co/hnpinq/Quantized_CL?library=None) but this might not include all optimizations by default. |
| |
| To ensure that all optimizations are applied, use the pruna library to load the model using the following code: |
| |
| ```python |
| from pruna import PrunaModel |
| |
| loaded_model = PrunaModel.from_hub( |
| "hnpinq/Quantized_CL" |
| ) |
| ``` |
| |
| After loading the model, you can use the inference methods of the original model. Take a look at the [documentation](https://pruna.readthedocs.io/en/latest/index.html) for more usage information. |
| |
| ## Smash Configuration |
| |
| The compression configuration of the model is stored in the `smash_config.json` file, which describes the optimization methods that were applied to the model. |
| |
| ```bash |
| { |
| "batcher": null, |
| "cacher": null, |
| "compiler": "torch_compile", |
| "factorizer": null, |
| "pruner": null, |
| "quantizer": "hqq_diffusers", |
| "hqq_diffusers_backend": "torchao_int4", |
| "hqq_diffusers_group_size": 64, |
| "hqq_diffusers_weight_bits": 8, |
| "torch_compile_backend": "inductor", |
| "torch_compile_dynamic": null, |
| "torch_compile_fullgraph": true, |
| "torch_compile_make_portable": false, |
| "torch_compile_max_kv_cache_size": 400, |
| "torch_compile_mode": "max-autotune", |
| "torch_compile_seqlen_manual_cuda_graph": 100, |
| "torch_compile_target": "model", |
| "batch_size": 1, |
| "device": "cuda", |
| "save_fns": [ |
| "hqq_diffusers", |
| "save_before_apply" |
| ], |
| "load_fns": [ |
| "hqq_diffusers" |
| ], |
| "reapply_after_load": { |
| "factorizer": null, |
| "pruner": null, |
| "quantizer": null, |
| "cacher": null, |
| "compiler": "torch_compile", |
| "batcher": null |
| } |
| } |
| ``` |
| |
| ## 🌍 Join the Pruna AI community! |
|
|
| [](https://twitter.com/PrunaAI) |
| [](https://github.com/PrunaAI) |
| [](https://www.linkedin.com/company/93832878/admin/feed/posts/?feedType=following) |
| [](https://discord.com/invite/rskEr4BZJx) |
| [](https://www.reddit.com/r/PrunaAI/) |
