aokit-tests/Model-cpu
Updated
AI & ML interests
None defined yet.
cbensimon
updated a
model 4 months ago
cbensimon
published a
model 4 months ago
cbensimon
updated a
model 4 months ago
cbensimon
published a
model 4 months ago
cbensimon
updated a
model 4 months ago
cbensimon
published a
model 4 months ago
Post
5441
🚀 ZeroGPU now supports PyTorch native quantization via
While it hasn’t been battle-tested yet,
Let us know if you run into any issues — and we’re excited to see what the community will build!
torchaoWhile it hasn’t been battle-tested yet,
Int8WeightOnlyConfig is already working flawlessly in our tests.Let us know if you run into any issues — and we’re excited to see what the community will build!
import spaces
from diffusers import FluxPipeline
from torchao.quantization.quant_api import Int8WeightOnlyConfig, quantize_
pipeline = FluxPipeline.from_pretrained(...).to('cuda')
quantize_(pipeline.transformer, Int8WeightOnlyConfig()) # Or any other component(s)
@spaces.GPU
def generate(prompt: str):
return pipeline(prompt).images[0]
- 5 replies
Post
6237
🚀 ZeroGPU
Nothing too fancy for now—ZeroGPU Spaces still default to
- 💰 size-based quotas / pricing (
- 🦣 the upcoming
You can as of now control GPU size via a Space variable. Accepted values:
-
-
-
The auto mode checks total CUDA tensor size during startup:
- More than 30GB →
- Otherwise →
medium size is now available as a power-user featureNothing too fancy for now—ZeroGPU Spaces still default to
large (70GB VRAM)—but this paves the way for:- 💰 size-based quotas / pricing (
medium will offer significantly more usage than large)- 🦣 the upcoming
xlarge size (141GB VRAM)You can as of now control GPU size via a Space variable. Accepted values:
-
auto (future default)-
medium-
large (current default)The auto mode checks total CUDA tensor size during startup:
- More than 30GB →
large- Otherwise →
medium 
- 3 replies
Post
4810
Hello everybody,
We've rolled out a major update to ZeroGPU! All the Spaces are now running on it.
Major improvements:
1. GPU cold starts about twice as fast!
2. RAM usage reduced by two-thirds, allowing more effective resource usage, meaning more GPUs for the community!
3. ZeroGPU initializations (coldstarts) can now be tracked and displayed (use
4. Improved compatibility and PyTorch integration, increasing ZeroGPU compatible spaces without requiring any modifications!
Feel free to answer in the post if you have any questions
🤗 Best regards,
Charles
We've rolled out a major update to ZeroGPU! All the Spaces are now running on it.
Major improvements:
1. GPU cold starts about twice as fast!
2. RAM usage reduced by two-thirds, allowing more effective resource usage, meaning more GPUs for the community!
3. ZeroGPU initializations (coldstarts) can now be tracked and displayed (use
progress=gr.Progress(track_tqdm=True))4. Improved compatibility and PyTorch integration, increasing ZeroGPU compatible spaces without requiring any modifications!
Feel free to answer in the post if you have any questions
🤗 Best regards,
Charles