Instructions to use Mayankm1411/GENAI_CAT2_STORYTELLING with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Mayankm1411/GENAI_CAT2_STORYTELLING with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Mayankm1411/GENAI_CAT2_STORYTELLING")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Mayankm1411/GENAI_CAT2_STORYTELLING")
model = AutoModelForCausalLM.from_pretrained("Mayankm1411/GENAI_CAT2_STORYTELLING", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Mayankm1411/GENAI_CAT2_STORYTELLING with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Mayankm1411/GENAI_CAT2_STORYTELLING"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Mayankm1411/GENAI_CAT2_STORYTELLING",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Mayankm1411/GENAI_CAT2_STORYTELLING

SGLang

How to use Mayankm1411/GENAI_CAT2_STORYTELLING with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Mayankm1411/GENAI_CAT2_STORYTELLING" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Mayankm1411/GENAI_CAT2_STORYTELLING",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Mayankm1411/GENAI_CAT2_STORYTELLING" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Mayankm1411/GENAI_CAT2_STORYTELLING",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Mayankm1411/GENAI_CAT2_STORYTELLING with Docker Model Runner:
```
docker model run hf.co/Mayankm1411/GENAI_CAT2_STORYTELLING
```

GENAI_CAT2_STORYTELLING / vae /config.json

Mayankm1411

Upload StableDiffusionPipeline

4d0f1dd verified about 1 year ago

Raw History Blame Contribute Delete

910 Bytes

{
  "_class_name": "AutoencoderKL",
  "_diffusers_version": "0.35.1",
  "_name_or_path": "/root/.cache/huggingface/hub/models--runwayml--stable-diffusion-v1-5/snapshots/451f4fe16113bff5a5d2269ed5ad43b0592e9a14/vae",
  "act_fn": "silu",
  "block_out_channels": [
    128,
    256,
    512,
    512
  ],
  "down_block_types": [
    "DownEncoderBlock2D",
    "DownEncoderBlock2D",
    "DownEncoderBlock2D",
    "DownEncoderBlock2D"
  ],
  "force_upcast": true,
  "in_channels": 3,
  "latent_channels": 4,
  "latents_mean": null,
  "latents_std": null,
  "layers_per_block": 2,
  "mid_block_add_attention": true,
  "norm_num_groups": 32,
  "out_channels": 3,
  "sample_size": 512,
  "scaling_factor": 0.18215,
  "shift_factor": null,
  "up_block_types": [
    "UpDecoderBlock2D",
    "UpDecoderBlock2D",
    "UpDecoderBlock2D",
    "UpDecoderBlock2D"
  ],
  "use_post_quant_conv": true,
  "use_quant_conv": true
}

	{
	"_class_name": "AutoencoderKL",
	"_diffusers_version": "0.35.1",
	"_name_or_path": "/root/.cache/huggingface/hub/models--runwayml--stable-diffusion-v1-5/snapshots/451f4fe16113bff5a5d2269ed5ad43b0592e9a14/vae",
	"act_fn": "silu",
	"block_out_channels": [
	128,
	256,
	512,
	512
	],
	"down_block_types": [
	"DownEncoderBlock2D",
	"DownEncoderBlock2D",
	"DownEncoderBlock2D",
	"DownEncoderBlock2D"
	],
	"force_upcast": true,
	"in_channels": 3,
	"latent_channels": 4,
	"latents_mean": null,
	"latents_std": null,
	"layers_per_block": 2,
	"mid_block_add_attention": true,
	"norm_num_groups": 32,
	"out_channels": 3,
	"sample_size": 512,
	"scaling_factor": 0.18215,
	"shift_factor": null,
	"up_block_types": [
	"UpDecoderBlock2D",
	"UpDecoderBlock2D",
	"UpDecoderBlock2D",
	"UpDecoderBlock2D"
	],
	"use_post_quant_conv": true,
	"use_quant_conv": true
	}