Instructions to use kavinduc/devops-mastermind with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kavinduc/devops-mastermind with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kavinduc/devops-mastermind")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kavinduc/devops-mastermind")
model = AutoModelForCausalLM.from_pretrained("kavinduc/devops-mastermind", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kavinduc/devops-mastermind with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kavinduc/devops-mastermind"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kavinduc/devops-mastermind",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/kavinduc/devops-mastermind

SGLang

How to use kavinduc/devops-mastermind with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kavinduc/devops-mastermind" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kavinduc/devops-mastermind",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kavinduc/devops-mastermind" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kavinduc/devops-mastermind",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use kavinduc/devops-mastermind with Docker Model Runner:
```
docker model run hf.co/kavinduc/devops-mastermind
```

devops-mastermind / README.md

kavinduc

Update README.md

a89232b verified over 1 year ago

preview code

Raw

History Blame Contribute Delete

1.79 kB

metadata

license: mit
datasets:
  - adeeshajayasinghe/devops-guide-demo
metrics:
  - accuracy
base_model:
  - microsoft/phi-2
new_version: microsoft/phi-2
pipeline_tag: text-generation
library_name: transformers
tags:
  - code
  - text-generation-inference

DevOps Mastermind Model

This repository hosts the DevOps Mastermind model, a pre-trained model based on microsoft/phi-2 with modifications tailored for specialized DevOps knowledge tasks. The model is designed to support various downstream tasks, such as code generation, documentation assistance, and knowledge inference in DevOps domains.

Model Details

Base Model: microsoft/phi-2
Purpose: Enhanced with additional training and modifications for DevOps and software engineering contexts.
Files Included:
- config.json: Model configuration.
- pytorch_model.bin: The primary model file containing weights.
- tokenizer.json: Tokenizer for processing text inputs.
- added_tokens.json: Additional tokens specific to DevOps vocabulary.
- generation_config.json: Generation configuration for text generation tasks.
- Other auxiliary files required for model usage and compatibility.

Usage

To load and use this model in your code, run the following commands:

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model_name = "kavinduc/devops-mastermind"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False)

# Example usage
input_text = "Explain how to set up a CI/CD pipeline"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text)