Instructions to use STEM-AI-mtl/phi-2-electrical-engineering with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use STEM-AI-mtl/phi-2-electrical-engineering with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="STEM-AI-mtl/phi-2-electrical-engineering", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("STEM-AI-mtl/phi-2-electrical-engineering", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("STEM-AI-mtl/phi-2-electrical-engineering", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use STEM-AI-mtl/phi-2-electrical-engineering with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "STEM-AI-mtl/phi-2-electrical-engineering"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "STEM-AI-mtl/phi-2-electrical-engineering",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/STEM-AI-mtl/phi-2-electrical-engineering

SGLang

How to use STEM-AI-mtl/phi-2-electrical-engineering with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "STEM-AI-mtl/phi-2-electrical-engineering" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "STEM-AI-mtl/phi-2-electrical-engineering",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "STEM-AI-mtl/phi-2-electrical-engineering" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "STEM-AI-mtl/phi-2-electrical-engineering",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use STEM-AI-mtl/phi-2-electrical-engineering with Docker Model Runner:
```
docker model run hf.co/STEM-AI-mtl/phi-2-electrical-engineering
```

phi-2-electrical-engineering / README.md

STEM-AI-mtl

Update README.md

9b32208 verified over 2 years ago

preview code

Raw

History Blame Contribute Delete

2.39 kB

	---
	license: other
	license_name: stem.ai.mtl
	license_link: LICENSE
	language:
	- en
	tags:
	- phi-2
	- electrical engineering
	- Microsoft
	datasets:
	- STEM-AI-mtl/Electrical-engineering
	- garage-bAInd/Open-Platypus
	task_categories:
	- question-answering
	- text-generation
	pipeline_tag: text-generation
	widget:
	- text: "Enter your instruction here"
	inference: true
	auto_sample: true
	inference_code: chat-GPTQ.py
	library_tag: transformers
	---
	# For the electrical engineering community

	A unique, deployable and efficient 2.7 billion parameters model in the field of electrical engineering. This repo contains the adapters from the LoRa fine-tuning of the phi-2 model from Microsoft. It was trained on the [STEM-AI-mtl/Electrical-engineering](https://huggingface.co/datasets/STEM-AI-mtl/Electrical-engineering) dataset combined with [garage-bAInd/Open-Platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus).

	- Developed by: STEM.AI
	- Model type: Q&A and code generation
	- Language(s) (NLP): English
	- Finetuned from model: [microsoft/phi-2](https://huggingface.co/microsoft/phi-2)


	### Direct Use

	Q&A related to electrical engineering, and Kicad software. Creation of Python code in general, and for Kicad's scripting console.

	Refer to [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) model card for recommended prompt format.

	### Inference script

	[Standard](https://github.com/STEM-ai/Phi-2/blob/4eaa6aaa2679427a810ace5a061b9c951942d66a/chat.py)

	[GPTQ format](https://github.com/STEM-ai/Phi-2/blob/ab1ced8d7922765344d824acf1924df99606b4fc/chat-GPTQ.py)

	## Training Details

	### Training Data

	Dataset related to electrical engineering: [STEM-AI-mtl/Electrical-engineering](https://huggingface.co/datasets/STEM-AI-mtl/Electrical-engineering)
	It is composed of queries, 65% about general electrical engineering, 25% about Kicad (EDA software) and 10% about Python code for Kicad's scripting console.

	In additionataset related to STEM and NLP: [garage-bAInd/Open-Platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus)

	### Training Procedure
	[LoRa script](https://github.com/STEM-ai/Phi-2/blob/4eaa6aaa2679427a810ace5a061b9c951942d66a/LoRa.py)

	A LoRa PEFT was performed on a 48 Gb A40 Nvidia GPU.

	## Model Card Authors

	STEM.AI: stem.ai.mtl@gmail.com\
	[William Harbec](https://www.linkedin.com/in/william-harbec-56a262248/)