Instructions to use Salesforce/codegen-350M-multi with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Salesforce/codegen-350M-multi with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Salesforce/codegen-350M-multi")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen-350M-multi") model = AutoModelForCausalLM.from_pretrained("Salesforce/codegen-350M-multi", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Salesforce/codegen-350M-multi with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Salesforce/codegen-350M-multi" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Salesforce/codegen-350M-multi", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Salesforce/codegen-350M-multi
- SGLang
How to use Salesforce/codegen-350M-multi with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Salesforce/codegen-350M-multi" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Salesforce/codegen-350M-multi", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Salesforce/codegen-350M-multi" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Salesforce/codegen-350M-multi", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Salesforce/codegen-350M-multi with Docker Model Runner:
docker model run hf.co/Salesforce/codegen-350M-multi
| import gradio as gr | |
| from transformers import pipeline | |
| # 🔥 Program Synthesis Modell | |
| synthesizer = pipeline("text-generation", model="microsoft/CodeGPT-small-py") | |
| # 🔹 Gravatar | |
| GRAVATAR_URL = "https://www.gravatar.com/avatar/7e6d02f7b39c0f35f7eae2f404a7d0b1?s=200" | |
| GRAVATAR_LINK = "https://gravatar.com/skymeilin" | |
| # 🔹 Finger Funktionen | |
| def code_analysis(prompt): | |
| return synthesizer(f"# Analysiere den Code:\n{prompt}\n# Analyse:", max_length=300)[0]['generated_text'] | |
| def code_optimization(prompt): | |
| return synthesizer(f"# Optimiere den folgenden Code:\n{prompt}\n# Optimierter Code:", max_length=300)[0]['generated_text'] | |
| def code_comment(prompt): | |
| return synthesizer(f"# Kommentiere den Code:\n{prompt}\n# Kommentar:", max_length=300)[0]['generated_text'] | |
| def code_multilang(prompt): | |
| return synthesizer(f"# Übersetze oder generiere in gewünschter Sprache:\n{prompt}\n# Code:", max_length=300)[0]['generated_text'] | |
| def code_debug(prompt): | |
| return synthesizer(f"# Finde Bugs und Vorschläge:\n{prompt}\n# Debug:", max_length=300)[0]['generated_text'] | |
| def code_test(prompt): | |
| return synthesizer(f"# Generiere Testfälle für:\n{prompt}\n# Tests:", max_length=300)[0]['generated_text'] | |
| def code_refactor(prompt): | |
| return synthesizer(f"# Refactore den Code nach Best Practices:\n{prompt}\n# Refactored Code:", max_length=300)[0]['generated_text'] | |
| def code_doc(prompt): | |
| return synthesizer(f"# Dokumentiere den Code:\n{prompt}\n# Dokumentation:", max_length=300)[0]['generated_text'] | |
| def code_boilerplate(prompt): | |
| return synthesizer(f"# Generiere Boilerplate / Deployment Code:\n{prompt}\n# Code:", max_length=300)[0]['generated_text'] | |
| def code_custom(prompt): | |
| return synthesizer(f"# Eigene Conversational Anfrage:\n{prompt}\n# Antwort:", max_length=300)[0]['generated_text'] | |
| # Mapping Finger-Buttons | |
| fingers = { | |
| "Analyse": code_analysis, | |
| "Optimierung": code_optimization, | |
| "Kommentierung": code_comment, | |
| "Mehrsprachig": code_multilang, | |
| "Debugging": code_debug, | |
| "Testfälle": code_test, | |
| "Refactoring": code_refactor, | |
| "Dokumentation": code_doc, | |
| "Boilerplate": code_boilerplate, | |
| "Conversational": code_custom | |
| } | |
| # Gradio UI | |
| with gr.Blocks() as app: | |
| # Header | |
| with gr.Row(): | |
| gr.Image(GRAVATAR_URL, shape=(100,100), tooltips="Sky Meilin", interactive=True) | |
| gr.Markdown(f"## 🔥 Anycoder 30de8a5b – Conversational Program Synthesis\n**Sky Meilin** – [Gravatar Profil]({GRAVATAR_LINK})") | |
| # Input | |
| prompt_input = gr.Textbox(label="Beschreibung oder Code eingeben", placeholder="Z.B. 'Sortiere eine Liste...'", lines=8) | |
| # Finger Buttons | |
| output_code = gr.Textbox(label="Ergebnis", lines=10) | |
| with gr.Row(): | |
| for name, func in fingers.items(): | |
| gr.Button(name).click(func, inputs=prompt_input, outputs=output_code) | |
| if __name__ == "__main__": | |
| app.launch() | |