Instructions to use raihan-js/orch-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use raihan-js/orch-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="raihan-js/orch-7b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("raihan-js/orch-7b")
model = AutoModelForCausalLM.from_pretrained("raihan-js/orch-7b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use raihan-js/orch-7b with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use raihan-js/orch-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "raihan-js/orch-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "raihan-js/orch-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/raihan-js/orch-7b

SGLang

How to use raihan-js/orch-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "raihan-js/orch-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "raihan-js/orch-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "raihan-js/orch-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "raihan-js/orch-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use raihan-js/orch-7b with Docker Model Runner:
```
docker model run hf.co/raihan-js/orch-7b
```

ORCH-7B

Orchestrated Recursive Code Hierarchy

QLoRA fine-tune of DeepSeek Coder 6.7B Instruct for autonomous Next.js code generation

TL;DR


Base model	`deepseek-ai/deepseek-coder-6.7b-instruct`
Fine-tuning method	QLoRA (4-bit NF4 quantization + double quant + LoRA adapters)
Hardware	Single NVIDIA A100
Training duration	43 hours
Training steps	5,238
Context length	16,384 tokens (linear RoPE scaling, 4×)
Parameter count (base)	6.7B
Format	Hugging Face Transformers (safetensors)

What this is

A QLoRA fine-tune of DeepSeek Coder 6.7B Instruct specialized for generating complete Next.js applications from natural language prompts. Where the from-scratch ORCH siblings explore the limits of pretraining small custom architectures, ORCH-7B takes the other approach: start from a strong code base model and specialize it cheaply via parameter-efficient fine-tuning.

ORCH-7B is the model powering the ORCH Studio Gradio Space — describe an application, get a downloadable Next.js 14 project ZIP.

Specialization

Framework: Next.js 14+ (App Router)
Language: TypeScript
Styling: Tailwind CSS
Database: Prisma ORM patterns
Auth: NextAuth.js patterns
Components: shadcn/ui compatible structure

Fine-tuning details


Quantization	4-bit NF4 (bitsandbytes)
Double quantization	yes
Compute dtype	bfloat16
Adapter	LoRA
RoPE scaling	linear, factor 4.0
RoPE theta	100,000
Hardware	NVIDIA A100
Duration	43 hours
Steps	5,238

The base config is preserved (LLaMA architecture, 32 layers, 4096 hidden, 32 heads, 32 KV); the fine-tune layers on top via LoRA adapters with 4-bit quantization to keep memory in check.

Usage

With Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "raihan-js/orch-7b"   # or use orch-ai/ORCH-7B mirror

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

prompt = """### Instruction:
Create a Next.js login page with email and password fields, validation, and error handling.

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, temperature=0.7, do_sample=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Easiest path: ORCH Studio

For non-developers, the ORCH Studio Gradio Space wraps this model with project templates and a ZIP-download workflow.

Intended use

Generating Next.js 14 application skeletons from natural language
Specialized completion for App Router + TypeScript + Tailwind code
A fine-tuned counterpart to the from-scratch ORCH series

Limitations

Domain-specialized: fine-tuned for Next.js. General-purpose code generation outside this domain will be weaker than the base DeepSeek model.
Training scope: 5,238 steps on a curated dataset — not a frontier-scale fine-tune.
No safety alignment beyond the base model. Treat outputs as untrusted code; review before deploying.
Linear RoPE 4× scaling extends usable context but is not a perfect substitute for native long-context training.

License

Released under the ORCH License v1.0 — see LICENSE in this repo. (DeepSeek Coder's base model license also applies.)

Related models

The from-scratch ORCH siblings (no base model, custom LLaMA architectures):

raihan-js/orch-fusion — 272M, tiny 2,103 vocab
raihan-js/orch-nextjs-350m-v2 — 287M, 16k vocab
raihan-js/orch-nextjs-3b — 3B, 32k vocab, 16K context

Plus the medical sibling:

And the Space that uses this model:

raihan-js/orch-studio — Gradio Space, autonomous Next.js generator

Author

Akteruzzaman Raihan Sikder — AI/ML engineer. Founding engineer and AI/ML lead at VETR Proposal; previously CTO of ClarioScope AI (2024–2026, sunset). Portfolio · GitHub.

Citation

@misc{sikder2026orch7b,
  title  = {ORCH-7B: A QLoRA Fine-Tune of DeepSeek Coder 6.7B for Autonomous Next.js Code Generation},
  author = {Sikder, Akteruzzaman Raihan},
  year   = {2026},
  url    = {https://huggingface.co/raihan-js/orch-7b}
}

Downloads last month: 3

Safetensors

Model size

7B params

Tensor type

F32

Model tree for raihan-js/orch-7b

Base model

deepseek-ai/deepseek-coder-6.7b-instruct

Adapter

(390)

this model