Text Generation
Transformers
Safetensors
English
qwen3_5
image-text-to-text
user-representation
user-simulation
simulation
multi-turn
conversational
Instructions to use microsoft/UserRepGen-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/UserRepGen-4B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="microsoft/UserRepGen-4B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("microsoft/UserRepGen-4B") model = AutoModelForMultimodalLM.from_pretrained("microsoft/UserRepGen-4B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use microsoft/UserRepGen-4B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "microsoft/UserRepGen-4B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/UserRepGen-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/microsoft/UserRepGen-4B
- SGLang
How to use microsoft/UserRepGen-4B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "microsoft/UserRepGen-4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/UserRepGen-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "microsoft/UserRepGen-4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/UserRepGen-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use microsoft/UserRepGen-4B with Docker Model Runner:
docker model run hf.co/microsoft/UserRepGen-4B
Document verified end-to-end example output
Browse filesCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
README.md
CHANGED
|
@@ -178,6 +178,30 @@ The example uses greedy decoding for a simple smoke test, not the sampling
|
|
| 178 |
settings used for the paper's evaluations. Invalid JSON or missing/empty fields
|
| 179 |
raise an error rather than silently repairing the model's output.
|
| 180 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 181 |
### Inference requirements and checkpoint format
|
| 182 |
|
| 183 |
The reported inference setup uses Python, PyTorch, and a Qwen3.5-compatible
|
|
@@ -186,7 +210,9 @@ vLLM on NVIDIA H100, A100, RTX A6000, and B200 GPUs. Other hardware and operatin
|
|
| 186 |
systems were not evaluated in the research. A Linux CPU smoke test with
|
| 187 |
Transformers 5.2.0, PyTorch 2.11.0, and Accelerate 1.10.1 successfully loaded the
|
| 188 |
checkpoint and generated valid JSON for a local WildChat conversation. This is
|
| 189 |
-
a functional check, not a CPU performance benchmark.
|
|
|
|
|
|
|
| 190 |
float32 and requires substantially more memory than the approximately 9.1 GB
|
| 191 |
weight file. No optimized or device-specific formats are currently available.
|
| 192 |
|
|
|
|
| 178 |
settings used for the paper's evaluations. Invalid JSON or missing/empty fields
|
| 179 |
raise an error rather than silently repairing the model's output.
|
| 180 |
|
| 181 |
+
### Example output
|
| 182 |
+
|
| 183 |
+
The following is the actual output from the standalone example above, run on
|
| 184 |
+
CPU using the published Hugging Face checkpoint. The prompt contained 481
|
| 185 |
+
tokens and generation produced 274 tokens, including the end-of-sequence token.
|
| 186 |
+
Outputs may vary with runtime, hardware, or generation settings.
|
| 187 |
+
|
| 188 |
+
<details>
|
| 189 |
+
<summary>Show generated task and user behavior</summary>
|
| 190 |
+
|
| 191 |
+
```json
|
| 192 |
+
{
|
| 193 |
+
"task": "The user seeks an Excel formula to count dates within a specific range (A2:A100) that fall before a calculated threshold (today plus a value from B1), while handling edge cases where the source data is stored as text, contains empty cells, and requires a step-by-step solution involving a helper column.",
|
| 194 |
+
"user_behavior": "The user initiates with a direct, concise request for a specific formula but immediately pivots to troubleshooting when the initial solution fails, providing brief, factual feedback about the nature of the data (e.g., 'text', 'empty cells') rather than emotional reactions. They decompose the problem incrementally across turns, asking for specific implementation details like 'steps using a helper column' rather than requesting a complete solution upfront. Their communication style is terse and functional, avoiding pleasantries or verbose explanations. They do not provide extensive context initially but reveal constraints (like data types) only after the assistant proposes a solution that doesn't work. They engage in a multi-turn iterative process, refining the request based on the assistant's output, and do not terminate the conversation prematurely or ask for unrelated tasks. They prefer the assistant to handle the logic and provide the formula, but they will explicitly request step-by-step instructions if the assistant's output is too abstract or if they need to verify the logic manually."
|
| 195 |
+
}
|
| 196 |
+
```
|
| 197 |
+
|
| 198 |
+
</details>
|
| 199 |
+
|
| 200 |
+
This is an inferred description, not verified facts about a person. Even this
|
| 201 |
+
example includes task-specific details and interpretations that go beyond
|
| 202 |
+
directly observed behavior. Review representations before sharing or reusing
|
| 203 |
+
them, as described in the limitations below.
|
| 204 |
+
|
| 205 |
### Inference requirements and checkpoint format
|
| 206 |
|
| 207 |
The reported inference setup uses Python, PyTorch, and a Qwen3.5-compatible
|
|
|
|
| 210 |
systems were not evaluated in the research. A Linux CPU smoke test with
|
| 211 |
Transformers 5.2.0, PyTorch 2.11.0, and Accelerate 1.10.1 successfully loaded the
|
| 212 |
checkpoint and generated valid JSON for a local WildChat conversation. This is
|
| 213 |
+
a functional check, not a CPU performance benchmark. The published standalone
|
| 214 |
+
example was also tested end-to-end using the Hugging Face repository ID, rather
|
| 215 |
+
than a local model directory. CPU inference uses
|
| 216 |
float32 and requires substantially more memory than the approximately 9.1 GB
|
| 217 |
weight file. No optimized or device-specific formats are currently available.
|
| 218 |
|