scripts/model_examples/README.md

# Model Call Examples

This directory contains example scripts for the major LLM providers supported by AgentScope, together with a unified test runner `run_tests.py`.
These scripts are designed to verify that AgentScope's chat model components function correctly across various input scenarios.

---

## Directory Layout

```

scripts/model_examples/

├── run_tests.py                          # Unified test runner

├── _utils.py                             # Shared helpers (stream_and_collect)

├── test.jpeg                             # Sample image for multimodal tests

│

├── openai_chat_call.py             # OpenAI Chat Completions – basic + tool call + structured output

├── openai_chat_multiagent.py       # OpenAI Chat Completions – multi-agent conversation

├── openai_chat_multimodal.py       # OpenAI Chat Completions – image/text multimodal

├── openai_chat_multiagent_multimodal.py

│

├── openai_response_call.py         # OpenAI Responses API – reasoning models (o1/o3)

├── openai_response_multiagent.py

├── openai_response_multimodal.py

├── openai_response_multiagent_multimodal.py

│

├── anthropic_call.py               # Anthropic Claude

├── anthropic_multiagent.py

├── anthropic_multimodal.py

├── anthropic_multiagent_multimodal.py

│

├── dashscope_call.py               # Alibaba DashScope / Qwen

├── dashscope_multiagent.py

├── dashscope_multimodal.py

├── dashscope_multiagent_multimodal.py

│

├── deepseek_call.py                # DeepSeek (no multimodal support)

├── deepseek_multiagent.py

│

├── gemini_call.py                  # Google Gemini

├── gemini_multiagent.py

├── gemini_multimodal.py

├── gemini_multiagent_multimodal.py

│

├── moonshot_call.py                 # Moonshot AI (Kimi)

├── moonshot_multiagent.py

├── moonshot_multimodal.py

├── moonshot_multiagent_multimodal.py

│

├── xai_call.py                     # xAI Grok

├── xai_multiagent.py

├── xai_multimodal.py

├── xai_multiagent_multimodal.py

│

├── ollama_call.py                  # Ollama local models (requires a running server)

├── ollama_multiagent.py

├── ollama_multimodal.py

└── ollama_multiagent_multimodal.py

```

---

## Test Types

| Suffix | File Pattern | What it covers |
|---|---|---|
| `call` | `*_call.py` | Basic text call + two-round tool calling + structured output |
| `multiagent` | `*_multiagent.py` | Multi-agent scenario using `MultiAgentFormatter` |
| `multimodal` | `*_multimodal.py` | Image + text multimodal input (some providers also test audio/video) |
| `multiagent_multimodal` | `*_multiagent_multimodal.py` | Multi-agent + multimodal combined |

---

## Providers and Their Environment Variables

| Provider | Env Variable | Notes |
|---|---|---|
| `openai_chat` | `OPENAI_API_KEY` | Chat Completions API – gpt-4.1, etc. |
| `openai_response` | `OPENAI_API_KEY` | Responses API – o1, o3, o4-mini, etc. |
| `anthropic` | `ANTHROPIC_API_KEY` | Claude models, supports extended thinking |
| `dashscope` | `DASHSCOPE_API_KEY` | Qwen series, supports `thinking_enable` |
| `deepseek` | `DEEPSEEK_API_KEY` | Supports only `call` / `multiagent` (no multimodal) |
| `gemini` | `GEMINI_API_KEY` | Gemini models, supports `thinking_budget` |
| `moonshot` | `MOONSHOT_API_KEY` | Moonshot AI kimi-k2.6, etc. |
| `xai` | `XAI_API_KEY` | Grok models, supports `reasoning_effort` |
| `ollama` | *(none – auto-detect)* | Local server, default `http://localhost:11434` |

---

## Quick Start

### 1. Export API Keys

Set the environment variables for the providers you want to test:

```bash

export OPENAI_API_KEY="sk-..."

export ANTHROPIC_API_KEY="sk-ant-..."

export DASHSCOPE_API_KEY="sk-..."

export DEEPSEEK_API_KEY="sk-..."

export GEMINI_API_KEY="AIza..."

export MOONSHOT_API_KEY="sk-..."

export XAI_API_KEY="xai-..."

```

For Ollama, no API key is required. Just make sure the server is running:

```bash

ollama serve

ollama pull qwen3:14b   # pull the default model used in the scripts

```

### 2. Check Provider Availability

```bash

python scripts/model_examples/run_tests.py --list

```

Sample output:
```

  Provider               Env Var                   Available    Description

  openai_chat            OPENAI_API_KEY            YES          OpenAI Chat Completions API

  anthropic              ANTHROPIC_API_KEY          NO           Anthropic Claude models

  ...

```

### 3. Run All Available Tests

```bash

python scripts/model_examples/run_tests.py

```

The runner auto-detects which providers have credentials, skips those that do not, and runs all test types for the rest.

---

## `run_tests.py` Reference



```

usage: run_tests.py [-h] [--providers NAME[,NAME...]] [--tests TYPE[,TYPE...]]
                    [--timeout SECONDS] [--list] [--verbose]

```


### Options

| Option | Short | Default | Description |
|---|---|---|---|
| `--providers` | `-p` | all | Comma-separated list of providers to run |
| `--tests` | `-t` | all | Comma-separated list of test types to run |
| `--timeout` | | `120` | Per-script timeout in seconds |
| `--list` | `-l` | | Print provider status table and exit |
| `--verbose` | `-v` | | Stream each script's output in real time. By default output is suppressed and shown only when a test fails. |

### Examples

```bash

# Only test specific providers

python scripts/model_examples/run_tests.py --providers openai_chat,anthropic



# Only run a specific test type (across all available providers)

python scripts/model_examples/run_tests.py --tests call



# Combine: run call + multiagent tests for dashscope and deepseek

python scripts/model_examples/run_tests.py -p dashscope,deepseek -t call,multiagent



# Only run multimodal tests

python scripts/model_examples/run_tests.py --tests multimodal,multiagent_multimodal



# Increase per-script timeout

python scripts/model_examples/run_tests.py --timeout 180



# Check provider status

python scripts/model_examples/run_tests.py --list

```

### Summary Table

At the end of a run, a summary table is printed:

```

  Provider               Test Type                    Status      Time

  ---------------------- ---------------------------- -------- -------

  openai_chat            call                         PASS       12.3s

  openai_chat            multiagent                   PASS        8.1s

  anthropic              call                         SKIP      (env var ANTHROPIC_API_KEY not set)

  deepseek               call                         PASS       15.7s

  deepseek               multimodal                   SKIP      (not supported)



  Total: 12  |  PASS:   8  |  FAIL:   0  |  SKIP:   4

```

| Status | Meaning |
|---|---|
| **PASS** | Script exited with code 0 |
| **FAIL** | Script exited with a non-zero code or timed out |
| **SKIP** | API key missing, test type not supported, or script file absent |

The runner exits with code `1` if any test fails.

---

## Running a Single Script

Every script can be executed independently once the relevant environment variable is set:

```bash

python scripts/model_examples/openai_chat_call.py

python scripts/model_examples/dashscope_multiagent.py

python scripts/model_examples/ollama_multimodal.py

```

Each script typically defines two or more async functions:

- `example_simple_call()` – basic text call with streaming
- `example_tool_call()` – two-round conversation with tool/function calling
- `example_structured_output()` – force a Pydantic-validated JSON output (in `_call.py` variants, uses a thinking-enabled model)
- `example_image_url()` / `example_image_local_path()` / `example_image_base64()` – image + text input (in `_multimodal.py` variants)
- `example_audio()` – audio input (e.g. `openai_chat_multimodal.py`, `dashscope_multimodal.py`)
- `example_video()` – video input (e.g. `dashscope_multimodal.py`)

---

## Ollama Notes

Ollama runs locally and requires no API key, but you must:

1. Start the service: `ollama serve`
2. Pull the model used by the scripts: `ollama pull qwen3:14b`
3. If the service runs on a non-default address, set: `export OLLAMA_HOST=http://your-host:11434`

`run_tests.py` pings the Ollama host before running any test. If the server is unreachable, all Ollama tests are automatically skipped.