gump2049

docs(brand): unify APUS Reflex naming across model cards and references

5575636 verified 13 days ago

13.5 kB

# APUS TypeSafe SDK gateway with explicit vLLM low/high routing

This directory adds the **official `typesafe-sdk==0.7.0` request/response protocol** to APUS Reflex. The local service is an APUS model, not the hosted Jev service. Its probabilities, calibration, quality, and billing do not inherit the hosted service's guarantees.

Two separately loaded vLLM engines serve the same trained checkpoint at different layer depths:

| Effort | Engine | Execution |
| --- | --- | --- |
| `high` (default) | Original merged `9B-5949` | 32 layers |
| `low` | Verified prefix export of that same checkpoint | 16 layers, original final norm and LM head |

This is **explicit request routing between two engines**. It is not automatic confidence-based early exit, shared weights between processes, hidden-state continuation, KV broadcast, or candidate-only LM-head execution. Both vLLM engines compute the full output head; the gateway extracts the requested candidate scores. A client-requested `low` never silently falls back to `high`.

## 1. Install and download the model

Use Linux, Python 3.12, CUDA-capable hardware, and vLLM 0.29.0. The 9B deployment work targets one RTX PRO 6000 Blackwell 96 GB; other hardware needs its own memory and numerical checks. Do not run the reference native runtime's Transformers 5.16.1 requirements in this vLLM environment.

```bash
python3.12 -m venv .venv
source .venv/bin/activate
pip install vllm==0.29.0 transformers==5.17.0 huggingface-hub==1.32.0 ninja==1.13.2
pip install fastapi==0.128.8 uvicorn==0.40.0 httpx==0.28.1 httpx2==2.13.0 typesafe-sdk==0.7.0
```

On Blackwell, configure the compatible CUDA 13 compiler in the same environment:

```bash
pip install nvidia-cuda-nvcc==13.4.92 nvidia-cuda-crt==13.4.92 nvidia-cuda-cccl==13.3.4.3.1
export CUDA_HOME="$(python -c 'import sysconfig; print(sysconfig.get_path("purelib") + "/nvidia/cu13")')"
export PATH="$CUDA_HOME/bin:$PATH"
"$CUDA_HOME/bin/nvcc" --version
```

The launch script disables the FlashInfer sampler to match the tested configuration. After downloading this directory, `pip install -r requirements.txt` installs the pinned deployment dependencies. `requirements-test.txt` supports the CPU gateway tests without installing a GPU runtime.

Resolve the repository revision once, then download by that immutable revision. The repository is a family bundle: use the **model subdirectory**, not its root.

```bash
export APUS_REVISION="$(python -c 'from huggingface_hub import HfApi; print(HfApi().model_info("apus-ailab/APUS-Reflex").sha)')"
hf download apus-ailab/APUS-Reflex --revision "$APUS_REVISION" \
  --include '9B-5949/*' 'deployment/*' --local-dir ./APUS-Reflex
cd ./APUS-Reflex/deployment
export APUS_FULL_MODEL="$(cd ../9B-5949 && pwd)"
```

If you already downloaded the deployment code separately, the included model-only helper performs the same immutable-revision resolution and prints the resulting model path:

```bash
python download_model.py --local-dir ../models --revision "$APUS_REVISION"
# With this layout, set APUS_FULL_MODEL to ../models/9B-5949.
```

Create a **new separate directory** for the trained 16-layer prefix. The exporter refuses an existing destination and verifies retained tensors; it never modifies the original weights.

```bash
export APUS_LOW_MODEL="$(dirname "$APUS_FULL_MODEL")/9B-5949-low16"
python export_low.py --source "$APUS_FULL_MODEL" --output "$APUS_LOW_MODEL"
```

Start the full engine in terminal A, using an otherwise available GPU:

```bash
export JEV_VLLM_PYTHON="$VIRTUAL_ENV/bin/python"
CUDA_VISIBLE_DEVICES=0 JEV_EFFORT=high \
  JEV_MODEL_PATH="$APUS_FULL_MODEL" JEV_ENGINE_PORT=30002 \
  JEV_GPU_MEMORY_UTILIZATION=0.70 JEV_MAX_NUM_SEQS=8 \
  bash run_backend.sh
```

In terminal B, activate the same environment, set the same `CUDA_HOME` and `APUS_LOW_MODEL` paths, and start the shallow engine:

```bash
export JEV_VLLM_PYTHON="$VIRTUAL_ENV/bin/python"
CUDA_VISIBLE_DEVICES=0 JEV_EFFORT=low \
  JEV_MODEL_PATH="$APUS_LOW_MODEL" JEV_ENGINE_PORT=30004 \
  JEV_GPU_MEMORY_UTILIZATION=0.22 JEV_MAX_NUM_SEQS=8 \
  bash run_backend.sh
```

These defaults allocate separate engine budgets of 70% and 22% of the same 96 GB GPU. They are not appropriate if another workload already occupies that GPU. Compilation can take several minutes. The launcher verifies metadata and weight hashes before starting; this adds sequential disk reads. In terminal C, check both engines:

```bash
curl -f http://127.0.0.1:30002/health
curl -f http://127.0.0.1:30004/health
```

Do not proceed until both are ready. Each foreground process can be stopped with Ctrl-C in its own terminal. Never reuse a port or GPU allocation belonging to another workload. The two engines must belong to the same source checkpoint; archive the export provenance and source revision together. The package uses `max_num_seqs=8` per engine; historical high-only measurements with 32 sequences are a different serving configuration and must not be copied as this setup's result.

## 2. Start the SDK gateway

```bash
export JEV_MODEL_PATH="$APUS_FULL_MODEL"
export JEV_HIGH_MODEL_PATH="$APUS_FULL_MODEL"
export JEV_LOW_MODEL_PATH="$APUS_LOW_MODEL"
export JEV_HIGH_URL=http://127.0.0.1:30002
export JEV_LOW_URL=http://127.0.0.1:30004
export JEV_HIGH_MODEL=apus-openjev-9b-high32
export JEV_LOW_MODEL=apus-openjev-9b-low16
export JEV_MODEL_NAME=APUS-OpenJev-v1-9B
# Supply your own local gateway secret; this is NOT your hosted TypeSafe API key.
export JEV_API_KEY='replace-with-your-local-gateway-secret'
python gateway.py --host 127.0.0.1 --port 30013
```

Startup requires both model directories on the same filesystem as the gateway. It checks 32/16-layer configs, merged indexes, completed low-export provenance, source/export metadata hashes, and each `/v1/models` alias/root against its configured artifact directory. Mismatches fail startup. The tokenizer must come from the full model directory. Engine URLs and aliases must be distinct. Depth headers report the selected verified backend configuration, not per-request layer-hook telemetry; live numerical evaluation remains necessary.

The gateway binds to localhost by default. Configure an authenticated HTTPS proxy before exposing it publicly. Backend URLs are operator configuration, never accepted from incoming requests. Both backend connections and the SDK client can reuse HTTP connections.

## 3. Use the official SDK against APUS

Use the **API root** as `base_url` — do not append `/v1`. The SDK appends `/v1/systemone` and `/v1/models` itself.

```python
import os
from typesafe_sdk import Choice, Noul, RetryPolicy, TypeSafeClient
with TypeSafeClient(
    base_url="http://127.0.0.1:30013",
    api_key=os.environ["JEV_API_KEY"],
    model="APUS-OpenJev-v1-9B",
    retry=RetryPolicy(max_retries=0),
) as client:
    print(client.models.list())
    result = client.system_one(
        state={
            "message": "My parcel arrived damaged yesterday. Can I get a refund?",
            "policy": "Damaged items reported within 30 days qualify for a refund.",
        },
        questions={
            "queue": Choice(
                instructions="Choose the appropriate customer-support queue.",
                criteria={"returns": "Returns and refunds", "billing": "Payments and invoices"},
            ),
            "eligible": Noul(instructions="Is the customer eligible under this refund policy?"),
        },
        extra_body={"effort": "low"},
    )
    print(result.answers["queue"].choice)
    print(result.answers["eligible"].noul)
    print(result.raw_http_response.headers["x-openjev-executed-layers"])
```

Use `extra_body={"effort": "high"}` for the full model, or omit it because high is the default. **`effort` is an APUS extension**, not a promise that the hosted TypeSafe API accepts the same field.

Run the complete English customer-support example (Choice + Noul + ordinal Score):

```bash
python sdk_example.py --effort high
python sdk_example.py --effort low
```

Async uses the same fields:

```python
import asyncio
import os
from typesafe_sdk import AsyncTypeSafeClient, Noul, RetryPolicy
async def main():
    async with AsyncTypeSafeClient(
        base_url="http://127.0.0.1:30013",
        api_key=os.environ["JEV_API_KEY"],
        model="APUS-OpenJev-v1-9B",
        retry=RetryPolicy(max_retries=0),
    ) as client:
        result = await client.system_one(
            state="The customer received a damaged item yesterday.",
            questions={"damaged": Noul(instructions="Did the customer report a damaged item?")},
            extra_body={"effort": "high"},
        )
        print(result.answers["damaged"].noul)
asyncio.run(main())
```

## 4. Use the official hosted TypeSafe service separately

This snippet sends a real request to the hosted service and can incur charges. Keep its credentials separate from the local gateway secret. It deliberately does not send the APUS `effort` extension.

```python
import os
from typesafe_sdk import Choice, RetryPolicy, TypeSafeClient
with TypeSafeClient(
    base_url="https://api.typesafe.ai",
    api_key=os.environ["TYPESAFE_API_KEY"],
    model="jev-latest",
    retry=RetryPolicy(max_retries=0),
) as client:
    result = client.system_one(
        state="The customer was charged twice for the same order.",
        questions={"queue": Choice(
            instructions="Choose the appropriate customer-support queue.",
            criteria={"billing": "Payments and invoices", "shipping": "Delivery and tracking"},
        )},
    )
    print(result.answers["queue"].choice)
```

## Contract and limits

- Supported SDK 0.7.0 methods: synchronous/asynchronous `system_one` and `models.list`. This SDK version exposes no `models.retrieve` method.
- `GET /v1/models` lists the APUS model and `jev-latest` as an explicitly labeled **local compatibility alias**. Responses name the actual APUS model.
- `POST /v1/systemone` accepts `state`, `questions`, `model`, and the optional APUS `effort` extension only. Unknown fields, invalid effort, unsupported models, malformed or duplicate-key JSON fail explicitly.
- State: string / object / array. Instructions: string / object / array / null. Choice descriptions also allow null; null uses the candidate name. JSON is serialized deterministically; ordinary string descriptions remain unchanged.
- Choice: 2–16 candidates. Noul: yes/no; optional true/false descriptions affect the question's meaning. Canonical true/false descriptions retain the original binary prompt. Custom descriptions are included explicitly.
- Score: 1–10 ordered levels, selected by their descriptions; returns the expected zero-based index, probabilities, and the original legend. This differs from the trained binary `score_level` task and has **not established ordinal-score quality or calibration**. A single-level Score has only one possible result: score 0, probability 1, no GPU request, zero model tokens, and executed-layers header 0. Its confidence 1 denotes a degenerate distribution, not evidence that the rubric is objectively correct.
- At most 16 questions, 1 MiB request body, and 8,191 compiled prompt tokens per question (one output token reserved). Inputs are never silently truncated. Maximum 32 in-flight requests, with a 120-second request deadline.
- Each question is compiled and scored separately. Usage sums the processed prompt tokens and one scoring token per question; `billing_units` means local decision count, not official monetary billing.
- Confidence is normalized distribution concentration. Candidate probabilities are uncalibrated conditional softmax; neither quantity inherits official calibration semantics.
- Response shape uses `model`, `answers`, `usage`; Choice has `choice/confidence/probabilities`, Noul has `noul`, Score has `score/confidence/probabilities/legend`.
- Headers `x-openjev-effort`, `x-openjev-executed-layers`, and `x-openjev-engine-model` record the chosen route. `x-typesafe-request-id` is a local request identifier.

## Verification status

CPU integration tests use the **real official SDK 0.7.0**, both sync and async, through an ASGI gateway with a deterministic mocked inference backend. They verify protocol, routing, payload validation, and numerical response assembly; they are not GPU quality or performance measurements.

```bash
pip install -r requirements-test.txt
python -m pytest -q tests/test_typesafe_gateway.py
```

Before reporting GPU performance, record immutable model/export identities, prove 16/32-layer engine admission, run the same frozen requests through both routes, retain raw responses and collection timestamps, and compare against a reference using **the same compiled prompts**. In particular, a historic `score_level` record mapped to TypeSafe Noul changes its prompt primitive; do not silently compare it as byte-identical input.

Run a real service smoke test and preserve its raw requests, responses, UTC timestamps, and per-call latency:

```bash
python smoke_sdk.py --base-url http://127.0.0.1:30013 --output ./sdk-smoke-new-run
```

This sends five HTTP calls: models, high multi-question, low multi-question, an expected 422 for invalid effort, and an async low request. The two multi-question requests include all three primitives. Use a new output directory for every run. No authentication headers are saved.

Deployment code retains the included Apache 2.0 license and source notices. Model weights retain their applicable upstream licenses.

	# APUS TypeSafe SDK gateway with explicit vLLM low/high routing

	This directory adds the official `typesafe-sdk==0.7.0` request/response protocol to APUS Reflex. The local service is an APUS model, not the hosted Jev service. Its probabilities, calibration, quality, and billing do not inherit the hosted service's guarantees.

	Two separately loaded vLLM engines serve the same trained checkpoint at different layer depths:

	\| Effort \| Engine \| Execution \|
	\| --- \| --- \| --- \|
	\| `high` (default) \| Original merged `9B-5949` \| 32 layers \|
	\| `low` \| Verified prefix export of that same checkpoint \| 16 layers, original final norm and LM head \|

	This is explicit request routing between two engines. It is not automatic confidence-based early exit, shared weights between processes, hidden-state continuation, KV broadcast, or candidate-only LM-head execution. Both vLLM engines compute the full output head; the gateway extracts the requested candidate scores. A client-requested `low` never silently falls back to `high`.

	## 1. Install and download the model

	Use Linux, Python 3.12, CUDA-capable hardware, and vLLM 0.29.0. The 9B deployment work targets one RTX PRO 6000 Blackwell 96 GB; other hardware needs its own memory and numerical checks. Do not run the reference native runtime's Transformers 5.16.1 requirements in this vLLM environment.

	```bash
	python3.12 -m venv .venv
	source .venv/bin/activate
	pip install vllm==0.29.0 transformers==5.17.0 huggingface-hub==1.32.0 ninja==1.13.2
	pip install fastapi==0.128.8 uvicorn==0.40.0 httpx==0.28.1 httpx2==2.13.0 typesafe-sdk==0.7.0
	```

	On Blackwell, configure the compatible CUDA 13 compiler in the same environment:

	```bash
	pip install nvidia-cuda-nvcc==13.4.92 nvidia-cuda-crt==13.4.92 nvidia-cuda-cccl==13.3.4.3.1
	export CUDA_HOME="$(python -c 'import sysconfig; print(sysconfig.get_path("purelib") + "/nvidia/cu13")')"
	export PATH="$CUDA_HOME/bin:$PATH"
	"$CUDA_HOME/bin/nvcc" --version
	```

	The launch script disables the FlashInfer sampler to match the tested configuration. After downloading this directory, `pip install -r requirements.txt` installs the pinned deployment dependencies. `requirements-test.txt` supports the CPU gateway tests without installing a GPU runtime.

	Resolve the repository revision once, then download by that immutable revision. The repository is a family bundle: use the model subdirectory, not its root.

	```bash
	export APUS_REVISION="$(python -c 'from huggingface_hub import HfApi; print(HfApi().model_info("apus-ailab/APUS-Reflex").sha)')"
	hf download apus-ailab/APUS-Reflex --revision "$APUS_REVISION" \
	--include '9B-5949/' 'deployment/' --local-dir ./APUS-Reflex
	cd ./APUS-Reflex/deployment
	export APUS_FULL_MODEL="$(cd ../9B-5949 && pwd)"
	```

	If you already downloaded the deployment code separately, the included model-only helper performs the same immutable-revision resolution and prints the resulting model path:

	```bash
	python download_model.py --local-dir ../models --revision "$APUS_REVISION"
	# With this layout, set APUS_FULL_MODEL to ../models/9B-5949.
	```

	Create a new separate directory for the trained 16-layer prefix. The exporter refuses an existing destination and verifies retained tensors; it never modifies the original weights.

	```bash
	export APUS_LOW_MODEL="$(dirname "$APUS_FULL_MODEL")/9B-5949-low16"
	python export_low.py --source "$APUS_FULL_MODEL" --output "$APUS_LOW_MODEL"
	```

	Start the full engine in terminal A, using an otherwise available GPU:

	```bash
	export JEV_VLLM_PYTHON="$VIRTUAL_ENV/bin/python"
	CUDA_VISIBLE_DEVICES=0 JEV_EFFORT=high \
	JEV_MODEL_PATH="$APUS_FULL_MODEL" JEV_ENGINE_PORT=30002 \
	JEV_GPU_MEMORY_UTILIZATION=0.70 JEV_MAX_NUM_SEQS=8 \
	bash run_backend.sh
	```

	In terminal B, activate the same environment, set the same `CUDA_HOME` and `APUS_LOW_MODEL` paths, and start the shallow engine:

	```bash
	export JEV_VLLM_PYTHON="$VIRTUAL_ENV/bin/python"
	CUDA_VISIBLE_DEVICES=0 JEV_EFFORT=low \
	JEV_MODEL_PATH="$APUS_LOW_MODEL" JEV_ENGINE_PORT=30004 \
	JEV_GPU_MEMORY_UTILIZATION=0.22 JEV_MAX_NUM_SEQS=8 \
	bash run_backend.sh
	```

	These defaults allocate separate engine budgets of 70% and 22% of the same 96 GB GPU. They are not appropriate if another workload already occupies that GPU. Compilation can take several minutes. The launcher verifies metadata and weight hashes before starting; this adds sequential disk reads. In terminal C, check both engines:

	```bash
	curl -f http://127.0.0.1:30002/health
	curl -f http://127.0.0.1:30004/health
	```

	Do not proceed until both are ready. Each foreground process can be stopped with Ctrl-C in its own terminal. Never reuse a port or GPU allocation belonging to another workload. The two engines must belong to the same source checkpoint; archive the export provenance and source revision together. The package uses `max_num_seqs=8` per engine; historical high-only measurements with 32 sequences are a different serving configuration and must not be copied as this setup's result.

	## 2. Start the SDK gateway

	```bash
	export JEV_MODEL_PATH="$APUS_FULL_MODEL"
	export JEV_HIGH_MODEL_PATH="$APUS_FULL_MODEL"
	export JEV_LOW_MODEL_PATH="$APUS_LOW_MODEL"
	export JEV_HIGH_URL=http://127.0.0.1:30002
	export JEV_LOW_URL=http://127.0.0.1:30004
	export JEV_HIGH_MODEL=apus-openjev-9b-high32
	export JEV_LOW_MODEL=apus-openjev-9b-low16
	export JEV_MODEL_NAME=APUS-OpenJev-v1-9B
	# Supply your own local gateway secret; this is NOT your hosted TypeSafe API key.
	export JEV_API_KEY='replace-with-your-local-gateway-secret'
	python gateway.py --host 127.0.0.1 --port 30013
	```

	Startup requires both model directories on the same filesystem as the gateway. It checks 32/16-layer configs, merged indexes, completed low-export provenance, source/export metadata hashes, and each `/v1/models` alias/root against its configured artifact directory. Mismatches fail startup. The tokenizer must come from the full model directory. Engine URLs and aliases must be distinct. Depth headers report the selected verified backend configuration, not per-request layer-hook telemetry; live numerical evaluation remains necessary.

	The gateway binds to localhost by default. Configure an authenticated HTTPS proxy before exposing it publicly. Backend URLs are operator configuration, never accepted from incoming requests. Both backend connections and the SDK client can reuse HTTP connections.

	## 3. Use the official SDK against APUS

	Use the API root as `base_url` — do not append `/v1`. The SDK appends `/v1/systemone` and `/v1/models` itself.

	```python
	import os
	from typesafe_sdk import Choice, Noul, RetryPolicy, TypeSafeClient

	with TypeSafeClient(
	base_url="http://127.0.0.1:30013",
	api_key=os.environ["JEV_API_KEY"],
	model="APUS-OpenJev-v1-9B",
	retry=RetryPolicy(max_retries=0),
	) as client:
	print(client.models.list())
	result = client.system_one(
	state={
	"message": "My parcel arrived damaged yesterday. Can I get a refund?",
	"policy": "Damaged items reported within 30 days qualify for a refund.",
	},
	questions={
	"queue": Choice(
	instructions="Choose the appropriate customer-support queue.",
	criteria={"returns": "Returns and refunds", "billing": "Payments and invoices"},
	),
	"eligible": Noul(instructions="Is the customer eligible under this refund policy?"),
	},
	extra_body={"effort": "low"},
	)
	print(result.answers["queue"].choice)
	print(result.answers["eligible"].noul)
	print(result.raw_http_response.headers["x-openjev-executed-layers"])
	```

	Use `extra_body={"effort": "high"}` for the full model, or omit it because high is the default. `effort` is an APUS extension, not a promise that the hosted TypeSafe API accepts the same field.

	Run the complete English customer-support example (Choice + Noul + ordinal Score):

	```bash
	python sdk_example.py --effort high
	python sdk_example.py --effort low
	```

	Async uses the same fields:

	```python
	import asyncio
	import os
	from typesafe_sdk import AsyncTypeSafeClient, Noul, RetryPolicy

	async def main():
	async with AsyncTypeSafeClient(
	base_url="http://127.0.0.1:30013",
	api_key=os.environ["JEV_API_KEY"],
	model="APUS-OpenJev-v1-9B",
	retry=RetryPolicy(max_retries=0),
	) as client:
	result = await client.system_one(
	state="The customer received a damaged item yesterday.",
	questions={"damaged": Noul(instructions="Did the customer report a damaged item?")},
	extra_body={"effort": "high"},
	)
	print(result.answers["damaged"].noul)

	asyncio.run(main())
	```

	## 4. Use the official hosted TypeSafe service separately

	This snippet sends a real request to the hosted service and can incur charges. Keep its credentials separate from the local gateway secret. It deliberately does not send the APUS `effort` extension.

	```python
	import os
	from typesafe_sdk import Choice, RetryPolicy, TypeSafeClient

	with TypeSafeClient(
	base_url="https://api.typesafe.ai",
	api_key=os.environ["TYPESAFE_API_KEY"],
	model="jev-latest",
	retry=RetryPolicy(max_retries=0),
	) as client:
	result = client.system_one(
	state="The customer was charged twice for the same order.",
	questions={"queue": Choice(
	instructions="Choose the appropriate customer-support queue.",
	criteria={"billing": "Payments and invoices", "shipping": "Delivery and tracking"},
	)},
	)
	print(result.answers["queue"].choice)
	```

	## Contract and limits

	- Supported SDK 0.7.0 methods: synchronous/asynchronous `system_one` and `models.list`. This SDK version exposes no `models.retrieve` method.
	- `GET /v1/models` lists the APUS model and `jev-latest` as an explicitly labeled local compatibility alias. Responses name the actual APUS model.
	- `POST /v1/systemone` accepts `state`, `questions`, `model`, and the optional APUS `effort` extension only. Unknown fields, invalid effort, unsupported models, malformed or duplicate-key JSON fail explicitly.
	- State: string / object / array. Instructions: string / object / array / null. Choice descriptions also allow null; null uses the candidate name. JSON is serialized deterministically; ordinary string descriptions remain unchanged.
	- Choice: 2–16 candidates. Noul: yes/no; optional true/false descriptions affect the question's meaning. Canonical true/false descriptions retain the original binary prompt. Custom descriptions are included explicitly.
	- Score: 1–10 ordered levels, selected by their descriptions; returns the expected zero-based index, probabilities, and the original legend. This differs from the trained binary `score_level` task and has not established ordinal-score quality or calibration. A single-level Score has only one possible result: score 0, probability 1, no GPU request, zero model tokens, and executed-layers header 0. Its confidence 1 denotes a degenerate distribution, not evidence that the rubric is objectively correct.
	- At most 16 questions, 1 MiB request body, and 8,191 compiled prompt tokens per question (one output token reserved). Inputs are never silently truncated. Maximum 32 in-flight requests, with a 120-second request deadline.
	- Each question is compiled and scored separately. Usage sums the processed prompt tokens and one scoring token per question; `billing_units` means local decision count, not official monetary billing.
	- Confidence is normalized distribution concentration. Candidate probabilities are uncalibrated conditional softmax; neither quantity inherits official calibration semantics.
	- Response shape uses `model`, `answers`, `usage`; Choice has `choice/confidence/probabilities`, Noul has `noul`, Score has `score/confidence/probabilities/legend`.
	- Headers `x-openjev-effort`, `x-openjev-executed-layers`, and `x-openjev-engine-model` record the chosen route. `x-typesafe-request-id` is a local request identifier.

	## Verification status

	CPU integration tests use the real official SDK 0.7.0, both sync and async, through an ASGI gateway with a deterministic mocked inference backend. They verify protocol, routing, payload validation, and numerical response assembly; they are not GPU quality or performance measurements.

	```bash
	pip install -r requirements-test.txt
	python -m pytest -q tests/test_typesafe_gateway.py
	```

	Before reporting GPU performance, record immutable model/export identities, prove 16/32-layer engine admission, run the same frozen requests through both routes, retain raw responses and collection timestamps, and compare against a reference using the same compiled prompts. In particular, a historic `score_level` record mapped to TypeSafe Noul changes its prompt primitive; do not silently compare it as byte-identical input.

	Run a real service smoke test and preserve its raw requests, responses, UTC timestamps, and per-call latency:

	```bash
	python smoke_sdk.py --base-url http://127.0.0.1:30013 --output ./sdk-smoke-new-run
	```

	This sends five HTTP calls: models, high multi-question, low multi-question, an expected 422 for invalid effort, and an async low request. The two multi-question requests include all three primitives. Use a new output directory for every run. No authentication headers are saved.

	Deployment code retains the included Apache 2.0 license and source notices. Model weights retain their applicable upstream licenses.