Set server context to 1Mi with a 16Ki draft window

Apply the server changes from ripple-inference PR #18 (08fd9f4383). Default target context to 1048576, retain draft window 16384, and enable the stock SGLang positional-limit override required for the draft. Update the server README.

https://github.com/ricursive/ripple-inference/pull/18

inference/server/README.md

@@ -28,10 +28,14 @@ curl http://127.0.0.1:30000/v1/chat/completions \
 ```
 
 Defaults: TP=2, BF16 weights and KV cache, TRT-LLM target attention, Triton draft
-attention, 16 draft tokens, and 16K context. Standard SGLang CLI flags apply.
+attention, 16 draft tokens, and a **1Mi-token target context** (`--context-length 1048576`,
+including prompt and generated tokens). Standard SGLang CLI flags apply.
 The DFlash2 draft's last layer is global attention without RoPE. It was trained on
-16K-token rows, so the draft keeps a 16K-token window (`--speculative-draft-window-size 16384`);
+16Ki-token rows, so the draft keeps a **16Ki-token window** (`--speculative-draft-window-size 16384`);
 smaller windows are refused.
+The launcher defaults `SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1` so stock SGLang can
+extend the draft's positional cache beyond its 128Ki checkpoint limit. This does not
+increase the draft attention window. The target checkpoint supports up to 2Mi positions.
 For four GPUs, run a second copy with `CUDA_VISIBLE_DEVICES=2,3` and `--port 30001`.
 This gives two independent replicas, each using two GPUs. Send eight concurrent
 requests to each port for sixteen total.

inference/server/serve.py

@@ -19,6 +19,11 @@ from pathlib import Path
 
 sys.dont_write_bytecode = True
 
+# SGLang sizes the draft's positional cache for the target context, even with
+# windowed draft attention. Allow its 128Ki position limit to grow to 1Mi;
+# --speculative-draft-window-size still limits draft attention to 16Ki tokens.
+os.environ.setdefault("SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN", "1")
+
 # Top-level registration also runs in SGLang's spawned worker processes.
 from sglang.srt.models.registry import ModelRegistry
 
@@ -35,7 +40,7 @@ DEFAULT_ARGS = shlex.split("""
     --speculative-algorithm DFLASH --speculative-draft-model-quantization unquant
     --speculative-num-draft-tokens 16 --speculative-draft-window-size 16384
     --speculative-draft-attention-backend triton
-    --context-length 16384 --mem-fraction-static 0.80 --max-running-requests 16
+    --context-length 1048576 --mem-fraction-static 0.80 --max-running-requests 16
     --cuda-graph-backend-prefill disabled
     --tool-call-parser ripple --reasoning-parser ripple
 """)