model-groups-and-failover.html

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Model groups and failover: how Modelflare routes a request | Modelflare Blog</title>
<meta name="description" content="Every request depends on two identifiers: the exact model ID in the body and the routing group on the key. Ordinary keys add ordered fallback groups that take over when a route cannot serve the request."><meta property="og:image" content="https://modelflare-blog.static.hf.space/images/model-groups-and-failover-cover.jpg"><meta property="og:image:width" content="1200"><meta property="og:image:height" content="630"><meta property="og:title" content="Model groups and failover: how Modelflare routes a request | Modelflare Blog"><meta property="og:description" content="Every request depends on two identifiers: the exact model ID in the body and the routing group on the key. Ordinary keys add ordered fallback groups that take over when a route cannot serve the request."><meta property="og:type" content="article"><meta property="og:url" content="https://modelflare-blog.static.hf.space/model-groups-and-failover.html"><meta property="og:site_name" content="Modelflare Blog"><link rel="canonical" href="https://modelflare-blog.static.hf.space/model-groups-and-failover.html"><meta name="twitter:card" content="summary_large_image"><meta name="twitter:title" content="Model groups and failover: how Modelflare routes a request | Modelflare Blog"><meta name="twitter:description" content="Every request depends on two identifiers: the exact model ID in the body and the routing group on the key. Ordinary keys add ordered fallback groups that take over when a route cannot serve the request."><meta name="twitter:image" content="https://modelflare-blog.static.hf.space/images/model-groups-and-failover-cover.jpg">
<style>
:root{--bg:#0b1020;--card:#121a2f;--text:#e8ecf7;--muted:#9aa6c2;--accent:#8b5cf6;--accent2:#ec4899;--border:rgba(255,255,255,.08)}
*{box-sizing:border-box}
body{margin:0;font-family:ui-sans-serif,system-ui,-apple-system,Segoe UI,Roboto,Helvetica,Arial,sans-serif;background:radial-gradient(1200px 600px at 10% -10%,#2a1850 0%,transparent 50%),radial-gradient(900px 500px at 100% 0%,#3b0f2e 0%,transparent 45%),var(--bg);color:var(--text);min-height:100vh;line-height:1.65}
.wrap{max-width:860px;margin:0 auto;padding:28px 18px 64px}
header.top{display:flex;flex-wrap:wrap;gap:14px;align-items:center;justify-content:space-between;margin-bottom:28px;font-size:14px}
header.top a{color:var(--muted);text-decoration:none}header.top a:hover{color:#fff}
header.top .brand{color:#fff;font-weight:700;font-size:16px}
header.top nav{display:flex;flex-wrap:wrap;gap:14px}
h1{font-size:clamp(26px,4.6vw,38px);line-height:1.2;margin:0 0 12px}
h2{font-size:22px;margin:36px 0 10px}h3{font-size:17px;margin:24px 0 8px;color:#d6c7ff}
p,li{color:#cfd6e8}a{color:#c4b5fd}
.lead{color:var(--muted);font-size:17px}
article img{max-width:100%;height:auto;border-radius:12px;border:1px solid var(--border);display:block;margin:18px auto 6px}
article p em:only-child{display:block;text-align:center;color:var(--muted);font-size:14px}
pre{background:#070b16;border:1px solid var(--border);border-radius:12px;padding:0;overflow-x:auto}
pre code.hljs{background:transparent;padding:14px 16px;font-size:13.5px;line-height:1.55}
code{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:.92em}
p code,li code,td code{background:rgba(139,92,246,.14);padding:1px 5px;border-radius:5px}
table{border-collapse:collapse;width:100%;font-size:14px;margin:14px 0;display:block;overflow-x:auto}
th,td{border:1px solid var(--border);padding:8px 10px;text-align:left;vertical-align:top}th{background:var(--card)}
hr{border:0;border-top:1px solid var(--border);margin:32px 0}
.list{display:grid;gap:14px}
.card{display:flex;gap:16px;background:var(--card);border:1px solid var(--border);border-radius:16px;padding:18px;text-decoration:none;color:inherit}
.card:hover{border-color:rgba(139,92,246,.5)}
.card img{width:200px;height:105px;object-fit:cover;border-radius:10px;flex-shrink:0}
.card h2{font-size:18px;margin:0 0 6px;color:#fff}.card p{margin:0;color:var(--muted);font-size:14px}
footer{margin-top:48px;padding-top:18px;border-top:1px solid var(--border);color:var(--muted);font-size:13px}
footer a{color:var(--muted)}
@media(max-width:600px){.card{flex-direction:column}.card img{width:100%;height:auto;aspect-ratio:1200/630}}
</style>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@highlightjs/cdn-assets@11.9.0/styles/github-dark.min.css"><script src="https://cdn.jsdelivr.net/npm/@highlightjs/cdn-assets@11.9.0/highlight.min.js"></script><script>document.addEventListener("DOMContentLoaded",()=>hljs.highlightAll());</script>
</head>
<body><div class="wrap"><header class="top"><a class="brand" href="index.html">Modelflare Blog</a><nav><a href="https://modelflare.dev/?utm_source=huggingface&utm_medium=space&utm_campaign=blog">modelflare.dev</a><a href="https://docs.modelflare.dev/?utm_source=huggingface&utm_medium=space&utm_campaign=blog">Docs</a><a href="https://castaly.modelflare.dev/?utm_source=huggingface&utm_medium=space&utm_campaign=blog">Castaly</a><a href="https://huggingface.co/Modelflare" target="_top">Hugging Face org</a></nav></header>
<article><h1>Model groups and failover: how Modelflare routes a request</h1>
<p class="lead">Every request depends on two identifiers: the exact model ID in the body and the routing group on the key. Ordinary keys add ordered fallback groups that take over when a route cannot serve the request.</p>
<img src="images/model-groups-and-failover-cover.jpg" alt="Illustration: parallel routes converging on one output">
<hr />
<p><a href="https://modelflare.dev/?utm_source=huggingface&amp;utm_medium=space&amp;utm_campaign=model-groups-and-failover">Modelflare</a> routes each request through a routing group, a set of service provider channels. This article explains how model IDs and groups interact, how group failover works on ordinary keys, which errors stop a request immediately, and how billing follows the group that served the request.</p>
<h2>Two identifiers on every request</h2>
<ol>
<li><strong>The model ID</strong> in the JSON body. It must match a public catalog ID exactly. Do not substitute a provider marketing name, a date suffix, or a nearby variant.</li>
<li><strong>The routing group</strong> bound to the API key. It decides which service routes can serve that model.</li>
</ol>
<p>Both have to agree. A valid JSON body still fails when the key's group has no channel for that model. Create the key in a non-default group that lists the model; there is no single required group. The live catalog, with prices, groups, and protocol labels, is <a href="https://modelflare.dev/pricing?utm_source=huggingface&amp;utm_medium=space&amp;utm_campaign=model-groups-and-failover">Models &amp; Pricing</a>.</p>
<p>To check which models a key can call:</p>
<pre><code class="language-bash">curl https://modelflare.dev/v1/models \
  -H &quot;Authorization: Bearer YOUR_MODELFLARE_API_KEY&quot;
</code></pre>
<p>Use the exact <code>id</code> from the response in the <code>model</code> field.</p>
<h2>The routing chain</h2>
<p>An ordinary API key names one primary group and, optionally, fallback groups in priority order. Configure the chain in <a href="https://modelflare.dev/keys?utm_source=huggingface&amp;utm_medium=space&amp;utm_campaign=model-groups-and-failover">API Keys</a> when you create or edit a key. Order matters, and the console lets you move entries up and down.</p>
<ul>
<li><strong>Primary group.</strong> The group named on the key.</li>
<li><strong>Channel priorities inside it.</strong> Every priority in the group is tried before the next group.</li>
<li><strong>Next fallback group.</strong> A route change, not another retry of the same service provider.</li>
<li><strong>Billed by the group that served.</strong> The resolved group is recorded in the consume log.</li>
</ul>
<h2>When a request moves to the next group</h2>
<ul>
<li>The group has no channel for the requested model.</li>
<li>The group is at its rate limit for your account.</li>
<li>Every channel priority inside the group has been tried and still returned a retryable error.</li>
</ul>
<p>A group exhausts its own channel priorities before the next group is tried, so a fallback is a route change rather than an extra retry of the same provider.</p>
<h2>When it does not</h2>
<p>Failover covers route and connection problems only. The request stops immediately on:</p>
<ul>
<li>client errors, such as malformed JSON, an invalid parameter, or context length exceeded;</li>
<li>a cancelled client connection;</li>
<li>a request pinned to one specific channel;</li>
<li>a streamed response that has already sent real body bytes to the client.</li>
</ul>
<p>Heartbeats alone do not prevent a switch, but once content has been written, the request is never replayed on another route. A client that streams should handle a mid-stream failure itself.</p>
<h2>Status codes to expect</h2>
<table>
<thead>
<tr>
<th>Status</th>
<th>Typical cause</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>400</code></td>
<td>Invalid JSON, unsupported field, or wrong endpoint for the model</td>
</tr>
<tr>
<td><code>401</code></td>
<td>Missing or invalid API key</td>
</tr>
<tr>
<td><code>403</code></td>
<td>The key cannot use this model or group</td>
</tr>
<tr>
<td><code>404</code></td>
<td>Unknown path or missing task</td>
</tr>
<tr>
<td><code>429</code></td>
<td>Rate limit</td>
</tr>
<tr>
<td><code>499</code></td>
<td>Client cancelled the request</td>
</tr>
<tr>
<td><code>5xx</code></td>
<td>Service provider or gateway failure</td>
</tr>
</tbody>
</table>
<p>Image and video requests sent to the wrong protocol return <code>400</code> with <code>unsupported_endpoint</code> and name the required API.</p>
<h2>Billing after a switch</h2>
<p>After a failover, the request is billed at the multiplier of the group that served it, not the primary group. The consume log records which group that was, so a route change is visible after the fact.</p>
<h2>Campaign groups require a fallback</h2>
<p>Limited-time routes are temporary by design. When a key's primary group is a campaign route, the key cannot be saved without at least one concrete fallback group, so it keeps working after that route ends.</p>
<h2>Smart API Keys</h2>
<p>A Smart API Key does not take a manual fallback list. It walks every group the account can use, ordered by its routing strategy (balanced, stability first, or low price first). Choose an ordinary key when you want to control the order yourself and a Smart API Key when you want the gateway to resolve it. See <a href="smart-api-key.html">Smart API Keys</a> for details.</p>
<p>Further reading: <a href="https://docs.modelflare.dev/guides/models-and-groups/?utm_source=huggingface&amp;utm_medium=space&amp;utm_campaign=model-groups-and-failover">Models and groups</a>, <a href="https://docs.modelflare.dev/guides/features/group-failover/?utm_source=huggingface&amp;utm_medium=space&amp;utm_campaign=model-groups-and-failover">Group failover</a>, and <a href="https://docs.modelflare.dev/guides/errors/?utm_source=huggingface&amp;utm_medium=space&amp;utm_campaign=model-groups-and-failover">Errors</a>. For higher limits or volume arrangements, contact <a href="mailto:business@modelflare.dev">business@modelflare.dev</a>.</p></article><p><a href="index.html">&larr; All articles</a></p>
<footer>Modelflare on Hugging Face: <a href="https://huggingface.co/spaces/Modelflare/start" target="_top">start</a> · <a href="https://huggingface.co/spaces/Modelflare/openai-sdk" target="_top">openai-sdk</a> · <a href="https://huggingface.co/spaces/Modelflare/coding-agents" target="_top">coding-agents</a> · <a href="https://huggingface.co/spaces/Modelflare/api-recipes" target="_top">api-recipes</a> · <a href="https://huggingface.co/datasets/Modelflare/api-examples" target="_top">api-examples dataset</a><br>Business: <a href="mailto:business@modelflare.dev">business@modelflare.dev</a></footer></div></body>
</html>