Routing and Failover
Routing in XSI answers two questions: which model a request goes to, and which upstream key carries it. The first is decided by model mapping; the second by Level (key pool group) and the scheduling policy.
Core objects
| Object | Description |
|---|---|
| Provider key | An upstream key added by the tenant, with attributes such as Provider (upstream URL), Level, and enabled status |
| Level | Key pool group number. Keys in the same Level form one round-robin pool |
| MODEL_MAPPER | Model rewrite rules: requested model β upstream model |
| LEVEL_MAPPER | Grouping rules: upstream model β Level |
| Gear | The account's routing mode: whether it uses tenant or account-level mappings, and whether it can retry across groups |
A typical setup: OpenAI keys in Level 1, Anthropic keys in Level 2, DeepSeek keys in Level 3, with LEVEL_MAPPER = gpt*=1, claude*=2, deepseek*=3. When you add a key, XSI fills in this kind of mapping automatically based on the provider.
Model resolution
The requested model is resolved in the following order, and the first match wins:
- The currently active model failover (
MODEL_FAILOVER, see below) - Account-level
ModelMapper: exact first, then wildcard - Tenant-level
MODEL_MAPPER: exact first, then wildcard - Fuzzy matching: if the model is already in the price table, it is used as is; otherwise the closest version among priced models of the same family and modality is chosen (for example
gpt-image-2.5-flareβgpt-image-2.6-flare), ignoring suffixes such aslatest,preview, and date snapshots - Used as is
The result actually rewrites the model name sent upstream (the JSON model, a multipart field, or the model in the URL), and is cached as an exact account-level mapping that later requests hit first. As a result, after the tenant mapping is changed with PUT /x-config, accounts that have already resolved that model keep using the cached result; to apply the change immediately, sync it to descendants with ModelMapper in POST /x-conf.
Wildcard rules: * matches any characters; all other characters match literally. When several wildcard patterns match, the preferred pattern is the one whose literal suffix sits closer to the end of the model name, then the one with a longer suffix, a longer prefix, a longer total literal length, and fewer *.
MODEL_MAPPER = gpt-latest=gpt-6.2-sol, claude-*-latest=claude-sonnet-5-5Level resolution
| Account Gear | Level source | On no match |
|---|---|---|
Gear > 0 (follow tenant) | Tenant LEVEL_MAPPER: exact first, then wildcard (longest pattern first) | The account's own Level |
Gear β€ 0 (independent) | The account's own LevelMapper | The account's own Level |
A new account inherits Gear from its parent; root and self-registered accounts default to Gear = 1, which follows the tenant configuration.
Key selection
- Each Level of each tenant maintains a round-robin ring that skips disabled and sleeping keys.
- Responses, Chat Completions, and Messages requests use sticky sessions: a session is pinned to the same key to improve prompt cache hit rates and so that
GET /v1/responses/{id}reads results from the same upstream account. The session identifier is taken, in order, from thesession_idandconversation_idrequest headers, theprompt_cache_keybody field, and thesession-idandthread-idrequest headers; if none is present, the binding is per "tenant + account + Level". Bindings are kept for 24 hours by default (STICKY_BINDING_TTL). - Sticky scheduling is enabled for all Levels by default (
STICKY_SCHEDULER=true); when set tofalse, it applies only to the Levels listed in the tenant'sSTICKY_LEVELS. - Video polling always uses the key that created the task; DeepSeek file requests always use the key that uploaded the file.
- If a Level has no available key, the request returns
500 No available key.
Automatic retries
Retries happen before any byte is written to the client, so they also work for streaming requests:
| Item | Rule |
|---|---|
| Applicable requests | POST |
| Trigger status | 400 < status < 600 and not in NO_RETRY_STATUS (default 200,400,403,404,405,413,421,422,499,500) |
| Same-group retries | Up to 7 within the current Level, avoiding the most recently failed key (with only one key in the pool, the same key may be retried) |
| Cross-group retries | Accounts with Gear β€ 0: up to 3 on each other Level in the account's AllowLevels; accounts with Gear > 0 do not cross AllowLevels, but may retry between the mapped Level and the account's own Level |
| Exceptions | When a Responses create call returns 400 with not supported, only another key in the same group is tried; video creation is never retried |
If every attempt fails, the client receives the upstream response from the first failure.
Key sleep and lifecycle
| Mechanism | Condition | Effect |
|---|---|---|
| Auto-sleep | Upstream returns a status in STATUS_CODES (default 429,401,402,503,408,524); the group has more than 1 key; the key's Level > 0; NoSleep is not set | Removed from rotation for KEY_BACK minutes (default 60) |
| Peer wake-up | The last remaining key in the group also fails | Auto-sleeping keys in the group are woken up so the whole group never goes unavailable |
| Manual sleep | POST /x-keys/{id}/sleep | 32 hours |
| Auto-disable | Upstream returns a status in AUTO_DISABLE_KEY_STATUS | Status=false until re-enabled manually |
| Auto-delete | Upstream returns a status in AUTO_DELETE_KEY_STATUS | The key is deleted |
Auto-disable and auto-delete are off by default and never apply to the last enabled key in a group. For example, AUTO_DISABLE_KEY_STATUS=401 makes keys revoked by the upstream drop out automatically.
Group switchover and model failover
Failover triggers when the failure count for the same model under the same tenant reaches MODEL_FAILOVER_THRESHOLD (default 3 in code, 2 in the Compose template). Failures keep accumulating as long as adjacent failures are no more than 1 minute apart, and successful requests in between do not reset the count:
| Mechanism | Configuration | Applies to | Effect | Duration |
|---|---|---|---|---|
| Group switchover | SWITCH_OVER = 1=3 | Gear > 0 | The model temporarily uses the target Level across the tenant | SWITCH_OVER_TTL (default 1 hour) |
| Model failover | MODEL_FAILOVER = gpt-6.2-sol=gpt-6.1-sol | When no group switchover has occurred | Temporarily replaces the model with the fallback model (which must be priced) for that account, and notifies the operator | MODEL_FAILOVER_TTL (default 1 hour) |
Failures with a status in NO_RETRY_STATUS are not counted. Switchover and failover state is kept in each instance's memory.
Configuration example
curl -X PUT "$XSI_BASE_URL/x-config" \
-H "Authorization: Bearer $XSI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"LEVEL_MAPPER": "gpt*=1, claude*=2, deepseek*=3",
"SWITCH_OVER": "1=4",
"MODEL_FAILOVER": "claude-opus-5-5=claude-sonnet-5-5"
}'- Put the primary OpenAI keys in Level 1 and backups (for example Azure OpenAI) in Level 4; traffic switches over automatically on sustained failures.
- When Opus keeps failing, affected accounts temporarily use Sonnet instead.