Routing and Failover

Routing in XSI answers two questions: which model a request goes to, and which upstream key carries it. The first is decided by model mapping; the second by Level (key pool group) and the scheduling policy.

Core objects

ObjectDescription
Provider keyAn upstream key added by the tenant, with attributes such as Provider (upstream URL), Level, and enabled status
LevelKey pool group number. Keys in the same Level form one round-robin pool
MODEL_MAPPERModel rewrite rules: requested model β†’ upstream model
LEVEL_MAPPERGrouping rules: upstream model β†’ Level
GearThe account's routing mode: whether it uses tenant or account-level mappings, and whether it can retry across groups

A typical setup: OpenAI keys in Level 1, Anthropic keys in Level 2, DeepSeek keys in Level 3, with LEVEL_MAPPER = gpt*=1, claude*=2, deepseek*=3. When you add a key, XSI fills in this kind of mapping automatically based on the provider.

Model resolution

The requested model is resolved in the following order, and the first match wins:

  1. The currently active model failover (MODEL_FAILOVER, see below)
  2. Account-level ModelMapper: exact first, then wildcard
  3. Tenant-level MODEL_MAPPER: exact first, then wildcard
  4. Fuzzy matching: if the model is already in the price table, it is used as is; otherwise the closest version among priced models of the same family and modality is chosen (for example gpt-image-2.5-flare β†’ gpt-image-2.6-flare), ignoring suffixes such as latest, preview, and date snapshots
  5. Used as is

The result actually rewrites the model name sent upstream (the JSON model, a multipart field, or the model in the URL), and is cached as an exact account-level mapping that later requests hit first. As a result, after the tenant mapping is changed with PUT /x-config, accounts that have already resolved that model keep using the cached result; to apply the change immediately, sync it to descendants with ModelMapper in POST /x-conf.

Wildcard rules: * matches any characters; all other characters match literally. When several wildcard patterns match, the preferred pattern is the one whose literal suffix sits closer to the end of the model name, then the one with a longer suffix, a longer prefix, a longer total literal length, and fewer *.

MODEL_MAPPER = gpt-latest=gpt-6.2-sol, claude-*-latest=claude-sonnet-5-5

Level resolution

Account GearLevel sourceOn no match
Gear > 0 (follow tenant)Tenant LEVEL_MAPPER: exact first, then wildcard (longest pattern first)The account's own Level
Gear ≀ 0 (independent)The account's own LevelMapperThe account's own Level

A new account inherits Gear from its parent; root and self-registered accounts default to Gear = 1, which follows the tenant configuration.

Key selection

  • Each Level of each tenant maintains a round-robin ring that skips disabled and sleeping keys.
  • Responses, Chat Completions, and Messages requests use sticky sessions: a session is pinned to the same key to improve prompt cache hit rates and so that GET /v1/responses/{id} reads results from the same upstream account. The session identifier is taken, in order, from the session_id and conversation_id request headers, the prompt_cache_key body field, and the session-id and thread-id request headers; if none is present, the binding is per "tenant + account + Level". Bindings are kept for 24 hours by default (STICKY_BINDING_TTL).
  • Sticky scheduling is enabled for all Levels by default (STICKY_SCHEDULER=true); when set to false, it applies only to the Levels listed in the tenant's STICKY_LEVELS.
  • Video polling always uses the key that created the task; DeepSeek file requests always use the key that uploaded the file.
  • If a Level has no available key, the request returns 500 No available key.

Automatic retries

Retries happen before any byte is written to the client, so they also work for streaming requests:

ItemRule
Applicable requestsPOST
Trigger status400 < status < 600 and not in NO_RETRY_STATUS (default 200,400,403,404,405,413,421,422,499,500)
Same-group retriesUp to 7 within the current Level, avoiding the most recently failed key (with only one key in the pool, the same key may be retried)
Cross-group retriesAccounts with Gear ≀ 0: up to 3 on each other Level in the account's AllowLevels; accounts with Gear > 0 do not cross AllowLevels, but may retry between the mapped Level and the account's own Level
ExceptionsWhen a Responses create call returns 400 with not supported, only another key in the same group is tried; video creation is never retried

If every attempt fails, the client receives the upstream response from the first failure.

Key sleep and lifecycle

MechanismConditionEffect
Auto-sleepUpstream returns a status in STATUS_CODES (default 429,401,402,503,408,524); the group has more than 1 key; the key's Level > 0; NoSleep is not setRemoved from rotation for KEY_BACK minutes (default 60)
Peer wake-upThe last remaining key in the group also failsAuto-sleeping keys in the group are woken up so the whole group never goes unavailable
Manual sleepPOST /x-keys/{id}/sleep32 hours
Auto-disableUpstream returns a status in AUTO_DISABLE_KEY_STATUSStatus=false until re-enabled manually
Auto-deleteUpstream returns a status in AUTO_DELETE_KEY_STATUSThe key is deleted

Auto-disable and auto-delete are off by default and never apply to the last enabled key in a group. For example, AUTO_DISABLE_KEY_STATUS=401 makes keys revoked by the upstream drop out automatically.

Group switchover and model failover

Failover triggers when the failure count for the same model under the same tenant reaches MODEL_FAILOVER_THRESHOLD (default 3 in code, 2 in the Compose template). Failures keep accumulating as long as adjacent failures are no more than 1 minute apart, and successful requests in between do not reset the count:

MechanismConfigurationApplies toEffectDuration
Group switchoverSWITCH_OVER = 1=3Gear > 0The model temporarily uses the target Level across the tenantSWITCH_OVER_TTL (default 1 hour)
Model failoverMODEL_FAILOVER = gpt-6.2-sol=gpt-6.1-solWhen no group switchover has occurredTemporarily replaces the model with the fallback model (which must be priced) for that account, and notifies the operatorMODEL_FAILOVER_TTL (default 1 hour)

Failures with a status in NO_RETRY_STATUS are not counted. Switchover and failover state is kept in each instance's memory.

Configuration example

curl -X PUT "$XSI_BASE_URL/x-config" \
  -H "Authorization: Bearer $XSI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "LEVEL_MAPPER": "gpt*=1, claude*=2, deepseek*=3",
    "SWITCH_OVER": "1=4",
    "MODEL_FAILOVER": "claude-opus-5-5=claude-sonnet-5-5"
  }'
  • Put the primary OpenAI keys in Level 1 and backups (for example Azure OpenAI) in Level 4; traffic switches over automatically on sustained failures.
  • When Opus keeps failing, affected accounts temporarily use Sonnet instead.