## Retries and Fallbacks

Retries and fallbacks solve different parts of a failed gateway request:

- A **retry** repeats the current model/provider route after a delay.
- A **fallback** advances to the next configured model after the current route exhausts its attempts.

The gateway returns as soon as one attempt succeeds. If every eligible attempt fails, it returns the terminal error; configuring fallback models improves resilience but does not guarantee success.

## Execution Order

For a request that reaches an upstream provider, Respan applies reliability controls in this order:

1. **Preflight checks** validate authentication, the request, and applicable limits. A request rejected here never reaches a model, so provider retries and model fallback do not run.
2. **Route selection** resolves the primary model, provider or load-balancing route, credentials, and ordered fallback models.
3. **Current-route attempts** call the selected route up to `num_retries` total attempts when retries are enabled. The delay begins at `retry_after`, increases exponentially, includes jitter, and is capped at 60 seconds.
4. **Fallback advance** moves to the next eligible model only after the current route exhausts its attempts and the failure is not marked fail-fast.
5. **Return** stops on the first success. If no route succeeds, Respan returns the last terminal error with per-model failure context where available.

Despite the field name, `num_retries` is the maximum number of attempts for the current route, including the first attempt. For example, `num_retries: 3` allows at most three calls to that route. Setting `retry_enabled: false` makes one attempt per eligible route before fallback is considered.

## Reliability Matrix

| Outcome or failure stage | Retry current route? | Advance to a fallback model? | Result |
| --- | --- | --- | --- |
| Preflight authentication, validation, or limit rejection | No provider call | No | Return the rejection immediately |
| Current route succeeds | No further attempt | No | Return the successful response |
| Retryable provider or network failure | Yes, up to `num_retries` total attempts | After attempts are exhausted | Try the next eligible model |
| Current route exhausts attempts with a non-fail-fast error | No attempts remain on that route | Yes | Continue through `fallback_models` in order |
| Deterministic or unsafe-to-repeat failure marked fail-fast, such as an oversized context, an invalid customer-owned credential, or a model-side read timeout | No further retry | No further model fallback | Return the terminal error |
| Every eligible route fails | No | No routes remain | Return the terminal gateway error |

Do not use retries to mask invalid input or limits. Fix request validation, credentials, context size, and spend/rate-limit configuration at their source. Repeating the same deterministic failure adds latency and can consume provider quota.

## Configure Retries

###### Via UI

###### OpenAI Python SDK

###### Standard API

Go to the [Retries page](https://platform.respan.ai/platform/api/retries), enable retries, set the maximum attempts, and set the initial delay.

###### Supported Retry Parameters

## Troubleshooting

- **The request failed without calling a fallback.** Check whether authentication, validation, limits, context size, or a customer-owned credential caused a preflight or fail-fast rejection.
- **Latency is higher than expected.** Each fallback model has its own attempt budget. Reduce `num_retries`, shorten the initial delay, or shorten the fallback chain.
- **A fallback uses the wrong credentials.** Credentials are resolved per model. Review [provider keys](/content/docs/documentation/admin/llm-provider-keys/index.html) and per-model credential overrides.
- **You need distribution rather than failure recovery.** Use [load balancing](/content/docs/documentation/features/gateway/load-balancing/index.html); it selects a route before the retry/fallback sequence.