Files
c7669d0968 [APIServer][Docs] Add user guide for retry behavior & configuration (#4144)
* [Docs] Add the draft description about feature intro, configurations, and usecases

Signed-off-by: justinyeh1995 <[email protected]>

* [Fix] Update the retry walk-through

Signed-off-by: justinyeh1995 <[email protected]>

* [Doc] rewrite the first 2 sections

Signed-off-by: justinyeh1995 <[email protected]>

* [Doc] Revise documentation wording and add Observing Retry Behavior section

Signed-off-by: justinyeh1995 <[email protected]>

* [Fix] fix linting issue by running pre-commit run berfore commiting

Signed-off-by: justinyeh1995 <[email protected]>

* [Fix] fix linting errors in the Markdown linting

Signed-off-by: justinyeh1995 <[email protected]>

* [Fix] Clean up the math equation

Signed-off-by: justinyeh1995 <[email protected]>

* Update the math formula of Backoff calculation.

Co-authored-by: Nary Yeh <[email protected]>
Signed-off-by: JustinYeh <[email protected]>

* [Fix] Explicitly mentioned exponential backoff and removed the customization parts

Signed-off-by: justinyeh1995 <[email protected]>

* [Docs] Clarify naming by replacing “APIServer” with “KubeRay APIServer”

Co-authored-by: Cheng-Yeh Chung <[email protected]>
Signed-off-by: JustinYeh <[email protected]>

* [Docs] Rename retry-configuration.md to retry-behavior.md for accuracy

Signed-off-by: justinyeh1995 <[email protected]>

* Update Title to KubeRay APIServer Retry Behavior

Co-authored-by: Cheng-Yeh Chung <[email protected]>
Signed-off-by: JustinYeh <[email protected]>

* [Docs] Add a note about the limitation of retry configuration

Signed-off-by: justinyeh1995 <[email protected]>

---------

Signed-off-by: justinyeh1995 <[email protected]>
Signed-off-by: JustinYeh <[email protected]>
Co-authored-by: Nary Yeh <[email protected]>
Co-authored-by: Cheng-Yeh Chung <[email protected]>
2025-11-19 20:50:54 -08:00

1.3 KiB

KubeRay APIServer Retry Behavior

The KubeRay APIServer automatically retries failed requests to the Kubernetes API when transient errors occur. This built-in mechanism uses exponential backoff to improve reliability without requiring manual intervention. As of v1.5.0, the retry configuration is hard-coded and cannot be customized. This guide describes the default retry behavior.

Default Retry Behavior

The KubeRay APIServer automatically retries with exponential backoff for these HTTP status codes:

  • 408 (Request Timeout)
  • 429 (Too Many Requests)
  • 500 (Internal Server Error)
  • 502 (Bad Gateway)
  • 503 (Service Unavailable)
  • 504 (Gateway Timeout)

Note that non-retryable errors (4xx except 408/429) fail immediately without retries.

The following default configuration explains how retry works:

  • MaxRetry: 3 retries (4 total attempts including the initial one)
  • InitBackoff: 500ms (initial wait time)
  • BackoffFactor: 2.0 (exponential multiplier)
  • MaxBackoff: 10s (maximum wait time between retries)
  • OverallTimeout: 30s (total timeout for all attempts)

which means \text{Backoff}_i = \min(\text{InitBackoff} \times \text{BackoffFactor}^i, \text{MaxBackoff})

where i is the attempt number (starting from 0). The retries will stop if the total time exceeds the OverallTimeout.