Files
2acc21964d [RayService] Support Incremental Zero-Downtime Upgrades (#3166)
* Add incremental upgrade API changes to KubeRay

Signed-off-by: Ryan O'Leary <[email protected]>

Update go mod dependencies for gateway v1

Signed-off-by: Ryan O'Leary <[email protected]>

Add reconcile Gateway and HTTPRoute

Signed-off-by: Ryan O'Leary <[email protected]>

Add TargetCapacity and TrafficRoutedPercent to RayServiceStatus

Signed-off-by: Ryan O'Leary <[email protected]>

Add controller logic initial commit

Signed-off-by: Ryan O'Leary <[email protected]>

Add IncrementalUpgrade check to ShouldUpdate

Signed-off-by: Ryan O'Leary <[email protected]>

Update controller logic to reconcile incremental upgrade

Signed-off-by: Ryan O'Leary <[email protected]>

TrafficRoutedPercent should not set default value

Signed-off-by: Ryan O'Leary <[email protected]>

Remove test changes to TPU manifest

Signed-off-by: Ryan O'Leary <[email protected]>

Move helper function to utils

Signed-off-by: Ryan O'Leary <[email protected]>

Fix lint

Signed-off-by: Ryan O'Leary <[email protected]>

Fix field alignment

Signed-off-by: Ryan O'Leary <[email protected]>

Fix bad merge

Signed-off-by: Ryan O'Leary <[email protected]>

Fix CRDs and add validation test case

Signed-off-by: Ryan O'Leary <[email protected]>

Test create HTTPRoute and create Gateway

Signed-off-by: Ryan O'Leary <[email protected]>

Add reconcile tests for Gateway and HTTPRoute

Signed-off-by: Ryan O'Leary <[email protected]>

Fix lint

Signed-off-by: Ryan O'Leary <[email protected]>

Add tests for util functions and fix golangci-lint

Signed-off-by: Ryan O'Leary <[email protected]>

Add basic e2e test case

Signed-off-by: Ryan O'Leary <[email protected]>

Fix GetGatewayListeners logic and test

Signed-off-by: Ryan O'Leary <[email protected]>

Add gatewayv1 scheme to util runtime

Signed-off-by: Ryan O'Leary <[email protected]>

Check if IncrementalUpgrade is enabled before checking Gateway

Signed-off-by: Ryan O'Leary <[email protected]>

Fix reconcile logic for Gateway and HTTPRoute

Signed-off-by: Ryan O'Leary <[email protected]>

Add feature gate

Signed-off-by: Ryan O'Leary <[email protected]>

Always create Gateway and HTTPRoute for IncrementalUpgrade

Signed-off-by: Ryan O'Leary <[email protected]>

Fix target_capacity reonciliation logic

Signed-off-by: Ryan O'Leary <[email protected]>

Add additional unit tests

Signed-off-by: Ryan O'Leary <[email protected]>

Move e2e test and add another unit test

Signed-off-by: Ryan O'Leary <[email protected]>

* Fix some tests and create Gateway for pending cluster

Signed-off-by: Ryan O'Leary <[email protected]>

* Fix merge errors

Signed-off-by: Ryan O'Leary <[email protected]>

* Manually sync rbac for gateway

Signed-off-by: Ryan O'Leary <[email protected]>

* Fix bugs and e2e test

Signed-off-by: Ryan O'Leary <[email protected]>

* Add Makefile command

Signed-off-by: Ryan O'Leary <[email protected]>

* Run 'make sync'

Signed-off-by: Ryan O'Leary <[email protected]>

* Run 'make generate'

Signed-off-by: Ryan O'Leary <[email protected]>

* Fix comments

Signed-off-by: Ryan O'Leary <[email protected]>

* Run 'make api-docs'

Signed-off-by: Ryan O'Leary <[email protected]>

* Fix tests after merge conflicts

Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Fix error return

Signed-off-by: Ryan O'Leary <[email protected]>

* Add RayServiceIncrementalUpgrade feature gate option to helm

Signed-off-by: Ryan O'Leary <[email protected]>

* Remove unnecessary perms

Signed-off-by: Ryan O'Leary <[email protected]>

* Remove delete perm and run lint

Signed-off-by: Ryan O'Leary <[email protected]>

* Fix helm roles

Signed-off-by: Ryan O'Leary <[email protected]>

* add back required perms

Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/utils/validation.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/utils/util.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Change controller to use two serve services during upgrade

Signed-off-by: Ryan O'Leary <[email protected]>

* Remove Gateway and HTTPRoute API fields

Signed-off-by: Ryan O'Leary <[email protected]>

* Fix port errors

Signed-off-by: Ryan O'Leary <[email protected]>

* Fix comments and build issues

Signed-off-by: Ryan O'Leary <[email protected]>

* fix helm-chart-verify-rbac

Signed-off-by: Ryan O'Leary <[email protected]>

* Refactor tests and create HTTPRoute to be clearer

Signed-off-by: Ryan O'Leary <[email protected]>

* Use time &now

Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Add function comments

Signed-off-by: Ryan O'Leary <[email protected]>

* Fix bad merge

Signed-off-by: Ryan O'Leary <[email protected]>

* Add more comments

Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Add Ray Serve hostname and serve port logic

Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/common/service.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/common/service.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Fix dropped requests and old cluster config not being served

Signed-off-by: Ryan O'Leary <[email protected]>

* Resolve readability comments and improve structure

Signed-off-by: Ryan O'Leary <[email protected]>

* Refactor based on comments

Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/common/service.go

Co-authored-by: Rueian <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Remove hostname from listener

Signed-off-by: Ryan O'Leary <[email protected]>

* ensure pending cluster scales from 0 target_capacity

Signed-off-by: Ryan O'Leary <[email protected]>

* Run make generate after rebase

Signed-off-by: Ryan O'Leary <[email protected]>

* rename upgrade type

Signed-off-by: Ryan O'Leary <[email protected]>

* Clean up utils and add more comments

Signed-off-by: Ryan O'Leary <[email protected]>

* reconcileHTTPRoute should pass created object to calculate status

Signed-off-by: Ryan O'Leary <[email protected]>

* lint

Signed-off-by: Ryan O'Leary <[email protected]>

* Update ray-operator/controllers/ray/rayservice_controller.go

Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>

* Fix test after suggested fix

Signed-off-by: Ryan O'Leary <[email protected]>

---------

Signed-off-by: Ryan O'Leary <[email protected]>
Signed-off-by: Ryan O'Leary <[email protected]>
Co-authored-by: Han-Ju Chen (Future-Outlier) <[email protected]>
Co-authored-by: Rueian <[email protected]>
2025-10-25 15:55:12 -07:00

91 lines
1.9 KiB
Go

package support
import (
"k8s.io/client-go/dynamic"
"k8s.io/client-go/kubernetes"
// Import all Kubernetes client auth plugins (e.g. Azure, GCP, OIDC, etc.)
// to ensure that exec-entrypoint and run can make use of them.
_ "k8s.io/client-go/plugin/pkg/client/auth"
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/clientcmd"
gatewayclient "sigs.k8s.io/gateway-api/pkg/client/clientset/versioned"
rayclient "github.com/ray-project/kuberay/ray-operator/pkg/client/clientset/versioned"
)
type Client interface {
Core() kubernetes.Interface
Ray() rayclient.Interface
Dynamic() dynamic.Interface
Config() rest.Config
Gateway() gatewayclient.Interface
}
type testClient struct {
core kubernetes.Interface
ray rayclient.Interface
dynamic dynamic.Interface
config rest.Config
gateway gatewayclient.Interface
}
var _ Client = (*testClient)(nil)
func (t *testClient) Core() kubernetes.Interface {
return t.core
}
func (t *testClient) Ray() rayclient.Interface {
return t.ray
}
func (t *testClient) Dynamic() dynamic.Interface {
return t.dynamic
}
func (t *testClient) Config() rest.Config {
return t.config
}
func (t *testClient) Gateway() gatewayclient.Interface {
return t.gateway
}
func newTestClient() (Client, error) {
cfg, err := clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
clientcmd.NewDefaultClientConfigLoadingRules(),
&clientcmd.ConfigOverrides{},
).ClientConfig()
if err != nil {
return nil, err
}
kubeClient, err := kubernetes.NewForConfig(cfg)
if err != nil {
return nil, err
}
rayClient, err := rayclient.NewForConfig(cfg)
if err != nil {
return nil, err
}
dynamicClient, err := dynamic.NewForConfig(cfg)
if err != nil {
return nil, err
}
gatewayClient, err := gatewayclient.NewForConfig(cfg)
if err != nil {
return nil, err
}
return &testClient{
core: kubeClient,
ray: rayClient,
dynamic: dynamicClient,
config: *cfg,
gateway: gatewayClient,
}, nil
}