Files
+6 a5adbbe18b Sync release-1.3 with master (#2983)
* [RayService] More envtests that follow the most common scenario in the RayService code path (#2880)

Signed-off-by: Rueian <[email protected]>

* [RayService] Remove outdated env tests (#2886)

Signed-off-by: kaihsun <[email protected]>

* [RayService] Refactor envtests (#2888)

* [docs][ray-operator] fix typo in Golang version (#2893)

Project uses Golang 1.22 not 1.20.

Signed-off-by: David Xia <[email protected]>

* [Fix][kubectl-plugin] Fix no context nil error SIGSEGV in tests (#2892)

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [CI] Enable testifylint rule (#2896)

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [CI] Enable `testifylint` `error-nil` rule (#2907)

* [kubectl-plugin][feat] support specifying number of head GPUs (#2895)

when creating a RayCluster with `kubectl ray create cluster NAME --head-gpu N`.
Similar to the `--worker-gpu` switch.

Signed-off-by: David Xia <[email protected]>

* Use webhook.CustomValidator instead of deprecated webhook.Validator. (#2803)

* Use webhook.CustomValidator instead of deprecated webhook.Validator.

* Move RayClusterWebhook to pkg/webhook/v1.

* [kubectl-plugin] update context error messages (#2891)

to tell user they can use `--context` to set the K8s context.

Add tests

Signed-off-by: David Xia <[email protected]>
Co-authored-by: Chi-Sheng Liu <[email protected]>

* [Fix][kubectl-plugin] make tests use a temporary kube config (#2894)

file that's not at the default path. Right now tests fail if there's a K8s
current context set with a command like `kubectl config use-context
my-context`.

This change allows tests to pass regardless of current context.

Signed-off-by: David Xia <[email protected]>

* [CI] Enable testifylint formatter rule (#2915)

Signed-off-by: 400Ping <[email protected]>

* [CI] Enable `testifylint` `empty` rule (#2908)

* [CI] Enable testifylint bool-compare rule (#2911)

Signed-off-by: 400Ping <[email protected]>

* [CI] Auto download golang tools in pre-commit (#2917)

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [RayService] a safeguard for preventing overriding the pending cluster during a upgrade (#2887)

Signed-off-by: Rueian <[email protected]>

* [RayService] Refactor unit tests for ShouldPrepareNewCluster (#2928)

Signed-off-by: kaihsun <[email protected]>

* [chore][kubectl-plugin] use consistent capitalization (#2922)

in comments and console messages.

* capitalize "Ray" when used as a proper noun
* capitalize Ray K8s CRDs like "RayJob" and "RayCluster"
* capitalize acronyms like "YAML"
* fix some minor typos

No functional changes.

Signed-off-by: David Xia <[email protected]>

* [CI] Enable `testifylint` `require-error` rule (#2909)

* [kubectl-plugin] support general `kubectl` switches like `--context` (#2883)

* [CI] Fix lint error (require-error) (#2931)

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [CI] Enable `testifylint` `float-compare` rule (#2910)

* [docs] move pre-commit instructions to main dev docs (#2921)

* [CI] Enable `testifylint` `expected-actual` rule (#2914)

* [CI] Generate CRD json schema separately in pre-commit (#2930)

* [release][1/N] Update YAMLs from Ray 2.9 to Ray 2.41 (#2934)

* Delete `[raycluster|rayjob|rayservice]_types_test.go` unnecessary tests (#2935)

* [release][2/N] Update RayCluster Helm chart from Ray 2.9 to Ray 2.41 (#2936)

Signed-off-by: kaihsun <[email protected]>

* [release][3/N] Update RayService e2e tests YAML files from Ray 2.9 to Ray 2.41 (#2937)

* [release][4/N] Update Ray images / versions in kubectl plugin (#2938)

Signed-off-by: kaihsun <[email protected]>

* [release][5/N] Update some RayJob YAMLs from Ray 2.9 to Ray 2.41 (#2941)

Signed-off-by: kaihsun <[email protected]>

* [release][6/N] Remove unnecessary YAMLs (#2946)

Signed-off-by: kaihsun <[email protected]>

* [docs][kubectl-plugin] add dev docs (#2912)

Signed-off-by: David Xia <[email protected]>
Co-authored-by: Chi-Sheng Liu <[email protected]>

* [CI] Add shellcheck and fix error of it (#2933)

* [chore][kubectl-plugin] use better test assertions (#2955)

Fix two places.

* use `assert.EqualError()` instead of `assert.Error()` to check the error
  string is what we expect
* use testify assertion instead of string checking for better error output

Signed-off-by: David Xia <[email protected]>

* [chore] add Markdown linting pre-commit hook (#2953)

Ignore most rules we violate for now.
Fix these two violations.

```
pre-commit run markdownlint --all-files --show-diff-on-failure

...
clients/python-client/README.md:106 MD001/heading-increment Heading levels should only increment by one level at a time [Expected: h3; Actual: h4]
CHANGELOG.md:420:1 MD030/list-marker-space Spaces after list markers [Expected: 1; Actual: 2]
```

Signed-off-by: David Xia <[email protected]>

* [chore][kubectl-plugin] use consistent capitalization (#2950)

in comments and console messages.

capitalize "Ray" when used as a proper noun. Similar to #2922.

Signed-off-by: David Xia <[email protected]>

* [Fix][RayJob] Invalid quote for RayJob submitter (#2949)

Closes: ray-project/kuberay#2943

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [release][7/N] Update RayService YAMLs (#2956)

* [CI] Enable testifylint len rule (#2945)

Signed-off-by: LeoLiao123 <[email protected]>

* [docs][kubectl-plugin] improve help messages (#2952)

Signed-off-by: David Xia <[email protected]>

* [kubectl-plugin] Fix panic when GPU resource is not set (#2954)

Signed-off-by: win5923 <[email protected]>

* [release][8/N] Upgrade Stable Diffusion RayService to Ray 2.41 (#2960)

* [docs][kubectl-plugin] fix incorrect example commands (#2951)

Signed-off-by: David Xia <[email protected]>
Co-authored-by: Chi-Sheng Liu <[email protected]>

* [fix][kubectl-plugin] set worker group CPU limit (#2958)

when creating a new worker group with `kubectl ray create workergroup`.
Write a unit test.

I noticed we are setting resource limits equal to resource requests everywhere
else but in this command. I have a K8s [LimitRange] that prevented the creation
of these worker Pods that had CPU limit defaulting to less than their CPU
requests. Describing the RayCluster showed this warning event.

`Failed to create worker Pod hyperkube/, Pod
"dxia-test-other-group-worker-pm2sh" is invalid:
spec.containers[0].resources.requests: Invalid value: "2": must be less than or
equal to cpu limit of 250m`

Signed-off-by: David Xia <[email protected]>

[LimitRange]: https://kubernetes.io/docs/concepts/policy/limit-range/

* [RayJob] Deflaky RayJob e2e tests (#2963)

Signed-off-by: kaihsun <[email protected]>

* [RayService] Deflaky RayService envtest (#2962)

Signed-off-by: kaihsun <[email protected]>

* [release][9/N] Update text summarizer RayService to Ray 2.41 (#2961)

* [RayService] adapter vllm 0.6.1.post2 (#2823)

* adapter vllm 0.6.1.post2

* fix var define

* Unify the cpu Settings in serve.py in service.yaml, all set to 1

* Maintain the configuration with vllm0.5x

* [Release] Upgrade ray-job.batch-inference.yaml image to 2.41 (#2971)

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [chore][docs] enable Markdownlint rule MD010 (#2975)

* [CI] Change Pre-commit-shellcheck-to-shellcheck-py (#2974)

Signed-off-by: owenowenisme <[email protected]>

* [release] Update Yunikorn YAML file to Ray 2.41 (#2969)

Signed-off-by: Cheng-Yeh Chung <[email protected]>

* [release] Update YuniKorn YAML files to Ray 2.41 (#2976)

Signed-off-by: win5923 <[email protected]>

* [chore][docs] enable Markdownlint rule MD004 (#2973)

[Unordered list style][1]

[1]: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md004---unordered-list-style

* [Test] Use GcsFaultToleranceOptions in test and backward compatibility (#2972)

* Update samples to use Ray 2.41.0 images (#2964)

Signed-off-by: Andrew Sy Kim <[email protected]>

* Update TPU Ray CR manifests to use Ray 2.41.0 (#2965)

* [Refactor] Use constants for image tag, image repo, and versions in golang to avoid hard-coded strings (#2978)

---------

Signed-off-by: Rueian <[email protected]>
Signed-off-by: kaihsun <[email protected]>
Signed-off-by: David Xia <[email protected]>
Signed-off-by: Chi-Sheng Liu <[email protected]>
Signed-off-by: 400Ping <[email protected]>
Signed-off-by: LeoLiao123 <[email protected]>
Signed-off-by: win5923 <[email protected]>
Signed-off-by: owenowenisme <[email protected]>
Signed-off-by: Cheng-Yeh Chung <[email protected]>
Signed-off-by: Andrew Sy Kim <[email protected]>
Co-authored-by: Rueian <[email protected]>
Co-authored-by: Kai-Hsun Chen <[email protected]>
Co-authored-by: David Xia <[email protected]>
Co-authored-by: Chi-Sheng Liu <[email protected]>
Co-authored-by: David Xia <[email protected]>
Co-authored-by: Mykhailo Bobrovskyi <[email protected]>
Co-authored-by: Ping <[email protected]>
Co-authored-by: Owen Lin <[email protected]>
Co-authored-by: Leo Liao <[email protected]>
Co-authored-by: Blocka <[email protected]>
Co-authored-by: zrant <[email protected]>
Co-authored-by: kenchung285 <[email protected]>
Co-authored-by: fscnick <[email protected]>
Co-authored-by: ryanaoleary <[email protected]>
2025-02-10 11:45:16 -05:00

215 lines
7.4 KiB
Go

package e2e
import (
"embed"
"strings"
"github.com/stretchr/testify/require"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
corev1ac "k8s.io/client-go/applyconfigurations/core/v1"
rayv1ac "github.com/ray-project/kuberay/ray-operator/pkg/client/applyconfiguration/ray/v1"
. "github.com/ray-project/kuberay/ray-operator/test/support"
)
//go:embed *.py
var _files embed.FS
func ReadFile(t Test, fileName string) []byte {
t.T().Helper()
file, err := _files.ReadFile(fileName)
require.NoError(t.T(), err)
return file
}
type option[T any] func(t *T) *T
func apply[T any](t *T, options ...option[T]) *T {
for _, opt := range options {
t = opt(t)
}
return t
}
func options[T any](options ...option[T]) option[T] {
return func(t *T) *T {
for _, opt := range options {
t = opt(t)
}
return t
}
}
func newConfigMap(namespace string, options ...option[corev1ac.ConfigMapApplyConfiguration]) *corev1ac.ConfigMapApplyConfiguration {
cmAC := corev1ac.ConfigMap("jobs", namespace).
WithBinaryData(map[string][]byte{}).
WithImmutable(true)
return configMapWith(cmAC, options...)
}
func configMapWith(configMapAC *corev1ac.ConfigMapApplyConfiguration, options ...option[corev1ac.ConfigMapApplyConfiguration]) *corev1ac.ConfigMapApplyConfiguration {
return apply(configMapAC, options...)
}
func file(t Test, fileName string) option[corev1ac.ConfigMapApplyConfiguration] {
return func(cmAC *corev1ac.ConfigMapApplyConfiguration) *corev1ac.ConfigMapApplyConfiguration {
cmAC.WithBinaryData(map[string][]byte{fileName: ReadFile(t, fileName)})
return cmAC
}
}
func files(t Test, fileNames ...string) option[corev1ac.ConfigMapApplyConfiguration] {
var files []option[corev1ac.ConfigMapApplyConfiguration]
for _, fileName := range fileNames {
files = append(files, file(t, fileName))
}
return options(files...)
}
func newRayClusterSpec(options ...option[rayv1ac.RayClusterSpecApplyConfiguration]) *rayv1ac.RayClusterSpecApplyConfiguration {
return rayClusterSpecWith(rayClusterSpec(), options...)
}
func rayClusterSpecWith(spec *rayv1ac.RayClusterSpecApplyConfiguration, options ...option[rayv1ac.RayClusterSpecApplyConfiguration]) *rayv1ac.RayClusterSpecApplyConfiguration {
return apply(spec, options...)
}
func mountConfigMap[T rayv1ac.RayClusterSpecApplyConfiguration | corev1ac.PodTemplateSpecApplyConfiguration](configMap *corev1.ConfigMap, mountPath string) option[T] {
return func(t *T) *T {
switch obj := (interface{})(t).(type) {
case *rayv1ac.RayClusterSpecApplyConfiguration:
obj.HeadGroupSpec.Template.Spec.Containers[0].WithVolumeMounts(corev1ac.VolumeMount().
WithName(configMap.Name).
WithMountPath(mountPath))
obj.HeadGroupSpec.Template.Spec.WithVolumes(corev1ac.Volume().
WithName(configMap.Name).
WithConfigMap(corev1ac.ConfigMapVolumeSource().WithName(configMap.Name)))
case *corev1ac.PodTemplateSpecApplyConfiguration:
obj.Spec.Containers[0].WithVolumeMounts(corev1ac.VolumeMount().
WithName(configMap.Name).
WithMountPath(mountPath))
obj.Spec.WithVolumes(corev1ac.Volume().
WithName(configMap.Name).
WithConfigMap(corev1ac.ConfigMapVolumeSource().WithName(configMap.Name)))
}
return t
}
}
func rayClusterSpec() *rayv1ac.RayClusterSpecApplyConfiguration {
return rayv1ac.RayClusterSpec().
WithRayVersion(GetRayVersion()).
WithHeadGroupSpec(rayv1ac.HeadGroupSpec().
WithRayStartParams(map[string]string{"dashboard-host": "0.0.0.0"}).
WithTemplate(headPodTemplateApplyConfiguration())).
WithWorkerGroupSpecs(rayv1ac.WorkerGroupSpec().
WithReplicas(1).
WithMinReplicas(1).
WithMaxReplicas(1).
WithGroupName("small-group").
WithRayStartParams(map[string]string{"num-cpus": "1"}).
WithTemplate(workerPodTemplateApplyConfiguration()))
}
func podTemplateSpecApplyConfiguration(template *corev1ac.PodTemplateSpecApplyConfiguration, options ...option[corev1ac.PodTemplateSpecApplyConfiguration]) *corev1ac.PodTemplateSpecApplyConfiguration {
return apply(template, options...)
}
func headPodTemplateApplyConfiguration() *corev1ac.PodTemplateSpecApplyConfiguration {
return corev1ac.PodTemplateSpec().
WithSpec(corev1ac.PodSpec().
WithContainers(corev1ac.Container().
WithName("ray-head").
WithImage(GetRayImage()).
WithPorts(
corev1ac.ContainerPort().WithName("gcs").WithContainerPort(6379),
corev1ac.ContainerPort().WithName("serve").WithContainerPort(8000),
corev1ac.ContainerPort().WithName("dashboard").WithContainerPort(8265),
corev1ac.ContainerPort().WithName("client").WithContainerPort(10001),
).
WithResources(corev1ac.ResourceRequirements().
WithRequests(corev1.ResourceList{
corev1.ResourceCPU: resource.MustParse("300m"),
corev1.ResourceMemory: resource.MustParse("1G"),
}).
WithLimits(corev1.ResourceList{
corev1.ResourceCPU: resource.MustParse("500m"),
corev1.ResourceMemory: resource.MustParse("2G"),
}))))
}
func workerPodTemplateApplyConfiguration() *corev1ac.PodTemplateSpecApplyConfiguration {
return corev1ac.PodTemplateSpec().
WithSpec(corev1ac.PodSpec().
WithContainers(corev1ac.Container().
WithName("ray-worker").
WithImage(GetRayImage()).
WithResources(corev1ac.ResourceRequirements().
WithRequests(corev1.ResourceList{
corev1.ResourceCPU: resource.MustParse("300m"),
corev1.ResourceMemory: resource.MustParse("1G"),
}).
WithLimits(corev1.ResourceList{
corev1.ResourceCPU: resource.MustParse("500m"),
corev1.ResourceMemory: resource.MustParse("1G"),
}))))
}
func jobSubmitterPodTemplateApplyConfiguration() *corev1ac.PodTemplateSpecApplyConfiguration {
return corev1ac.PodTemplateSpec().
WithSpec(corev1ac.PodSpec().
WithRestartPolicy(corev1.RestartPolicyNever).
WithContainers(corev1ac.Container().
WithName("ray-job-submitter").
WithImage(GetRayImage()).
WithResources(corev1ac.ResourceRequirements().
WithRequests(corev1.ResourceList{
corev1.ResourceCPU: resource.MustParse("200m"),
corev1.ResourceMemory: resource.MustParse("200Mi"),
}).
WithLimits(corev1.ResourceList{
corev1.ResourceCPU: resource.MustParse("500m"),
corev1.ResourceMemory: resource.MustParse("500Mi"),
}))))
}
func deployRedis(t Test, namespace string, password string) func() string {
redisContainer := corev1ac.Container().WithName("redis").WithImage("redis:7.4").
WithPorts(corev1ac.ContainerPort().WithContainerPort(6379))
dbSizeCmd := []string{"redis-cli", "--no-auth-warning", "DBSIZE"}
if password != "" {
redisContainer.WithCommand("redis-server", "--requirepass", password)
dbSizeCmd = []string{"redis-cli", "--no-auth-warning", "-a", password, "DBSIZE"}
}
pod, err := t.Client().Core().CoreV1().Pods(namespace).Apply(
t.Ctx(),
corev1ac.Pod("redis", namespace).
WithLabels(map[string]string{"app": "redis"}).
WithSpec(corev1ac.PodSpec().WithContainers(redisContainer)),
TestApplyOptions,
)
require.NoError(t.T(), err)
_, err = t.Client().Core().CoreV1().Services(namespace).Apply(
t.Ctx(),
corev1ac.Service("redis", namespace).
WithSpec(corev1ac.ServiceSpec().
WithSelector(map[string]string{"app": "redis"}).
WithPorts(corev1ac.ServicePort().
WithPort(6379),
),
),
TestApplyOptions,
)
require.NoError(t.T(), err)
return func() string {
stdout, stderr := ExecPodCmd(t, pod, "redis", dbSizeCmd)
return strings.TrimSpace(stdout.String() + stderr.String())
}
}