Files
+6 a5adbbe18b Sync release-1.3 with master (#2983)
* [RayService] More envtests that follow the most common scenario in the RayService code path (#2880)

Signed-off-by: Rueian <[email protected]>

* [RayService] Remove outdated env tests (#2886)

Signed-off-by: kaihsun <[email protected]>

* [RayService] Refactor envtests (#2888)

* [docs][ray-operator] fix typo in Golang version (#2893)

Project uses Golang 1.22 not 1.20.

Signed-off-by: David Xia <[email protected]>

* [Fix][kubectl-plugin] Fix no context nil error SIGSEGV in tests (#2892)

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [CI] Enable testifylint rule (#2896)

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [CI] Enable `testifylint` `error-nil` rule (#2907)

* [kubectl-plugin][feat] support specifying number of head GPUs (#2895)

when creating a RayCluster with `kubectl ray create cluster NAME --head-gpu N`.
Similar to the `--worker-gpu` switch.

Signed-off-by: David Xia <[email protected]>

* Use webhook.CustomValidator instead of deprecated webhook.Validator. (#2803)

* Use webhook.CustomValidator instead of deprecated webhook.Validator.

* Move RayClusterWebhook to pkg/webhook/v1.

* [kubectl-plugin] update context error messages (#2891)

to tell user they can use `--context` to set the K8s context.

Add tests

Signed-off-by: David Xia <[email protected]>
Co-authored-by: Chi-Sheng Liu <[email protected]>

* [Fix][kubectl-plugin] make tests use a temporary kube config (#2894)

file that's not at the default path. Right now tests fail if there's a K8s
current context set with a command like `kubectl config use-context
my-context`.

This change allows tests to pass regardless of current context.

Signed-off-by: David Xia <[email protected]>

* [CI] Enable testifylint formatter rule (#2915)

Signed-off-by: 400Ping <[email protected]>

* [CI] Enable `testifylint` `empty` rule (#2908)

* [CI] Enable testifylint bool-compare rule (#2911)

Signed-off-by: 400Ping <[email protected]>

* [CI] Auto download golang tools in pre-commit (#2917)

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [RayService] a safeguard for preventing overriding the pending cluster during a upgrade (#2887)

Signed-off-by: Rueian <[email protected]>

* [RayService] Refactor unit tests for ShouldPrepareNewCluster (#2928)

Signed-off-by: kaihsun <[email protected]>

* [chore][kubectl-plugin] use consistent capitalization (#2922)

in comments and console messages.

* capitalize "Ray" when used as a proper noun
* capitalize Ray K8s CRDs like "RayJob" and "RayCluster"
* capitalize acronyms like "YAML"
* fix some minor typos

No functional changes.

Signed-off-by: David Xia <[email protected]>

* [CI] Enable `testifylint` `require-error` rule (#2909)

* [kubectl-plugin] support general `kubectl` switches like `--context` (#2883)

* [CI] Fix lint error (require-error) (#2931)

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [CI] Enable `testifylint` `float-compare` rule (#2910)

* [docs] move pre-commit instructions to main dev docs (#2921)

* [CI] Enable `testifylint` `expected-actual` rule (#2914)

* [CI] Generate CRD json schema separately in pre-commit (#2930)

* [release][1/N] Update YAMLs from Ray 2.9 to Ray 2.41 (#2934)

* Delete `[raycluster|rayjob|rayservice]_types_test.go` unnecessary tests (#2935)

* [release][2/N] Update RayCluster Helm chart from Ray 2.9 to Ray 2.41 (#2936)

Signed-off-by: kaihsun <[email protected]>

* [release][3/N] Update RayService e2e tests YAML files from Ray 2.9 to Ray 2.41 (#2937)

* [release][4/N] Update Ray images / versions in kubectl plugin (#2938)

Signed-off-by: kaihsun <[email protected]>

* [release][5/N] Update some RayJob YAMLs from Ray 2.9 to Ray 2.41 (#2941)

Signed-off-by: kaihsun <[email protected]>

* [release][6/N] Remove unnecessary YAMLs (#2946)

Signed-off-by: kaihsun <[email protected]>

* [docs][kubectl-plugin] add dev docs (#2912)

Signed-off-by: David Xia <[email protected]>
Co-authored-by: Chi-Sheng Liu <[email protected]>

* [CI] Add shellcheck and fix error of it (#2933)

* [chore][kubectl-plugin] use better test assertions (#2955)

Fix two places.

* use `assert.EqualError()` instead of `assert.Error()` to check the error
  string is what we expect
* use testify assertion instead of string checking for better error output

Signed-off-by: David Xia <[email protected]>

* [chore] add Markdown linting pre-commit hook (#2953)

Ignore most rules we violate for now.
Fix these two violations.

```
pre-commit run markdownlint --all-files --show-diff-on-failure

...
clients/python-client/README.md:106 MD001/heading-increment Heading levels should only increment by one level at a time [Expected: h3; Actual: h4]
CHANGELOG.md:420:1 MD030/list-marker-space Spaces after list markers [Expected: 1; Actual: 2]
```

Signed-off-by: David Xia <[email protected]>

* [chore][kubectl-plugin] use consistent capitalization (#2950)

in comments and console messages.

capitalize "Ray" when used as a proper noun. Similar to #2922.

Signed-off-by: David Xia <[email protected]>

* [Fix][RayJob] Invalid quote for RayJob submitter (#2949)

Closes: ray-project/kuberay#2943

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [release][7/N] Update RayService YAMLs (#2956)

* [CI] Enable testifylint len rule (#2945)

Signed-off-by: LeoLiao123 <[email protected]>

* [docs][kubectl-plugin] improve help messages (#2952)

Signed-off-by: David Xia <[email protected]>

* [kubectl-plugin] Fix panic when GPU resource is not set (#2954)

Signed-off-by: win5923 <[email protected]>

* [release][8/N] Upgrade Stable Diffusion RayService to Ray 2.41 (#2960)

* [docs][kubectl-plugin] fix incorrect example commands (#2951)

Signed-off-by: David Xia <[email protected]>
Co-authored-by: Chi-Sheng Liu <[email protected]>

* [fix][kubectl-plugin] set worker group CPU limit (#2958)

when creating a new worker group with `kubectl ray create workergroup`.
Write a unit test.

I noticed we are setting resource limits equal to resource requests everywhere
else but in this command. I have a K8s [LimitRange] that prevented the creation
of these worker Pods that had CPU limit defaulting to less than their CPU
requests. Describing the RayCluster showed this warning event.

`Failed to create worker Pod hyperkube/, Pod
"dxia-test-other-group-worker-pm2sh" is invalid:
spec.containers[0].resources.requests: Invalid value: "2": must be less than or
equal to cpu limit of 250m`

Signed-off-by: David Xia <[email protected]>

[LimitRange]: https://kubernetes.io/docs/concepts/policy/limit-range/

* [RayJob] Deflaky RayJob e2e tests (#2963)

Signed-off-by: kaihsun <[email protected]>

* [RayService] Deflaky RayService envtest (#2962)

Signed-off-by: kaihsun <[email protected]>

* [release][9/N] Update text summarizer RayService to Ray 2.41 (#2961)

* [RayService] adapter vllm 0.6.1.post2 (#2823)

* adapter vllm 0.6.1.post2

* fix var define

* Unify the cpu Settings in serve.py in service.yaml, all set to 1

* Maintain the configuration with vllm0.5x

* [Release] Upgrade ray-job.batch-inference.yaml image to 2.41 (#2971)

Signed-off-by: Chi-Sheng Liu <[email protected]>

* [chore][docs] enable Markdownlint rule MD010 (#2975)

* [CI] Change Pre-commit-shellcheck-to-shellcheck-py (#2974)

Signed-off-by: owenowenisme <[email protected]>

* [release] Update Yunikorn YAML file to Ray 2.41 (#2969)

Signed-off-by: Cheng-Yeh Chung <[email protected]>

* [release] Update YuniKorn YAML files to Ray 2.41 (#2976)

Signed-off-by: win5923 <[email protected]>

* [chore][docs] enable Markdownlint rule MD004 (#2973)

[Unordered list style][1]

[1]: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md004---unordered-list-style

* [Test] Use GcsFaultToleranceOptions in test and backward compatibility (#2972)

* Update samples to use Ray 2.41.0 images (#2964)

Signed-off-by: Andrew Sy Kim <[email protected]>

* Update TPU Ray CR manifests to use Ray 2.41.0 (#2965)

* [Refactor] Use constants for image tag, image repo, and versions in golang to avoid hard-coded strings (#2978)

---------

Signed-off-by: Rueian <[email protected]>
Signed-off-by: kaihsun <[email protected]>
Signed-off-by: David Xia <[email protected]>
Signed-off-by: Chi-Sheng Liu <[email protected]>
Signed-off-by: 400Ping <[email protected]>
Signed-off-by: LeoLiao123 <[email protected]>
Signed-off-by: win5923 <[email protected]>
Signed-off-by: owenowenisme <[email protected]>
Signed-off-by: Cheng-Yeh Chung <[email protected]>
Signed-off-by: Andrew Sy Kim <[email protected]>
Co-authored-by: Rueian <[email protected]>
Co-authored-by: Kai-Hsun Chen <[email protected]>
Co-authored-by: David Xia <[email protected]>
Co-authored-by: Chi-Sheng Liu <[email protected]>
Co-authored-by: David Xia <[email protected]>
Co-authored-by: Mykhailo Bobrovskyi <[email protected]>
Co-authored-by: Ping <[email protected]>
Co-authored-by: Owen Lin <[email protected]>
Co-authored-by: Leo Liao <[email protected]>
Co-authored-by: Blocka <[email protected]>
Co-authored-by: zrant <[email protected]>
Co-authored-by: kenchung285 <[email protected]>
Co-authored-by: fscnick <[email protected]>
Co-authored-by: ryanaoleary <[email protected]>
2025-02-10 11:45:16 -05:00
..
2025-02-10 11:45:16 -05:00

Overview

This python client library provide APIs to handle raycluster from your python application.

Prerequisites

It is assumed that your k8s cluster in already setup. Your kubectl configuration is expected to be in ~/.kube/config if you are running the code directly from you terminal.

It is also expected that the kuberay operator is installed. Installation instructions are here.

Usage

There are multiple levels of using the api with increasing levels of complexity.

director

This is the easiest form of using the api to create rayclusters with predefined cluster sizes

my_kuberay_api = kuberay_cluster_api.RayClusterApi()

my_cluster_director = kuberay_cluster_builder.Director()

cluster0 = my_cluster_director.build_small_cluster(name="new-cluster0")

if cluster0:
    my_kuberay_api.create_ray_cluster(body=cluster0)

the director create the custer definition, and the custer_api acts as the http client sending the create (post) request to the k8s api-server

cluster_builder

The builder allows you to build the cluster piece by piece, you are can customize more the values of the cluster definition

cluster1 = (
        my_cluster_builder.build_meta(name="new-cluster1")
        .build_head()
        .build_worker(group_name="workers", replicas=3)
        .get_cluster()
    )

if not my_cluster_builder.succeeded:
    return

my_kuberay_api.create_ray_cluster(body=cluster1)

cluster_utils

the cluster_utils gives you even more options to modify your cluster definition, add/remove worker groups, change replicas in a worker group, duplicate a worker group, etc.

my_Cluster_utils = kuberay_cluster_utils.ClusterUtils()

cluster_to_patch, succeeded = my_Cluster_utils.update_worker_group_replicas(
        cluster2, group_name="workers", max_replicas=4, min_replicas=1, replicas=2
    )

if succeeded:
        my_kuberay_api.patch_ray_cluster(
            name=cluster_to_patch["metadata"]["name"], ray_patch=cluster_to_patch
        )

cluster_api

Finally, the cluster_api is the one you always use to implement your cluster change in k8s. You can use it with raw JSON if you wish. The director/cluster_builder/cluster_utils are just tools to shield the user from using raw JSON.

Code Organization

clients/
└── python-client
    ├── LICENSE
    ├── README.md
    ├── examples
    │   ├── complete-example.py
    │   ├── use-builder.py
    │   ├── use-director.py
    │   ├── use-raw-config_map_with-api.py
    │   ├── use-raw-with-api.py
    │   └── use-utils.py
    ├── pyproject.toml
    ├── python_client
    │   ├── __init__.py
    │   ├── constants.py
    │   ├── kuberay_cluster_api.py
    │   └── utils
    │       ├── __init__.py
    │       ├── kuberay_cluster_builder.py
    │       └── kuberay_cluster_utils.py
    ├── python_client_test
    │   ├── README.md
    │   ├── test_api.py
    │   ├── test_director.py
    │   └── test_utils.py
    └── setup.cfg

For developers

make sure you have installed setuptool

pip install -U pip setuptools

run the pip command

from the directory path/to/kuberay/clients/python-client

pip install -e .

to uninstall the module run

pip uninstall python-client

For testing run

python -m unittest discover 'path/to/kuberay/clients/python-client/python_client_test/'