mirror of
https://github.com/hanzo-ml/kuberay.git
synced 2026-10-30 19:48:47 +00:00
* feat: add raycronjob editor and viewer role Signed-off-by: machichima <[email protected]> * feat: add feature gate to values.yaml Signed-off-by: machichima <[email protected]> * feat: add raycronjob editor and viewer role to rbac Signed-off-by: machichima <[email protected]> * feat: add raycronjob editor/viewer YAML in helm-render-yaml script Signed-off-by: machichima <[email protected]> --------- Signed-off-by: machichima <[email protected]>
146 lines
4.2 KiB
Go Template
146 lines
4.2 KiB
Go Template
{{ template "chart.header" . }}
|
|
|
|
{{ template "chart.deprecationWarning" . }}
|
|
|
|
{{ template "chart.badgesSection" . }}
|
|
|
|
{{ template "chart.description" . }}
|
|
|
|
{{ template "chart.homepageLine" . }}
|
|
|
|
## Introduction
|
|
|
|
This document provides instructions to install both CRDs (RayCluster, RayJob, RayService, RayCronJob) and
|
|
KubeRay operator with a Helm chart.
|
|
|
|
## Prerequisites
|
|
|
|
- [Kubernetes](https://kubernetes.io)
|
|
- [Helm](https://helm.sh) >= 3
|
|
|
|
Make sure the version of Helm is v3+. Currently, [existing CI tests] are based on Helm v3.4.1 and v3.9.4.
|
|
|
|
```bash
|
|
helm version
|
|
```
|
|
|
|
## Install CRDs and KubeRay operator
|
|
|
|
- Install a stable version via Helm repository (only supports KubeRay v0.4.0+)
|
|
|
|
```sh
|
|
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
|
|
|
|
# Install both CRDs and KubeRay operator v{{ template "chart.version" . }}.
|
|
helm install kuberay-operator kuberay/kuberay-operator --version {{ template "chart.version" . }}
|
|
|
|
# Check the KubeRay operator Pod in `default` namespace
|
|
kubectl get pods
|
|
# NAME READY STATUS RESTARTS AGE
|
|
# kuberay-operator-6fcbb94f64-mbfnr 1/1 Running 0 17s
|
|
```
|
|
|
|
- Install the nightly version
|
|
|
|
```sh
|
|
# Step1: Clone KubeRay repository
|
|
|
|
# Step2: Move to `helm-chart/kuberay-operator`
|
|
|
|
# Step3: Install KubeRay operator
|
|
helm install kuberay-operator .
|
|
```
|
|
|
|
- Install KubeRay operator without installing CRDs
|
|
- In some cases, the installation of the CRDs and the installation of the operator may require
|
|
different levels of admin permissions, so these two installations could be handled as different
|
|
steps by different roles.
|
|
- Use Helm's built-in `--skip-crds` flag to install the operator only.
|
|
See [this document] for more details.
|
|
|
|
```sh
|
|
# Step 1: Install CRDs only (for cluster admin)
|
|
kubectl create -k "github.com/ray-project/kuberay/ray-operator/config/crd?ref=v{{ template "chart.version" . }}&timeout=90s"
|
|
|
|
# Step 2: Install KubeRay operator only. (for developer)
|
|
helm install kuberay-operator kuberay/kuberay-operator --version {{ template "chart.version" . }} --skip-crds
|
|
```
|
|
|
|
## List the chart
|
|
|
|
To list the `my-release` deployment:
|
|
|
|
```sh
|
|
helm ls
|
|
# NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
|
|
# kuberay-operator default 1 2023-09-22 02:57:17.306616331 +0000 UTC deployed kuberay-operator-{{ template "chart.version" . }}
|
|
```
|
|
|
|
## Uninstall the Chart
|
|
|
|
```sh
|
|
# Uninstall the `kuberay-operator` release
|
|
helm uninstall kuberay-operator
|
|
|
|
# The operator Pod should be removed.
|
|
kubectl get pods
|
|
# No resources found in default namespace.
|
|
```
|
|
|
|
## Working with Argo CD
|
|
|
|
If you are using [Argo CD] to manage the operator, you will encounter the issue which complains the
|
|
CRDs too long. Same with [this issue]. The recommended solution is to split the operator into two
|
|
Argo apps, such as:
|
|
|
|
- The first app just for installing the CRDs with `Replace=true` directly, snippet:
|
|
|
|
```yaml
|
|
apiVersion: argoproj.io/v1alpha1
|
|
kind: Application
|
|
metadata:
|
|
name: ray-operator-crds
|
|
spec:
|
|
project: default
|
|
source:
|
|
repoURL: https://github.com/ray-project/kuberay
|
|
targetRevision: v1.0.0-rc.0
|
|
path: helm-chart/kuberay-operator/crds
|
|
destination:
|
|
server: https://kubernetes.default.svc
|
|
syncPolicy:
|
|
syncOptions:
|
|
- Replace=true
|
|
...
|
|
```
|
|
|
|
- The second app that installs the Helm chart with `skipCrds=true` (new feature in Argo CD 2.3.0), snippet:
|
|
|
|
```yaml
|
|
apiVersion: argoproj.io/v1alpha1
|
|
kind: Application
|
|
metadata:
|
|
name: ray-operator
|
|
spec:
|
|
source:
|
|
repoURL: https://github.com/ray-project/kuberay
|
|
targetRevision: v1.0.0-rc.0
|
|
path: helm-chart/kuberay-operator
|
|
helm:
|
|
skipCrds: true
|
|
destination:
|
|
server: https://kubernetes.default.svc
|
|
namespace: ray-operator
|
|
syncPolicy:
|
|
syncOptions:
|
|
- CreateNamespace=true
|
|
...
|
|
```
|
|
|
|
[existing CI tests]: https://github.com/ray-project/kuberay/blob/master/.github/workflows/helm.yaml
|
|
[Argo CD]: https://argoproj.github.io
|
|
[this issue]: https://github.com/prometheus-operator/prometheus-operator/issues/4439
|
|
[this document]: https://helm.sh/docs/chart_best_practices/custom_resource_definitions/
|
|
|
|
{{ template "chart.valuesSection" . }}
|