mirror of
https://github.com/hanzo-ml/kuberay.git
synced 2026-10-30 19:48:47 +00:00
+6








a5adbbe18b
* [RayService] More envtests that follow the most common scenario in the RayService code path (#2880) Signed-off-by: Rueian <[email protected]> * [RayService] Remove outdated env tests (#2886) Signed-off-by: kaihsun <[email protected]> * [RayService] Refactor envtests (#2888) * [docs][ray-operator] fix typo in Golang version (#2893) Project uses Golang 1.22 not 1.20. Signed-off-by: David Xia <[email protected]> * [Fix][kubectl-plugin] Fix no context nil error SIGSEGV in tests (#2892) Signed-off-by: Chi-Sheng Liu <[email protected]> * [CI] Enable testifylint rule (#2896) Signed-off-by: Chi-Sheng Liu <[email protected]> * [CI] Enable `testifylint` `error-nil` rule (#2907) * [kubectl-plugin][feat] support specifying number of head GPUs (#2895) when creating a RayCluster with `kubectl ray create cluster NAME --head-gpu N`. Similar to the `--worker-gpu` switch. Signed-off-by: David Xia <[email protected]> * Use webhook.CustomValidator instead of deprecated webhook.Validator. (#2803) * Use webhook.CustomValidator instead of deprecated webhook.Validator. * Move RayClusterWebhook to pkg/webhook/v1. * [kubectl-plugin] update context error messages (#2891) to tell user they can use `--context` to set the K8s context. Add tests Signed-off-by: David Xia <[email protected]> Co-authored-by: Chi-Sheng Liu <[email protected]> * [Fix][kubectl-plugin] make tests use a temporary kube config (#2894) file that's not at the default path. Right now tests fail if there's a K8s current context set with a command like `kubectl config use-context my-context`. This change allows tests to pass regardless of current context. Signed-off-by: David Xia <[email protected]> * [CI] Enable testifylint formatter rule (#2915) Signed-off-by: 400Ping <[email protected]> * [CI] Enable `testifylint` `empty` rule (#2908) * [CI] Enable testifylint bool-compare rule (#2911) Signed-off-by: 400Ping <[email protected]> * [CI] Auto download golang tools in pre-commit (#2917) Signed-off-by: Chi-Sheng Liu <[email protected]> * [RayService] a safeguard for preventing overriding the pending cluster during a upgrade (#2887) Signed-off-by: Rueian <[email protected]> * [RayService] Refactor unit tests for ShouldPrepareNewCluster (#2928) Signed-off-by: kaihsun <[email protected]> * [chore][kubectl-plugin] use consistent capitalization (#2922) in comments and console messages. * capitalize "Ray" when used as a proper noun * capitalize Ray K8s CRDs like "RayJob" and "RayCluster" * capitalize acronyms like "YAML" * fix some minor typos No functional changes. Signed-off-by: David Xia <[email protected]> * [CI] Enable `testifylint` `require-error` rule (#2909) * [kubectl-plugin] support general `kubectl` switches like `--context` (#2883) * [CI] Fix lint error (require-error) (#2931) Signed-off-by: Chi-Sheng Liu <[email protected]> * [CI] Enable `testifylint` `float-compare` rule (#2910) * [docs] move pre-commit instructions to main dev docs (#2921) * [CI] Enable `testifylint` `expected-actual` rule (#2914) * [CI] Generate CRD json schema separately in pre-commit (#2930) * [release][1/N] Update YAMLs from Ray 2.9 to Ray 2.41 (#2934) * Delete `[raycluster|rayjob|rayservice]_types_test.go` unnecessary tests (#2935) * [release][2/N] Update RayCluster Helm chart from Ray 2.9 to Ray 2.41 (#2936) Signed-off-by: kaihsun <[email protected]> * [release][3/N] Update RayService e2e tests YAML files from Ray 2.9 to Ray 2.41 (#2937) * [release][4/N] Update Ray images / versions in kubectl plugin (#2938) Signed-off-by: kaihsun <[email protected]> * [release][5/N] Update some RayJob YAMLs from Ray 2.9 to Ray 2.41 (#2941) Signed-off-by: kaihsun <[email protected]> * [release][6/N] Remove unnecessary YAMLs (#2946) Signed-off-by: kaihsun <[email protected]> * [docs][kubectl-plugin] add dev docs (#2912) Signed-off-by: David Xia <[email protected]> Co-authored-by: Chi-Sheng Liu <[email protected]> * [CI] Add shellcheck and fix error of it (#2933) * [chore][kubectl-plugin] use better test assertions (#2955) Fix two places. * use `assert.EqualError()` instead of `assert.Error()` to check the error string is what we expect * use testify assertion instead of string checking for better error output Signed-off-by: David Xia <[email protected]> * [chore] add Markdown linting pre-commit hook (#2953) Ignore most rules we violate for now. Fix these two violations. ``` pre-commit run markdownlint --all-files --show-diff-on-failure ... clients/python-client/README.md:106 MD001/heading-increment Heading levels should only increment by one level at a time [Expected: h3; Actual: h4] CHANGELOG.md:420:1 MD030/list-marker-space Spaces after list markers [Expected: 1; Actual: 2] ``` Signed-off-by: David Xia <[email protected]> * [chore][kubectl-plugin] use consistent capitalization (#2950) in comments and console messages. capitalize "Ray" when used as a proper noun. Similar to #2922. Signed-off-by: David Xia <[email protected]> * [Fix][RayJob] Invalid quote for RayJob submitter (#2949) Closes: ray-project/kuberay#2943 Signed-off-by: Chi-Sheng Liu <[email protected]> * [release][7/N] Update RayService YAMLs (#2956) * [CI] Enable testifylint len rule (#2945) Signed-off-by: LeoLiao123 <[email protected]> * [docs][kubectl-plugin] improve help messages (#2952) Signed-off-by: David Xia <[email protected]> * [kubectl-plugin] Fix panic when GPU resource is not set (#2954) Signed-off-by: win5923 <[email protected]> * [release][8/N] Upgrade Stable Diffusion RayService to Ray 2.41 (#2960) * [docs][kubectl-plugin] fix incorrect example commands (#2951) Signed-off-by: David Xia <[email protected]> Co-authored-by: Chi-Sheng Liu <[email protected]> * [fix][kubectl-plugin] set worker group CPU limit (#2958) when creating a new worker group with `kubectl ray create workergroup`. Write a unit test. I noticed we are setting resource limits equal to resource requests everywhere else but in this command. I have a K8s [LimitRange] that prevented the creation of these worker Pods that had CPU limit defaulting to less than their CPU requests. Describing the RayCluster showed this warning event. `Failed to create worker Pod hyperkube/, Pod "dxia-test-other-group-worker-pm2sh" is invalid: spec.containers[0].resources.requests: Invalid value: "2": must be less than or equal to cpu limit of 250m` Signed-off-by: David Xia <[email protected]> [LimitRange]: https://kubernetes.io/docs/concepts/policy/limit-range/ * [RayJob] Deflaky RayJob e2e tests (#2963) Signed-off-by: kaihsun <[email protected]> * [RayService] Deflaky RayService envtest (#2962) Signed-off-by: kaihsun <[email protected]> * [release][9/N] Update text summarizer RayService to Ray 2.41 (#2961) * [RayService] adapter vllm 0.6.1.post2 (#2823) * adapter vllm 0.6.1.post2 * fix var define * Unify the cpu Settings in serve.py in service.yaml, all set to 1 * Maintain the configuration with vllm0.5x * [Release] Upgrade ray-job.batch-inference.yaml image to 2.41 (#2971) Signed-off-by: Chi-Sheng Liu <[email protected]> * [chore][docs] enable Markdownlint rule MD010 (#2975) * [CI] Change Pre-commit-shellcheck-to-shellcheck-py (#2974) Signed-off-by: owenowenisme <[email protected]> * [release] Update Yunikorn YAML file to Ray 2.41 (#2969) Signed-off-by: Cheng-Yeh Chung <[email protected]> * [release] Update YuniKorn YAML files to Ray 2.41 (#2976) Signed-off-by: win5923 <[email protected]> * [chore][docs] enable Markdownlint rule MD004 (#2973) [Unordered list style][1] [1]: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md004---unordered-list-style * [Test] Use GcsFaultToleranceOptions in test and backward compatibility (#2972) * Update samples to use Ray 2.41.0 images (#2964) Signed-off-by: Andrew Sy Kim <[email protected]> * Update TPU Ray CR manifests to use Ray 2.41.0 (#2965) * [Refactor] Use constants for image tag, image repo, and versions in golang to avoid hard-coded strings (#2978) --------- Signed-off-by: Rueian <[email protected]> Signed-off-by: kaihsun <[email protected]> Signed-off-by: David Xia <[email protected]> Signed-off-by: Chi-Sheng Liu <[email protected]> Signed-off-by: 400Ping <[email protected]> Signed-off-by: LeoLiao123 <[email protected]> Signed-off-by: win5923 <[email protected]> Signed-off-by: owenowenisme <[email protected]> Signed-off-by: Cheng-Yeh Chung <[email protected]> Signed-off-by: Andrew Sy Kim <[email protected]> Co-authored-by: Rueian <[email protected]> Co-authored-by: Kai-Hsun Chen <[email protected]> Co-authored-by: David Xia <[email protected]> Co-authored-by: Chi-Sheng Liu <[email protected]> Co-authored-by: David Xia <[email protected]> Co-authored-by: Mykhailo Bobrovskyi <[email protected]> Co-authored-by: Ping <[email protected]> Co-authored-by: Owen Lin <[email protected]> Co-authored-by: Leo Liao <[email protected]> Co-authored-by: Blocka <[email protected]> Co-authored-by: zrant <[email protected]> Co-authored-by: kenchung285 <[email protected]> Co-authored-by: fscnick <[email protected]> Co-authored-by: ryanaoleary <[email protected]>
551 lines
16 KiB
Go
551 lines
16 KiB
Go
package log
|
|
|
|
import (
|
|
"archive/tar"
|
|
"bytes"
|
|
"context"
|
|
"io"
|
|
"net/http"
|
|
"net/url"
|
|
"os"
|
|
"path/filepath"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/ray-project/kuberay/kubectl-plugin/pkg/util"
|
|
"github.com/spf13/cobra"
|
|
"github.com/stretchr/testify/assert"
|
|
"github.com/stretchr/testify/require"
|
|
v1 "k8s.io/api/core/v1"
|
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
|
"k8s.io/cli-runtime/pkg/genericclioptions"
|
|
"k8s.io/cli-runtime/pkg/genericiooptions"
|
|
"k8s.io/cli-runtime/pkg/resource"
|
|
"k8s.io/client-go/rest"
|
|
restclient "k8s.io/client-go/rest"
|
|
"k8s.io/client-go/rest/fake"
|
|
"k8s.io/client-go/tools/remotecommand"
|
|
cmdtesting "k8s.io/kubectl/pkg/cmd/testing"
|
|
"k8s.io/kubectl/pkg/scheme"
|
|
)
|
|
|
|
// Mocked NewSPDYExecutor
|
|
var fakeNewSPDYExecutor = func(method string, url *url.URL, inputbuf *bytes.Buffer) (remotecommand.Executor, error) {
|
|
return &fakeExecutor{method: method, url: url, buf: inputbuf}, nil
|
|
}
|
|
|
|
type fakeExecutor struct {
|
|
url *url.URL
|
|
buf *bytes.Buffer
|
|
method string
|
|
}
|
|
|
|
// Stream is needed for implementing remotecommand.Execute
|
|
func (f *fakeExecutor) Stream(_ remotecommand.StreamOptions) error {
|
|
return nil
|
|
}
|
|
|
|
// downloadRayLogFiles uses StreamWithContext so this is the real function that we are mocking
|
|
func (f *fakeExecutor) StreamWithContext(_ context.Context, options remotecommand.StreamOptions) error {
|
|
_, err := io.Copy(options.Stdout, f.buf)
|
|
return err
|
|
}
|
|
|
|
// createFakeTarFile creates the fake tar file that will be used for testing
|
|
func createFakeTarFile() (*bytes.Buffer, error) {
|
|
// Create a buffer to hold the tar archive
|
|
tarbuff := new(bytes.Buffer)
|
|
|
|
// Create a tar writer
|
|
tw := tar.NewWriter(tarbuff)
|
|
|
|
// Define the files/directories to include
|
|
files := []struct {
|
|
ModTime time.Time
|
|
Name string
|
|
Body string
|
|
IsDir bool
|
|
Mode int64
|
|
}{
|
|
{time.Now(), "/", "", true, 0o755},
|
|
{time.Now(), "file1.txt", "This is the content of file1.txt\n", false, 0o644},
|
|
{time.Now(), "file2.txt", "Content of file2.txt inside subdir\n", false, 0o644},
|
|
}
|
|
|
|
// Add each file/directory to the tar archive
|
|
for _, file := range files {
|
|
hdr := &tar.Header{
|
|
Name: file.Name,
|
|
Mode: file.Mode,
|
|
ModTime: file.ModTime,
|
|
Size: int64(len(file.Body)),
|
|
}
|
|
if file.IsDir {
|
|
hdr.Typeflag = tar.TypeDir
|
|
} else {
|
|
hdr.Typeflag = tar.TypeReg
|
|
}
|
|
|
|
// Write the header
|
|
if err := tw.WriteHeader(hdr); err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
// Write the file content (if not a directory)
|
|
if !file.IsDir {
|
|
if _, err := tw.Write([]byte(file.Body)); err != nil {
|
|
return nil, err
|
|
}
|
|
}
|
|
}
|
|
|
|
// Close the tar writer
|
|
if err := tw.Close(); err != nil {
|
|
return nil, err
|
|
}
|
|
return tarbuff, nil
|
|
}
|
|
|
|
type FakeRemoteExecutor struct{}
|
|
|
|
func (dre *FakeRemoteExecutor) CreateExecutor(_ *rest.Config, url *url.URL) (remotecommand.Executor, error) {
|
|
return fakeNewSPDYExecutor("GET", url, new(bytes.Buffer))
|
|
}
|
|
|
|
func TestRayClusterLogComplete(t *testing.T) {
|
|
cmd := &cobra.Command{Use: "log"}
|
|
|
|
tests := []struct {
|
|
name string
|
|
nodeType string
|
|
expectedResourceType util.ResourceType
|
|
expectedResourceName string
|
|
expectedNodeType string
|
|
args []string
|
|
hasErr bool
|
|
}{
|
|
{
|
|
name: "valid request with RayCluster with empty nodetype input",
|
|
expectedResourceType: util.RayCluster,
|
|
expectedResourceName: "test-raycluster",
|
|
expectedNodeType: "all",
|
|
args: []string{"test-raycluster"},
|
|
hasErr: false,
|
|
},
|
|
{
|
|
name: "valid request with RayCluster",
|
|
expectedResourceType: util.RayCluster,
|
|
expectedResourceName: "test-raycluster",
|
|
args: []string{"rayCluster/test-raycluster"},
|
|
expectedNodeType: "all",
|
|
hasErr: false,
|
|
},
|
|
{
|
|
name: "valid request with RayService",
|
|
expectedResourceType: util.RayService,
|
|
expectedResourceName: "test-rayservice",
|
|
args: []string{"rayservice/test-rayservice"},
|
|
expectedNodeType: "all",
|
|
hasErr: false,
|
|
},
|
|
{
|
|
name: "valid request with RayJob",
|
|
expectedResourceType: util.RayJob,
|
|
expectedResourceName: "test-rayJob",
|
|
args: []string{"rayJob/test-rayJob"},
|
|
expectedNodeType: "all",
|
|
hasErr: false,
|
|
},
|
|
{
|
|
name: "invalid args (no args)",
|
|
args: []string{},
|
|
hasErr: true,
|
|
},
|
|
{
|
|
name: "invalid args (too many args)",
|
|
args: []string{"raycluster/test-raycluster", "extra-arg"},
|
|
hasErr: true,
|
|
},
|
|
{
|
|
name: "invalid args (no resource type)",
|
|
args: []string{"/test-resource"},
|
|
hasErr: true,
|
|
},
|
|
{
|
|
name: "invalid args (no resource name)",
|
|
args: []string{"raycluster/"},
|
|
hasErr: true,
|
|
},
|
|
{
|
|
name: "invalid args (invalid resource type)",
|
|
args: []string{"invalid-type/test-resource"},
|
|
hasErr: true,
|
|
},
|
|
}
|
|
|
|
for _, tc := range tests {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
testStreams, _, _, _ := genericclioptions.NewTestIOStreams()
|
|
fakeClusterLogOptions := NewClusterLogOptions(testStreams)
|
|
err := fakeClusterLogOptions.Complete(cmd, tc.args)
|
|
if tc.hasErr {
|
|
require.Error(t, err)
|
|
} else {
|
|
require.NoError(t, err)
|
|
assert.Equal(t, tc.expectedResourceType, fakeClusterLogOptions.ResourceType)
|
|
assert.Equal(t, tc.expectedResourceName, fakeClusterLogOptions.ResourceName)
|
|
assert.Equal(t, tc.expectedNodeType, fakeClusterLogOptions.nodeType)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestRayClusterLogValidate(t *testing.T) {
|
|
testStreams, _, _, _ := genericclioptions.NewTestIOStreams()
|
|
|
|
testNS, testContext, testBT, testImpersonate := "test-namespace", "test-context", "test-bearer-token", "test-person"
|
|
|
|
fakeDir := t.TempDir()
|
|
|
|
kubeConfigWithCurrentContext, err := util.CreateTempKubeConfigFile(t, testContext)
|
|
require.NoError(t, err)
|
|
|
|
kubeConfigWithoutCurrentContext, err := util.CreateTempKubeConfigFile(t, "")
|
|
require.NoError(t, err)
|
|
|
|
// Initialize the fake config flag with the fake kubeconfig and values
|
|
fakeConfigFlags := &genericclioptions.ConfigFlags{
|
|
Namespace: &testNS,
|
|
Context: &testContext,
|
|
KubeConfig: &kubeConfigWithCurrentContext,
|
|
BearerToken: &testBT,
|
|
Impersonate: &testImpersonate,
|
|
ImpersonateGroup: &[]string{"fake-group"},
|
|
}
|
|
|
|
tests := []struct {
|
|
name string
|
|
opts *ClusterLogOptions
|
|
expect string
|
|
expectError string
|
|
}{
|
|
{
|
|
name: "Test validation when no context is set",
|
|
opts: &ClusterLogOptions{
|
|
configFlags: &genericclioptions.ConfigFlags{
|
|
KubeConfig: &kubeConfigWithoutCurrentContext,
|
|
},
|
|
outputDir: fakeDir,
|
|
ResourceName: "fake-cluster",
|
|
nodeType: "head",
|
|
ioStreams: &testStreams,
|
|
},
|
|
expectError: "no context is currently set, use \"--context\" or \"kubectl config use-context <context>\" to select a new one",
|
|
},
|
|
{
|
|
name: "no error when kubeconfig has current context and --context switch isn't set",
|
|
opts: &ClusterLogOptions{
|
|
// Use fake config to bypass the config flag checks
|
|
configFlags: &genericclioptions.ConfigFlags{
|
|
KubeConfig: &kubeConfigWithCurrentContext,
|
|
},
|
|
outputDir: fakeDir,
|
|
ResourceName: "fake-cluster",
|
|
nodeType: "head",
|
|
ioStreams: &testStreams,
|
|
},
|
|
},
|
|
{
|
|
name: "no error when kubeconfig has no current context and --context switch is set",
|
|
opts: &ClusterLogOptions{
|
|
configFlags: &genericclioptions.ConfigFlags{
|
|
KubeConfig: &kubeConfigWithoutCurrentContext,
|
|
Context: &testContext,
|
|
},
|
|
outputDir: fakeDir,
|
|
ResourceName: "fake-cluster",
|
|
nodeType: "head",
|
|
ioStreams: &testStreams,
|
|
},
|
|
},
|
|
{
|
|
name: "Test validation when node type is `random-string`",
|
|
opts: &ClusterLogOptions{
|
|
// Use fake config to bypass the config flag checks
|
|
configFlags: fakeConfigFlags,
|
|
outputDir: fakeDir,
|
|
ResourceName: "fake-cluster",
|
|
nodeType: "random-string",
|
|
ioStreams: &testStreams,
|
|
},
|
|
expectError: "unknown node type `random-string`",
|
|
},
|
|
{
|
|
name: "Successful validation call",
|
|
opts: &ClusterLogOptions{
|
|
// Use fake config to bypass the config flag checks
|
|
configFlags: fakeConfigFlags,
|
|
outputDir: fakeDir,
|
|
ResourceName: "fake-cluster",
|
|
nodeType: "head",
|
|
ioStreams: &testStreams,
|
|
},
|
|
expectError: "",
|
|
},
|
|
{
|
|
name: "Validate output directory when no out-dir is set.",
|
|
opts: &ClusterLogOptions{
|
|
// Use fake config to bypass the config flag checks
|
|
configFlags: fakeConfigFlags,
|
|
outputDir: "",
|
|
ResourceName: "fake-cluster",
|
|
nodeType: "head",
|
|
ioStreams: &testStreams,
|
|
},
|
|
expectError: "",
|
|
},
|
|
{
|
|
name: "Failed validation call with output directory not exist",
|
|
opts: &ClusterLogOptions{
|
|
// Use fake config to bypass the config flag checks
|
|
configFlags: fakeConfigFlags,
|
|
outputDir: "randomPath-here",
|
|
ResourceName: "fake-cluster",
|
|
nodeType: "head",
|
|
ioStreams: &testStreams,
|
|
},
|
|
expectError: "Directory does not exist. Failed with: stat randomPath-here: no such file or directory",
|
|
},
|
|
{
|
|
name: "Failed validation call with output directory is file",
|
|
opts: &ClusterLogOptions{
|
|
// Use fake config to bypass the config flag checks
|
|
configFlags: fakeConfigFlags,
|
|
outputDir: kubeConfigWithCurrentContext,
|
|
ResourceName: "fake-cluster",
|
|
nodeType: "head",
|
|
ioStreams: &testStreams,
|
|
},
|
|
expectError: "Path is not a directory. Please input a directory and try again",
|
|
},
|
|
}
|
|
|
|
for _, tc := range tests {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
err := tc.opts.Validate()
|
|
if tc.expectError != "" {
|
|
require.Error(t, err, tc.expectError)
|
|
} else {
|
|
if tc.opts.outputDir == "" {
|
|
assert.Equal(t, tc.opts.ResourceName, tc.opts.outputDir)
|
|
}
|
|
assert.True(t, err == nil)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestRayClusterLogRun(t *testing.T) {
|
|
tf := cmdtesting.NewTestFactory().WithNamespace("test")
|
|
defer tf.Cleanup()
|
|
|
|
fakeDir, err := os.MkdirTemp("", "fake-directory")
|
|
require.NoError(t, err)
|
|
defer os.RemoveAll(fakeDir)
|
|
|
|
testStreams, _, _, _ := genericiooptions.NewTestIOStreams()
|
|
|
|
fakeClusterLogOptions := NewClusterLogOptions(testStreams)
|
|
// Uses the mocked executor
|
|
fakeClusterLogOptions.Executor = &FakeRemoteExecutor{}
|
|
fakeClusterLogOptions.ResourceName = "test-cluster"
|
|
fakeClusterLogOptions.outputDir = fakeDir
|
|
fakeClusterLogOptions.ResourceType = util.RayCluster
|
|
fakeClusterLogOptions.nodeType = "all"
|
|
|
|
// Create list of fake ray heads
|
|
rayHeadsList := &v1.PodList{
|
|
ListMeta: metav1.ListMeta{
|
|
ResourceVersion: "15",
|
|
},
|
|
Items: []v1.Pod{
|
|
{
|
|
ObjectMeta: metav1.ObjectMeta{
|
|
Name: "test-cluster-kuberay-head-1",
|
|
Namespace: "test",
|
|
Labels: map[string]string{
|
|
"ray.io/group": "headgroup",
|
|
"ray.io/clusters": "test-cluster",
|
|
},
|
|
},
|
|
Spec: v1.PodSpec{
|
|
Containers: []v1.Container{
|
|
{
|
|
Name: "mycontainer",
|
|
Image: "nginx:latest",
|
|
},
|
|
},
|
|
},
|
|
Status: v1.PodStatus{
|
|
Phase: v1.PodRunning,
|
|
PodIP: "10.0.0.1",
|
|
},
|
|
},
|
|
{
|
|
ObjectMeta: metav1.ObjectMeta{
|
|
Name: "test-cluster-kuberay-head-2",
|
|
Namespace: "test",
|
|
Labels: map[string]string{
|
|
"ray.io/group": "headgroup",
|
|
"ray.io/clusters": "test-cluster",
|
|
},
|
|
},
|
|
Spec: v1.PodSpec{
|
|
Containers: []v1.Container{
|
|
{
|
|
Name: "anothercontainer",
|
|
Image: "busybox:latest",
|
|
},
|
|
},
|
|
},
|
|
Status: v1.PodStatus{
|
|
Phase: v1.PodPending,
|
|
},
|
|
},
|
|
},
|
|
}
|
|
|
|
// create logs for multiple head pods and turn them into io streams so they can be returned with the fake client
|
|
fakeLogs := []string{
|
|
"This is some fake log data for first pod.\nStill first pod logs\n",
|
|
"This is some fake log data for second pod.\nStill second pod logs\n",
|
|
}
|
|
logReader1 := io.NopCloser(bytes.NewReader([]byte(fakeLogs[0])))
|
|
logReader2 := io.NopCloser(bytes.NewReader([]byte(fakeLogs[1])))
|
|
|
|
// fakes the client and the REST calls.
|
|
codec := scheme.Codecs.LegacyCodec(scheme.Scheme.PrioritizedVersionsAllGroups()...)
|
|
tf.Client = &fake.RESTClient{
|
|
GroupVersion: v1.SchemeGroupVersion,
|
|
NegotiatedSerializer: resource.UnstructuredPlusDefaultContentConfig().NegotiatedSerializer,
|
|
Client: fake.CreateHTTPClient(func(req *http.Request) (*http.Response, error) {
|
|
switch req.URL.Path {
|
|
case "/api/v1/pods":
|
|
return &http.Response{StatusCode: http.StatusOK, Header: cmdtesting.DefaultHeader(), Body: cmdtesting.ObjBody(codec, rayHeadsList)}, nil
|
|
case "/api/v1/namespaces/test/pods/test-cluster-kuberay-head-1/log":
|
|
return &http.Response{StatusCode: http.StatusOK, Header: cmdtesting.DefaultHeader(), Body: logReader1}, nil
|
|
case "/api/v1/namespaces/test/pods/test-cluster-kuberay-head-2/log":
|
|
return &http.Response{StatusCode: http.StatusOK, Header: cmdtesting.DefaultHeader(), Body: logReader2}, nil
|
|
default:
|
|
t.Fatalf("request url: %#v,and request: %#v", req.URL, req)
|
|
return nil, nil
|
|
}
|
|
}),
|
|
}
|
|
|
|
tf.ClientConfigVal = &restclient.Config{
|
|
ContentConfig: restclient.ContentConfig{GroupVersion: &v1.SchemeGroupVersion},
|
|
}
|
|
|
|
err = fakeClusterLogOptions.Run(context.Background(), tf)
|
|
require.NoError(t, err)
|
|
|
|
// Check that the two directories are there
|
|
entries, err := os.ReadDir(fakeDir)
|
|
require.NoError(t, err)
|
|
assert.Len(t, entries, 2)
|
|
|
|
assert.Equal(t, "test-cluster-kuberay-head-1", entries[0].Name())
|
|
assert.Equal(t, "test-cluster-kuberay-head-2", entries[1].Name())
|
|
|
|
// Check the first directory for the logs
|
|
for ind, entry := range entries {
|
|
currPath := filepath.Join(fakeDir, entry.Name())
|
|
currDir, err := os.ReadDir(currPath)
|
|
require.NoError(t, err)
|
|
assert.Len(t, currDir, 1)
|
|
openfile, err := os.Open(filepath.Join(currPath, "stdout.log"))
|
|
require.NoError(t, err)
|
|
actualContent, err := io.ReadAll(openfile)
|
|
require.NoError(t, err)
|
|
assert.Equal(t, fakeLogs[ind], string(actualContent))
|
|
}
|
|
}
|
|
|
|
func TestDownloadRayLogFiles(t *testing.T) {
|
|
fakeDir, err := os.MkdirTemp("", "fake-directory")
|
|
require.NoError(t, err)
|
|
defer os.RemoveAll(fakeDir)
|
|
|
|
testStreams, _, _, _ := genericiooptions.NewTestIOStreams()
|
|
|
|
fakeClusterLogOptions := NewClusterLogOptions(testStreams)
|
|
fakeClusterLogOptions.ResourceName = "test-cluster"
|
|
fakeClusterLogOptions.outputDir = fakeDir
|
|
|
|
// create fake tar files to test
|
|
fakeTar, err := createFakeTarFile()
|
|
require.NoError(t, err)
|
|
|
|
// Ray head needed for calling the downloadRayLogFiles command
|
|
rayHead := v1.Pod{
|
|
ObjectMeta: metav1.ObjectMeta{
|
|
Name: "test-cluster-kuberay-head-1",
|
|
Namespace: "test",
|
|
Labels: map[string]string{
|
|
"ray.io/group": "headgroup",
|
|
"ray.io/clusters": "test-cluster",
|
|
},
|
|
},
|
|
Spec: v1.PodSpec{
|
|
Containers: []v1.Container{
|
|
{
|
|
Name: "mycontainer",
|
|
Image: "nginx:latest",
|
|
},
|
|
},
|
|
},
|
|
Status: v1.PodStatus{
|
|
Phase: v1.PodRunning,
|
|
PodIP: "10.0.0.1",
|
|
},
|
|
}
|
|
|
|
executor, _ := fakeNewSPDYExecutor("GET", &url.URL{}, fakeTar)
|
|
|
|
err = fakeClusterLogOptions.downloadRayLogFiles(context.Background(), executor, rayHead)
|
|
require.NoError(t, err)
|
|
|
|
entries, err := os.ReadDir(fakeDir)
|
|
require.NoError(t, err)
|
|
assert.Len(t, entries, 1)
|
|
|
|
// Assert the files
|
|
assert.True(t, entries[0].IsDir())
|
|
files, err := os.ReadDir(filepath.Join(fakeDir, entries[0].Name()))
|
|
require.NoError(t, err)
|
|
assert.Len(t, files, 2)
|
|
|
|
expectedfileoutput := []struct {
|
|
Name string
|
|
Body string
|
|
}{
|
|
{"file1.txt", "This is the content of file1.txt\n"},
|
|
{"file2.txt", "Content of file2.txt inside subdir\n"},
|
|
}
|
|
|
|
// Goes through and check the temp directory with the downloaded files
|
|
for ind, file := range files {
|
|
fileInfo, err := file.Info()
|
|
require.NoError(t, err)
|
|
curr := expectedfileoutput[ind]
|
|
|
|
assert.Equal(t, curr.Name, fileInfo.Name())
|
|
openfile, err := os.Open(filepath.Join(fakeDir, entries[0].Name(), file.Name()))
|
|
require.NoError(t, err)
|
|
actualContent, err := io.ReadAll(openfile)
|
|
require.NoError(t, err)
|
|
assert.Equal(t, curr.Body, string(actualContent))
|
|
}
|
|
}
|