* Use JAX image published by Kubeflow
Signed-off-by: Sandipan Panda <[email protected]>
* Update README to include JAX API Definition
Signed-off-by: Sandipan Panda <[email protected]>
---------
Signed-off-by: Sandipan Panda <[email protected]>
* Introduce ManagedBy field to RunPolicy that is used by each Kubeflow Job Spec
Signed-off-by: Michal Szadkowski <[email protected]>
* Update Kubeflow JOb manifests
Signed-off-by: Michal Szadkowski <[email protected]>
* Update Kubeflow Jobs Reconcile to use ManagedBy field to decide if skip the process
Signed-off-by: Michal Szadkowski <[email protected]>
* job controller test
Signed-off-by: Michal Szadkowski <[email protected]>
* spec validation webhook
Signed-off-by: Michal Szadkowski <[email protected]>
* add manageBy maxLenght const
Signed-off-by: Michal Szadkowski <[email protected]>
* generate new manifest
Signed-off-by: Michal Szadkowski <[email protected]>
* revert webhook formatting
Signed-off-by: Michal Szadkowski <[email protected]>
* Move allowed controllers constants in one place
Signed-off-by: Michal Szadkowski <[email protected]>
* Make validatation for allowed managedBy values
Signed-off-by: Michal Szadkowski <[email protected]>
* Update after controllers constants move
Signed-off-by: Michal Szadkowski <[email protected]>
* Update jobs controller tests
Signed-off-by: Michal Szadkowski <[email protected]>
* Update validateManagedBy webhook
Signed-off-by: Michal Szadkowski <[email protected]>
* Remove validation for the length of ManagedBy field
Signed-off-by: Michal Szadkowski <[email protected]>
* Update after code review
Signed-off-by: Michal Szadkowski <[email protected]>
* Update ManagedBy comment
Signed-off-by: Michal Szadkowski <[email protected]>
* E2E tests for managedBy
Signed-off-by: Michal Szadkowski <[email protected]>
* Update generated files and manifests
Signed-off-by: Michal Szadkowski <[email protected]>
* Rework after code review
Signed-off-by: Michal Szadkowski <[email protected]>
* Revert kustomization change
Signed-off-by: Michal Szadkowski <[email protected]>
* Update job_test and logging
Signed-off-by: Michal Szadkowski <[email protected]>
* Provide immutability check for ManagedBy
Signed-off-by: Michal Szadkowski <[email protected]>
* Avoid making copy of runPolicy
Signed-off-by: Michal Szadkowski <[email protected]>
* Split RunPolicy validators to Update and Create
Signed-off-by: Michal Szadkowski <[email protected]>
* Fix the naming and call validate always
Signed-off-by: Michal Szadkowski <[email protected]>
* Update tests
Signed-off-by: Michal Szadkowski <[email protected]>
---------
Signed-off-by: Michal Szadkowski <[email protected]>
* [SDK] Add resources for create Job API
* Fix unbound var
* Assign values in get pod template
* Add torchrun issue
* Test to create PyTorchJob from Image
* Fix e2e to create from image
* Fix condition
* Modify check test conditions
* Add test to create PyTorchJob from func
* Use unique Job name for each test
* Fix job name in test
* Describe PyTorchJob when test fails
* Skip test for gang scheduling
* Add Flake and Black Lint
* Change SDK APIs
* Update E2E tests
* Fix a few function parameters
* Fix black format
* Fix a few comments
* Fix conftest location
* Fix Job kind in tests
* Fix client creation in test
* Fix namespace arg in get_job_conditions
* Update SDK examples with the latest changes
* Rename SDK examples
* Fix black action
* Update checkout action version
Co-authored-by: Yuki Iwai <[email protected]>
* Use Black 23.9.1 version
* Fix GitHub Action for Black
* Add unit test to create PyTorchJob from func
* Rename timeout to wait_timeout
* Validate that Job is not set with other input parameters
* Update black in developer guide
* Remove pip_index_url validation
* Use locals to verify input
* Print Job info when E2E fails
* Remove duplicated delete
---------
Co-authored-by: Yuki Iwai <[email protected]>
* fix pytorch WORLD_SIZE env inconsistent
* add correct env node_rank, nnodes for torch
* restore elastic nnproc
* use string for nproc_per_node
* add defaults in api
* add validation for two nproc_per_node, use auto for defaulter
* add ut for defaults and validation
* fix ut
* add doc for nproc_per_node
* Support for k8s v1.25 in CI
* Support for k8s v1.25 in CI
* Change k8s api to v1.25
* Upgrade golangci-lint version
* Add changelog
* Update CHANGELOG.md
* Update Changelog
* Merge common repo
* Avoid to depend on local env when installing the code-generators (#1810)
Signed-off-by: Yuki Iwai <[email protected]>
* Make scheduler-plugins the default gang scheduler. (#1747)
Signed-off-by: Syulin7 <[email protected]>
* Fix tests
* Fix merge conflicts
* Fix CI issues
* Fix CI issues
* Fix review comments
* Add contributors in Readme file
* Fix review comments
* Fix review comments
---------
Signed-off-by: Yuki Iwai <[email protected]>
Signed-off-by: Syulin7 <[email protected]>
Co-authored-by: Yuki Iwai <[email protected]>
Co-authored-by: yu lin <[email protected]>
* docker file to create conformance tests container
* add resources to containers created by test jobs
* use setup.py to install dependencies
* update resources to fix the test in CI
* update resource limits
* update resource limits
* build and publish conformance test image in CI
* fix conformance file path
* fix python lint error
* fix python lint error
* use python 3.10
* update mpijob e2e test to use fixtures
* update mxjob e2e test to use fixtures
* update paddlejob e2e test to use fixtures
* update pytorchjob e2e test to use fixtures
* update xgboostjob e2e test to use fixtures
* update tfjob e2e test to use fixtures
* remove training_client fixture
* move conftest to test root dir
* delete only autogenerated test files
* add default namespace option in integration-tests github action
* Remove legacy ksonnet tests
* [SDK] Create Unify Training Client
* Modify E2E tests
* Rename Training to Operator version
* Add missing exception
* Add API server timeout parameter
* Add delete options
* Fix import for V1JobCondition
* Fix container name in e2e tests
* Fix mxnet container
* Import all Kubernetes models
* Update SDK Examples
* Verify other SDK APIs in e2e tests
* Add replica types to const
* Use logging in e2e tests
* Fix logging for status
* Use const for job types
* Support for k8s v1.25 in CI
* Support for k8s v1.25 in CI
* Change k8s api to v1.25
* Upgrade golangci-lint version
* Fixes for HPA in pytorchjob
* Common changes
* support paddlejob
* fix ci
* fix tools diff for ci
* fix ci test
* rm todo
* add e2e test
* fix e2e test
* fix e2e ci
* add paddlejob to kustomize
* set registry to docker.io
* rebase
* add rbac for paddlejob
* add unitest cover container port
* fix rebase
* update Copyright
