* Introduce ManagedBy field to RunPolicy that is used by each Kubeflow Job Spec
Signed-off-by: Michal Szadkowski <[email protected]>
* Update Kubeflow JOb manifests
Signed-off-by: Michal Szadkowski <[email protected]>
* Update Kubeflow Jobs Reconcile to use ManagedBy field to decide if skip the process
Signed-off-by: Michal Szadkowski <[email protected]>
* job controller test
Signed-off-by: Michal Szadkowski <[email protected]>
* spec validation webhook
Signed-off-by: Michal Szadkowski <[email protected]>
* add manageBy maxLenght const
Signed-off-by: Michal Szadkowski <[email protected]>
* generate new manifest
Signed-off-by: Michal Szadkowski <[email protected]>
* revert webhook formatting
Signed-off-by: Michal Szadkowski <[email protected]>
* Move allowed controllers constants in one place
Signed-off-by: Michal Szadkowski <[email protected]>
* Make validatation for allowed managedBy values
Signed-off-by: Michal Szadkowski <[email protected]>
* Update after controllers constants move
Signed-off-by: Michal Szadkowski <[email protected]>
* Update jobs controller tests
Signed-off-by: Michal Szadkowski <[email protected]>
* Update validateManagedBy webhook
Signed-off-by: Michal Szadkowski <[email protected]>
* Remove validation for the length of ManagedBy field
Signed-off-by: Michal Szadkowski <[email protected]>
* Update after code review
Signed-off-by: Michal Szadkowski <[email protected]>
* Update ManagedBy comment
Signed-off-by: Michal Szadkowski <[email protected]>
* E2E tests for managedBy
Signed-off-by: Michal Szadkowski <[email protected]>
* Update generated files and manifests
Signed-off-by: Michal Szadkowski <[email protected]>
* Rework after code review
Signed-off-by: Michal Szadkowski <[email protected]>
* Revert kustomization change
Signed-off-by: Michal Szadkowski <[email protected]>
* Update job_test and logging
Signed-off-by: Michal Szadkowski <[email protected]>
* Provide immutability check for ManagedBy
Signed-off-by: Michal Szadkowski <[email protected]>
* Avoid making copy of runPolicy
Signed-off-by: Michal Szadkowski <[email protected]>
* Split RunPolicy validators to Update and Create
Signed-off-by: Michal Szadkowski <[email protected]>
* Fix the naming and call validate always
Signed-off-by: Michal Szadkowski <[email protected]>
* Update tests
Signed-off-by: Michal Szadkowski <[email protected]>
---------
Signed-off-by: Michal Szadkowski <[email protected]>
* adding bug fix in gen-sdk.sh and unit test for create_job function in training client
* code reformatted using black
* move unittest to a different folder
* code review comments changes
* adding pytest github action
* removing 3rd party github pytest action
* Add Flake and Black Lint
* Change SDK APIs
* Update E2E tests
* Fix a few function parameters
* Fix black format
* Fix a few comments
* Fix conftest location
* Fix Job kind in tests
* Fix client creation in test
* Fix namespace arg in get_job_conditions
* Update SDK examples with the latest changes
* Rename SDK examples
* Fix black action
* Update checkout action version
Co-authored-by: Yuki Iwai <[email protected]>
* Use Black 23.9.1 version
* Fix GitHub Action for Black
* Add unit test to create PyTorchJob from func
* Rename timeout to wait_timeout
* Validate that Job is not set with other input parameters
* Update black in developer guide
* Remove pip_index_url validation
* Use locals to verify input
* Print Job info when E2E fails
* Remove duplicated delete
---------
Co-authored-by: Yuki Iwai <[email protected]>
* fix pytorch WORLD_SIZE env inconsistent
* add correct env node_rank, nnodes for torch
* restore elastic nnproc
* use string for nproc_per_node
* add defaults in api
* add validation for two nproc_per_node, use auto for defaulter
* add ut for defaults and validation
* fix ut
* add doc for nproc_per_node
* Support for k8s v1.25 in CI
* Support for k8s v1.25 in CI
* Change k8s api to v1.25
* Upgrade golangci-lint version
* Add changelog
* Update CHANGELOG.md
* Update Changelog
* Merge common repo
* Avoid to depend on local env when installing the code-generators (#1810)
Signed-off-by: Yuki Iwai <[email protected]>
* Make scheduler-plugins the default gang scheduler. (#1747)
Signed-off-by: Syulin7 <[email protected]>
* Fix tests
* Fix merge conflicts
* Fix CI issues
* Fix CI issues
* Fix review comments
* Add contributors in Readme file
* Fix review comments
* Fix review comments
---------
Signed-off-by: Yuki Iwai <[email protected]>
Signed-off-by: Syulin7 <[email protected]>
Co-authored-by: Yuki Iwai <[email protected]>
Co-authored-by: yu lin <[email protected]>
* update mpijob e2e test to use fixtures
* update mxjob e2e test to use fixtures
* update paddlejob e2e test to use fixtures
* update pytorchjob e2e test to use fixtures
* update xgboostjob e2e test to use fixtures
* update tfjob e2e test to use fixtures
* remove training_client fixture
* move conftest to test root dir
* delete only autogenerated test files
* add default namespace option in integration-tests github action
* Remove legacy ksonnet tests
* [SDK] Create Unify Training Client
* Modify E2E tests
* Rename Training to Operator version
* Add missing exception
* Add API server timeout parameter
* Add delete options
* Fix import for V1JobCondition
* Fix container name in e2e tests
* Fix mxnet container
* Import all Kubernetes models
* Update SDK Examples
* Verify other SDK APIs in e2e tests
* Add replica types to const
* Use logging in e2e tests
* Fix logging for status
* Use const for job types
* Support for k8s v1.25 in CI
* Support for k8s v1.25 in CI
* Change k8s api to v1.25
* Upgrade golangci-lint version
* Fixes for HPA in pytorchjob
* Common changes
* support paddlejob
* fix ci
* fix tools diff for ci
* fix ci test
* rm todo
* add e2e test
* fix e2e test
* fix e2e ci
* add paddlejob to kustomize
* set registry to docker.io
* rebase
* add rbac for paddlejob
* add unitest cover container port
* fix rebase
* update Copyright
* Slim OpenAPI definitions by removing unrelated api
openapi-gen input-dirs includes a few k8s.io/api and k8s.io/apimachinery packages which makes openapi_generated.go very large. Swagger files created by them are large as well.
This change help remove unnecessary references.
Signed-off-by: Jiaxin Shan <[email protected]>
* Update OpenAPI generation scripts
1. Update gen-sdk.sh and main.go to support all frameworks
2. Generate swagger.json for all frameworks
Signed-off-by: Jiaxin Shan <[email protected]>
* Merge swagger files into one
The swagger.json can not be merged smartly. I manually update info.description, info.title in hack/python-sdk/swagger.json
Signed-off-by: Jiaxin Shan <[email protected]>
