Commit Graph
19 Commits
Author SHA1 Message Date
Yuki IwaiandGitHub de6d52bf3a Implement suspend semantics (#1859)
Signed-off-by: Yuki Iwai <[email protected]>
2023-07-20 20:25:24 +00:00
Yuki IwaiandGitHub 38092c3b6e Make Condition and ReplicaStatus optional (#1862)
Signed-off-by: Yuki Iwai <[email protected]>
2023-07-13 17:07:25 +00:00
Chitsing KUIandGitHub a81242a4cb Set correct ENV for PytorchJob to support torchrun (#1840)
* fix pytorch WORLD_SIZE env inconsistent

* add correct env node_rank, nnodes for torch

* restore elastic nnproc

* use string for nproc_per_node

* add defaults in api

* add validation for two nproc_per_node, use auto for defaulter

* add ut for defaults and validation

* fix ut

* add doc for nproc_per_node
2023-07-12 17:11:07 +00:00
c320716a1a Merge kubeflow/common to training-operator (#1813)
* Support for k8s v1.25 in CI

* Support for k8s v1.25 in CI

* Change k8s api to v1.25

* Upgrade golangci-lint version

* Add changelog

* Update CHANGELOG.md

* Update Changelog

* Merge common repo

* Avoid to depend on local env when installing the code-generators (#1810)

Signed-off-by: Yuki Iwai <[email protected]>

* Make scheduler-plugins the default gang scheduler. (#1747)

Signed-off-by: Syulin7 <[email protected]>

* Fix tests

* Fix merge conflicts

* Fix CI issues

* Fix CI issues

* Fix review comments

* Add contributors in Readme file

* Fix review comments

* Fix review comments

---------

Signed-off-by: Yuki Iwai <[email protected]>
Signed-off-by: Syulin7 <[email protected]>
Co-authored-by: Yuki Iwai <[email protected]>
Co-authored-by: yu lin <[email protected]>
2023-05-31 11:04:04 +00:00
yu linandGitHub 959a7b8711 Make scheduler-plugins the default gang scheduler. (#1747)
Signed-off-by: Syulin7 <[email protected]>
2023-05-23 11:10:35 +00:00
29c3feb9f4 Adopting coschduling plugin (#1724)
* Add scheduler plugin for coscheduling

Signed-off-by: Yuki Iwai <[email protected]>
Co-authored-by: Wang Zhang <[email protected]>
Signed-off-by: Yuki Iwai <[email protected]>

* generate codes

Signed-off-by: Yuki Iwai <[email protected]>

* Add a custom Watcher for scheduler-plugins PodGroup to controllers

Signed-off-by: Yuki Iwai <[email protected]>

* Fix predicates of paddlepaddle-controller for scheduling.volcano.sh/v1beta1 PodGroup

Signed-off-by: Yuki Iwai <[email protected]>

* Upgrade kubeflow/common version to v0.4.6 with HPA changes

Signed-off-by: Yuki Iwai <[email protected]>

Signed-off-by: Yuki Iwai <[email protected]>
Co-authored-by: Wang Zhang <[email protected]>
2023-01-21 17:58:50 +00:00
Andrey VelichkevichandGitHub 38c63d5b35 [SDK] Create Unify Training Client (#1719)
* Remove legacy ksonnet tests

* [SDK] Create Unify Training Client

* Modify E2E tests

* Rename Training to Operator version

* Add missing exception

* Add API server timeout parameter

* Add delete options

* Fix import for V1JobCondition

* Fix container name in e2e tests

* Fix mxnet container

* Import all Kubernetes models

* Update SDK Examples

* Verify other SDK APIs in e2e tests

* Add replica types to const

* Use logging in e2e tests

* Fix logging for status

* Use const for job types
2023-01-17 15:28:20 +00:00
Johnu GeorgeandGitHub 8c07ee0f54 HPA support for PyTorch Elastic (#1701)
* Support for k8s v1.25 in CI

* Support for k8s v1.25 in CI

* Change k8s api to v1.25

* Upgrade golangci-lint version

* Fixes for HPA in pytorchjob

* Common changes
2022-12-21 15:49:13 +00:00
Joeshi KuiandGitHub 483380337a [PaddlePaddle] support paddlejob (#1675)
* support paddlejob

* fix ci

* fix tools diff for ci

* fix ci test

* rm todo

* add e2e test

* fix e2e test

* fix e2e ci

* add paddlejob to kustomize

* set registry to docker.io

* rebase

* add rbac for paddlejob

* add unitest cover container port

* fix rebase

* update Copyright
2022-11-07 20:18:37 +00:00
Yuki IwaiandGitHub 0166d87791 Add clientset for MPIJob, PytorchJob, MXJob, and XGBoostJob (#1610) 2022-06-14 04:20:31 +00:00
Johnu GeorgeandGitHub 9a7e72b19e Generating MPI python sdk (#1606)
* Adding latest image tag

* Update manifests with latest image tag

* Update k8s dependencies to v0.24.1

* Update manifests

* Update k8s matrix for integration tests

* Update k8s matrix for unit tests

* Fix k8s versions

* Fix version

* Add scripts in separate file

* Fix Makefile

* Cleanup Makefile

* Addressing review comments

* Generating sdk for MPI

* Make list metadata optional

* Adding open gen file

* Commit swagger change
2022-06-09 12:54:01 +00:00
Johnu GeorgeandGitHub f7140508cd Update k8s dependencies to v0.24.1 (#1604)
* Adding latest image tag

* Update manifests with latest image tag

* Update k8s dependencies to v0.24.1

* Update manifests

* Update k8s matrix for integration tests

* Update k8s matrix for unit tests

* Fix k8s versions

* Fix version

* Add scripts in separate file

* Fix Makefile

* Cleanup Makefile

* Addressing review comments
2022-06-06 01:25:28 +00:00
Ce GaoandGitHub 8082044198 feat(pytorch): Support elastic training (#1453)
* chore: Update codegen and vendor

Signed-off-by: Ce Gao <[email protected]>

* fix: Update git config

Signed-off-by: Ce Gao <[email protected]>

* fix: Codegen

Signed-off-by: Ce Gao <[email protected]>

* feat: Add elastic examples

Signed-off-by: Ce Gao <[email protected]>

* chore: Update go mod

Signed-off-by: Ce Gao <[email protected]>

* chore: Update manifest

Signed-off-by: Ce Gao <[email protected]>

* feat: Implement elstic policy

Signed-off-by: Ce Gao <[email protected]>

* chore: Update vendor

Signed-off-by: Ce Gao <[email protected]>

* chore: Update swagger

Signed-off-by: Ce Gao <[email protected]>

* fix: Address comments

Signed-off-by: Ce Gao <[email protected]>
2021-11-26 06:35:12 +00:00
Alex LembiyeuskiandGitHub 3daf40fe6a Add Python SDK for Kubeflow Training Operator (#1420)
* Add Python SDK for Kubeflow Training Operator

* Update example notebooks

* Update SDK generation tooling and docs

* Re-generate SDK

* Allow to specify container name in 'get_logs' methods

* Generalize job labels

* Check if attribute exists

* Address code review comments
2021-10-03 15:46:49 -07:00
Jin Chi HeandKubernetes Prow Robot 0bc41e8fe9 SDK support getting the TFJob training logs (#1130) 2020-01-14 00:07:08 -08:00
Jin Chi HeandKubernetes Prow Robot 95758acf19 Add watch function for TFJob python Client API (#1122) 2020-01-02 01:35:40 -08:00
Jin Chi HeandKubernetes Prow Robot ee8d099fee Update check status API name (#1117) 2019-12-18 00:41:57 -08:00
Jin Chi HeandKubernetes Prow Robot cfba7579a4 add API to wait for TFJob done (#1116) 2019-12-17 21:35:56 -08:00
Jin Chi HeandKubernetes Prow Robot c69649a30b generate TFJob Python SDK (#1103) 2019-12-09 00:05:10 -08:00