hanzo-ml/trainer
1
0
Fork 0
mirror of https://github.com/hanzo-ml/trainer.git synced 2026-09-24 00:31:34 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,022 Commits 2 Branches 1 Tag
31ba928cf668d4e7fef735cd16345992dc60438e
Commit Graph
18 Commits
Author SHA1 Message Date
Yuki IwaiandGitHub 7b09fd0449 Support K8s v1.29 and Drop K8s v1.26 (#2039) 
Signed-off-by: Yuki Iwai <[email protected]>
 2024-03-28 21:02:18 +00:00
Johnu GeorgeandGitHub 051a68a3b0 Merge v1.7 branch changes to Main (#1940) 
* Update latest training operator images in v1.7 branch (#1881)

* Removing dead code

* Update monitoring guide

* Update latest training operator image

* Update sdk to v1.7.0rc0 (#1882)

* Removing dead code

* Update monitoring guide

* Update latest training operator image

* Update SDK version to v1.7.0rc0

* Update sdk to v1.7.0 (#1939)
 2023-11-01 12:31:57 +00:00
Yuki IwaiandGitHub de6d52bf3a Implement suspend semantics (#1859) 
Signed-off-by: Yuki Iwai <[email protected]>
 2023-07-20 20:25:24 +00:00
Yuki IwaiandGitHub 38092c3b6e Make Condition and ReplicaStatus optional (#1862) 
Signed-off-by: Yuki Iwai <[email protected]>
 2023-07-13 17:07:25 +00:00
Chitsing KUIandGitHub a81242a4cb Set correct ENV for PytorchJob to support torchrun (#1840) 
* fix pytorch WORLD_SIZE env inconsistent

* add correct env node_rank, nnodes for torch

* restore elastic nnproc

* use string for nproc_per_node

* add defaults in api

* add validation for two nproc_per_node, use auto for defaulter

* add ut for defaults and validation

* fix ut

* add doc for nproc_per_node
 2023-07-12 17:11:07 +00:00
Johnu GeorgeGitHubYuki Iwaiyu lin
 		c320716a1a Merge kubeflow/common to training-operator (#1813) 
* Support for k8s v1.25 in CI

* Support for k8s v1.25 in CI

* Change k8s api to v1.25

* Upgrade golangci-lint version

* Add changelog

* Update CHANGELOG.md

* Update Changelog

* Merge common repo

* Avoid to depend on local env when installing the code-generators (#1810)

Signed-off-by: Yuki Iwai <[email protected]>

* Make scheduler-plugins the default gang scheduler. (#1747)

Signed-off-by: Syulin7 <[email protected]>

* Fix tests

* Fix merge conflicts

* Fix CI issues

* Fix CI issues

* Fix review comments

* Add contributors in Readme file

* Fix review comments

* Fix review comments

---------

Signed-off-by: Yuki Iwai <[email protected]>
Signed-off-by: Syulin7 <[email protected]>
Co-authored-by: Yuki Iwai <[email protected]>
Co-authored-by: yu lin <[email protected]>
 2023-05-31 11:04:04 +00:00
yu linandGitHub 959a7b8711 Make scheduler-plugins the default gang scheduler. (#1747) 
Signed-off-by: Syulin7 <[email protected]>
 2023-05-23 11:10:35 +00:00
Yuki IwaiGitHubWang Zhang
 		29c3feb9f4 Adopting coschduling plugin (#1724) 
* Add scheduler plugin for coscheduling

Signed-off-by: Yuki Iwai <[email protected]>
Co-authored-by: Wang Zhang <[email protected]>
Signed-off-by: Yuki Iwai <[email protected]>

* generate codes

Signed-off-by: Yuki Iwai <[email protected]>

* Add a custom Watcher for scheduler-plugins PodGroup to controllers

Signed-off-by: Yuki Iwai <[email protected]>

* Fix predicates of paddlepaddle-controller for scheduling.volcano.sh/v1beta1 PodGroup

Signed-off-by: Yuki Iwai <[email protected]>

* Upgrade kubeflow/common version to v0.4.6 with HPA changes

Signed-off-by: Yuki Iwai <[email protected]>

Signed-off-by: Yuki Iwai <[email protected]>
Co-authored-by: Wang Zhang <[email protected]>
 2023-01-21 17:58:50 +00:00
Johnu GeorgeandGitHub 8c07ee0f54 HPA support for PyTorch Elastic (#1701) 
* Support for k8s v1.25 in CI

* Support for k8s v1.25 in CI

* Change k8s api to v1.25

* Upgrade golangci-lint version

* Fixes for HPA in pytorchjob

* Common changes
 2022-12-21 15:49:13 +00:00
Joeshi KuiandGitHub 483380337a [PaddlePaddle] support paddlejob (#1675) 
* support paddlejob

* fix ci

* fix tools diff for ci

* fix ci test

* rm todo

* add e2e test

* fix e2e test

* fix e2e ci

* add paddlejob to kustomize

* set registry to docker.io

* rebase

* add rbac for paddlejob

* add unitest cover container port

* fix rebase

* update Copyright
 2022-11-07 20:18:37 +00:00
Johnu GeorgeandGitHub 28f2e4f7ae Update SDK version to 1.5.0 (#1624) 
* Adding latest image tag

* Update manifests with latest image tag

* Upgrading sdk to 1.5.0
 2022-06-28 16:23:34 +00:00
Yuki IwaiandGitHub 0166d87791 Add clientset for MPIJob, PytorchJob, MXJob, and XGBoostJob (#1610) 2022-06-14 04:20:31 +00:00
Johnu GeorgeandGitHub 9a7e72b19e Generating MPI python sdk (#1606) 
* Adding latest image tag

* Update manifests with latest image tag

* Update k8s dependencies to v0.24.1

* Update manifests

* Update k8s matrix for integration tests

* Update k8s matrix for unit tests

* Fix k8s versions

* Fix version

* Add scripts in separate file

* Fix Makefile

* Cleanup Makefile

* Addressing review comments

* Generating sdk for MPI

* Make list metadata optional

* Adding open gen file

* Commit swagger change
 2022-06-09 12:54:01 +00:00
Johnu GeorgeandGitHub f7140508cd Update k8s dependencies to v0.24.1 (#1604) 
* Adding latest image tag

* Update manifests with latest image tag

* Update k8s dependencies to v0.24.1

* Update manifests

* Update k8s matrix for integration tests

* Update k8s matrix for unit tests

* Fix k8s versions

* Fix version

* Add scripts in separate file

* Fix Makefile

* Cleanup Makefile

* Addressing review comments
 2022-06-06 01:25:28 +00:00
Alex LembiyeuskiandGitHub f7fad2b95b Release Python SDK 1.4.0 (#1541) 
* Bump python package and api versions

* Prevent path extension fix from being overriden by code gen

* ./hack/python-sdk/gen-sdk.sh
 2022-03-07 09:42:53 +00:00
Ce GaoandGitHub 8082044198 feat(pytorch): Support elastic training (#1453) 
* chore: Update codegen and vendor

Signed-off-by: Ce Gao <[email protected]>

* fix: Update git config

Signed-off-by: Ce Gao <[email protected]>

* fix: Codegen

Signed-off-by: Ce Gao <[email protected]>

* feat: Add elastic examples

Signed-off-by: Ce Gao <[email protected]>

* chore: Update go mod

Signed-off-by: Ce Gao <[email protected]>

* chore: Update manifest

Signed-off-by: Ce Gao <[email protected]>

* feat: Implement elstic policy

Signed-off-by: Ce Gao <[email protected]>

* chore: Update vendor

Signed-off-by: Ce Gao <[email protected]>

* chore: Update swagger

Signed-off-by: Ce Gao <[email protected]>

* fix: Address comments

Signed-off-by: Ce Gao <[email protected]>
 2021-11-26 06:35:12 +00:00
Alex LembiyeuskiandGitHub 3daf40fe6a Add Python SDK for Kubeflow Training Operator (#1420) 
* Add Python SDK for Kubeflow Training Operator

* Update example notebooks

* Update SDK generation tooling and docs

* Re-generate SDK

* Allow to specify container name in 'get_logs' methods

* Generalize job labels

* Check if attribute exists

* Address code review comments
 2021-10-03 15:46:49 -07:00
Jiaxin ShanandGitHub d7a46413f2 Update scripts to generate sdk for all frameworks (#1389) 
* Slim OpenAPI definitions by removing unrelated api

openapi-gen input-dirs includes a few k8s.io/api and k8s.io/apimachinery packages which makes openapi_generated.go very large. Swagger files created by them are large as well.
This change help remove unnecessary references.

Signed-off-by: Jiaxin Shan <[email protected]>

* Update OpenAPI generation scripts

1. Update gen-sdk.sh and main.go to support all frameworks
2. Generate swagger.json for all frameworks

Signed-off-by: Jiaxin Shan <[email protected]>

* Merge swagger files into one

The swagger.json can not be merged smartly. I manually update info.description, info.title in hack/python-sdk/swagger.json

Signed-off-by: Jiaxin Shan <[email protected]>
 2021-09-28 12:26:47 -07:00