mirror of
https://github.com/hanzo-ml/trainer.git
synced 2026-09-24 00:31:34 +00:00
master
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rob BellandGitHub
|
78b4a00929 |
feat: add support for tracking TrainJob progress and training metrics (#3227)
* feat(api): add progress types to api Signed-off-by: Rob Bell <[email protected]> * feat: progress server stub Signed-off-by: Rob Bell <[email protected]> * feat: setup tls for server Signed-off-by: Rob Bell <[email protected]> * refactor: pull out middleware Signed-off-by: Rob Bell <[email protected]> * refactor: remove logger message noise from tests Signed-off-by: Rob Bell <[email protected]> * fix: send server error logs to main logger Signed-off-by: Rob Bell <[email protected]> * feat: update the trainer status Signed-off-by: Rob Bell <[email protected]> * feat: use separate client to avoid impacting main reconciler client-side rate limits Signed-off-by: Rob Bell <[email protected]> * feat: configure progress server in helm chart and configmap Signed-off-by: Rob Bell <[email protected]> * fix: prevent race condition Signed-off-by: Rob Bell <[email protected]> * feat: inject progress runtime config Signed-off-by: Rob Bell <[email protected]> * fix: progress client config Signed-off-by: Rob Bell <[email protected]> * fix: update generated config Signed-off-by: Rob Bell <[email protected]> * feat: validate auth tokens in endpoint Signed-off-by: Rob Bell <[email protected]> * test: validate auth tokens in endpoint Signed-off-by: Rob Bell <[email protected]> * refactor: tidy tests to use same format as other tests Signed-off-by: Rob Bell <[email protected]> * fix: look up the cluster issuer from the service account token Different clusters can use different issuers so look up the issuer from the trusted in-cluster service account token. Signed-off-by: Rob Bell <[email protected]> * refactor: rename authn middleware Signed-off-by: Rob Bell <[email protected]> * feat: add authn to status update endpoint Signed-off-by: Rob Bell <[email protected]> * refactor: tidy up reading in-cluster service account token Signed-off-by: Rob Bell <[email protected]> * fix: rbac permissions Signed-off-by: Rob Bell <[email protected]> * fix: better error response Signed-off-by: Rob Bell <[email protected]> * refactor: move kubebuilder annotation higher up Signed-off-by: Rob Bell <[email protected]> * fix: make lastUpdateTime required Signed-off-by: Rob Bell <[email protected]> * chore: fix config tests after rebase Signed-off-by: Rob Bell <[email protected]> * test: update config test with new progress server config Signed-off-by: Rob Bell <[email protected]> * test: add progress tracking e2e test Signed-off-by: Rob Bell <[email protected]> * fix: allow feature gates to be configurable from command line argument So they can be overridden in e2e tests Signed-off-by: Rob Bell <[email protected]> * refactor: reuse shared progress StatusUrl Signed-off-by: Rob Bell <[email protected]> * test: tidy up tests Signed-off-by: Rob Bell <[email protected]> * refactor: allow plugins to use config Signed-off-by: Rob Bell <[email protected]> * refactor: centralise shared constants Signed-off-by: Rob Bell <[email protected]> * fix: disable http2 in progress server tls opts if disabled by config Signed-off-by: Rob Bell <[email protected]> * fix: minor improvements suggested by copilot Signed-off-by: Rob Bell <[email protected]> * chore: rename progress server => status server Signed-off-by: Rob Bell <[email protected]> * chore: update type doc Signed-off-by: Rob Bell <[email protected]> * chore: update name of feature gate Signed-off-by: Rob Bell <[email protected]> * fix: after rebase Signed-off-by: Rob Bell <[email protected]> * chore: fix typo Signed-off-by: Rob Bell <[email protected]> * chore: rename handler Signed-off-by: Rob Bell <[email protected]> * fix: run status server on all replicas Signed-off-by: Rob Bell <[email protected]> * fix: deep copy the rest config Signed-off-by: Rob Bell <[email protected]> * refactor: use ssa Signed-off-by: Rob Bell <[email protected]> * refactor: separate token audience per train job Signed-off-by: Rob Bell <[email protected]> * chore: rename status server Signed-off-by: Rob Bell <[email protected]> * fix: revert adding status pkg to manifest gen paths Signed-off-by: Rob Bell <[email protected]> * chore: update log messages Signed-off-by: Rob Bell <[email protected]> * chore: update api docs Signed-off-by: Rob Bell <[email protected]> * chore: update format Signed-off-by: Rob Bell <[email protected]> * fix: revert inadvertent change Signed-off-by: Rob Bell <[email protected]> * chore: update api name and type docs Signed-off-by: Rob Bell <[email protected]> * refactor: initialise oidc provider in manager startup Signed-off-by: Rob Bell <[email protected]> * fix: allow empty update request Signed-off-by: Rob Bell <[email protected]> * fix: wait for server shut down Signed-off-by: Rob Bell <[email protected]> * fix: handle missing train job Signed-off-by: Rob Bell <[email protected]> * refactor: update env var names Signed-off-by: Rob Bell <[email protected]> * chore: rename packages, plugins, feature gate Signed-off-by: Rob Bell <[email protected]> * refactor: update env var names Signed-off-by: Rob Bell <[email protected]> * chore: regenerate after rebase Signed-off-by: Rob Bell <[email protected]> * chore: fixes after rebase Signed-off-by: Rob Bell <[email protected]> * chore: minor doc updates Signed-off-by: Rob Bell <[email protected]> * chore: rename status server config Signed-off-by: Rob Bell <[email protected]> * chore: replace placeholder config with nil Signed-off-by: Rob Bell <[email protected]> * chore: revert separate runtime utils package Signed-off-by: Rob Bell <[email protected]> * chore: remove unused config from tests Signed-off-by: Rob Bell <[email protected]> --------- Signed-off-by: Rob Bell <[email protected]> |
||
|
Antonin StefanuttiandGitHub
|
9519b9960f |
fix(ci): Fix kube-api-linter install (#3023)
Signed-off-by: Antonin Stefanutti <[email protected]> |
||
|
Antonin StefanuttiandGitHub
|
0f9825e89e |
chore(operator): Use SSA throughout runtime framework (#2877)
* chore(operator): Use SSA throughout runtime framework Signed-off-by: Antonin Stefanutti <[email protected]> * Fix lint error Signed-off-by: Antonin Stefanutti <[email protected]> * Update go.mod file Signed-off-by: Antonin Stefanutti <[email protected]> --------- Signed-off-by: Antonin Stefanutti <[email protected]> |
||
|
|
9802f954ab |
feat: KEP-2437 - PodGroup Creation for Volcano Scheduler (#2729)
* feat: api for volcano scheduling plugin Signed-off-by: Xinmin Du <[email protected]> * feat: init volcano-plugin Signed-off-by: Xinmin Du <[email protected]> * feat: init test file Signed-off-by: Xinmin Du <[email protected]> * feat: register volcano plugin Signed-off-by: Xinmin Du <[email protected]> * feat: deal with minTaskMember, minMember, NetworkTopo Signed-off-by: Xinmin Du <[email protected]> * fix: calculate of minResource Signed-off-by: Xinmin Du <[email protected]> * test: build PodGroup test Signed-off-by: Xinmin Du <[email protected]> * refactor: separate to 2 prs（build&handler） Signed-off-by: Xinmin Du <[email protected]> * test: add test for new&reconcile_builder Signed-off-by: Xinmin Du <[email protected]> * fix: typo Signed-off-by: Xinmin Du <[email protected]> * fix: trainer/v2 import Signed-off-by: Xinmin Du <[email protected]> * fix: networktopo type Signed-off-by: Xinmin Du <[email protected]> * fix: OpenAPI validation errors Signed-off-by: Xinmin Du <[email protected]> * fix: remove minTaskMembers Signed-off-by: Xinmin Du <[email protected]> * test: test coverage 100% Signed-off-by: Xinmin Du <[email protected]> * feat: update apis Signed-off-by: Xinmin Du <[email protected]> * feat: replace testify Signed-off-by: Xinmin Du <[email protected]> * fix: registry Volcano CRDs to the scheme Signed-off-by: Xinmin Du <[email protected]> * fix: add volcano to scheme Signed-off-by: Xinmin Du <[email protected]> * fix: fix networktopo schema Signed-off-by: Xinmin Du <[email protected]> * fix: add networktopo spec in trainer Signed-off-by: Xinmin Du <[email protected]> * fix: unit test Signed-off-by: Xinmin Du <[email protected]> * feat: import networkTopo directly Signed-off-by: Xinmin Du <[email protected]> * fix: make generate Signed-off-by: Xinmin Du <[email protected]> * fix: make generate Signed-off-by: Xinmin Du <[email protected]> * fix: golangci-lint Signed-off-by: Xinmin Du <[email protected]> * fix: golangci-lint Signed-off-by: Xinmin Du <[email protected]> * feat: add volcano installation in integration test Signed-off-by: Xinmin Du <[email protected]> * fix: filter volcano api Signed-off-by: Xinmin Du <[email protected]> * fix: get volcano.podgroup with local version Signed-off-by: Xinmin Du <[email protected]> * fix: init test env with volcano podgroup installed Signed-off-by: Xinmin Du <[email protected]> * fix: check plugin in enforcePodgroupPolicy Signed-off-by: Xinmin Du <[email protected]> * fix: group-name label in unit test Signed-off-by: Xinmin Du <[email protected]> * fix: ReconcilerBuilders Signed-off-by: Xinmin Du <[email protected]> * feat: add PodGroupHandler Signed-off-by: Xinmin Du <[email protected]> * feat: unit test for handlers Signed-off-by: Xinmin Du <[email protected]> * fix: group name annotation Signed-off-by: Xinmin Du <[email protected]> * Update hack/swagger/main.go Co-authored-by: Andrey Velichkevich <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: no need to delete RBAC Signed-off-by: Xinmin Du <[email protected]> * Update pkg/runtime/framework/plugins/volcano/indexer.go Co-authored-by: Andrey Velichkevich <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: nil checking for trainjob Signed-off-by: Xinmin Du <[email protected]> * Update pkg/runtime/framework/plugins/volcano/volcano.go Co-authored-by: Andrey Velichkevich <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: make generate Signed-off-by: Xinmin Du <[email protected]> * fix: index conflict Signed-off-by: Xinmin Du <[email protected]> * Update pkg/runtime/framework/plugins/coscheduling/coscheduling.go Co-authored-by: Shao Wang <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: update volcano to v1.12.2 Signed-off-by: Xinmin Du <[email protected]> * feat: re-use indexer Signed-off-by: Xinmin Du <[email protected]> * feat: add validate Signed-off-by: Xinmin Du <[email protected]> * fix: no scheduler when coscheduling is nil Signed-off-by: Xinmin Du <[email protected]> * fix: put group-name in annotations Signed-off-by: Xinmin Du <[email protected]> * feat: validate if priorityClass installed Signed-off-by: Xinmin Du <[email protected]> * feat: propagate annotations to pod Signed-off-by: Xinmin Du <[email protected]> * feat: integration test for volcano Signed-off-by: Xinmin Du <[email protected]> * fix: golangci-lint check Signed-off-by: Xinmin Du <[email protected]> * feat: use shared indexer Signed-off-by: Xinmin Du <[email protected]> * feat: remove indexer to runtime/ Signed-off-by: Xinmin Du <[email protected]> * Update hack/swagger/main.go Co-authored-by: Yuki Iwai <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * Update hack/swagger/main.go Co-authored-by: Yuki Iwai <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: append owner reference & missing import Signed-off-by: Xinmin Du <[email protected]> * fix: rewrite volcano UT Signed-off-by: Xinmin Du <[email protected]> * feat: add copyright Signed-off-by: Xinmin Du <[email protected]> * fix: sync RBAC to Helm charts Signed-off-by: Xinmin Du <[email protected]> * fix: refactor UTs Signed-off-by: Xinmin Du <[email protected]> * fix: test validation separately Signed-off-by: Xinmin Du <[email protected]> * Update hack/swagger/main.go Co-authored-by: Yuki Iwai <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: refactor TestVolcano Signed-off-by: Xinmin Du <[email protected]> * fix: refactor TestValidate Signed-off-by: Xinmin Du <[email protected]> * Update pkg/runtime/framework/plugins/volcano/volcano_test.go Co-authored-by: Yuki Iwai <[email protected]> Signed-off-by: Du Xinmin <[email protected]> --------- Signed-off-by: Xinmin Du <[email protected]> Signed-off-by: Du Xinmin <[email protected]> Co-authored-by: Andrey Velichkevich <[email protected]> Co-authored-by: Shao Wang <[email protected]> Co-authored-by: Yuki Iwai <[email protected]> |