24 Commits

Include renames

Author	SHA1	Message	Date
3 people	37c8a07a8a	feat(api): Replace PodTemplateOverrides with RuntimePatches API (#3309) ... * feat(api): Replace PodTemplateOverrides with RuntimePatches API Signed-off-by: Andrey Velichkevich <[email protected]> * Update goal with RuntimePatches API Signed-off-by: Andrey Velichkevich <[email protected]> * Add diagram for RuntimePatches Signed-off-by: Andrey Velichkevich <[email protected]> * feat(api): Replace PodTemplateOverrides with RuntimePatches API Signed-off-by: Antonin Stefanutti <[email protected]> * Fix kube lint errors Signed-off-by: Antonin Stefanutti <[email protected]> * Update Python models Signed-off-by: Antonin Stefanutti <[email protected]> * Enable maxlength Kube-API-Linter rule Signed-off-by: Antonin Stefanutti <[email protected]> * Restore manger field IsDomainPrefixedPath validation Signed-off-by: Antonin Stefanutti <[email protected]> * Fix integration tests Signed-off-by: Antonin Stefanutti <[email protected]> * Apply StrategicMergePatch to batchv1.JobTemplateSpec Signed-off-by: Antonin Stefanutti <[email protected]> --------- Signed-off-by: Andrey Velichkevich <[email protected]> Signed-off-by: Antonin Stefanutti <[email protected]> Co-authored-by: Andrey Velichkevich <[email protected]>	2026-03-12 16:29:44 +00:00
Krishna GuptaandGitHub	ccaf20bc20	feat(runtimes): Add XGBoost runtime(KEP-2598) (#3200) ... * Add XGBoost runtime plugin scaffold and register in framework Signed-off-by: Krishna-kg732 <[email protected]> Signed-off-by: krishna-kg732 <[email protected]> * regenerated assets Signed-off-by: Krishna-kg732 <[email protected]> Signed-off-by: krishna-kg732 <[email protected]> * Add XGBoost plugin unit tests and update framework test registry Signed-off-by: Krishna-kg732 <[email protected]> Signed-off-by: krishna-kg732 <[email protected]> * added :Runtime image , unit tests for :(Resources are not set, Resources are set in TrainJob) ,using the ContainerTrainerPort instead of default port Signed-off-by: krishna-kg732 <[email protected]> * minor changes Signed-off-by: krishna-kg732 <[email protected]> * added plainML fallback(JAX+XGBOOST) Signed-off-by: krishna-kg732 <[email protected]> * test: add XGBoost runtime resource resolution unit tests - Add test case for resources set in Runtime only - Add test case for resources set in both Runtime and TrainJob - Add xgboost_distributed.yaml ClusterTrainingRuntime manifest - Update kustomization.yaml with xgboost-runtime image Addresses feedback from PR #3200 for complete resource resolution test coverage. Signed-off-by: krishna-kg732 <[email protected]> * added autogenerat3ed files Signed-off-by: krishna-kg732 <[email protected]> * CI fixes Signed-off-by: krishna-kg732 <[email protected]> * chore: fix ci lint errors Signed-off-by: krishna-kg732 <[email protected]> * Add XGBoost integration and E2E tests, fix Validate guard clause and nil safety Signed-off-by: krishna-kg732 <[email protected]> * trigger ci Signed-off-by: krishna-kg732 <[email protected]> * fix: Add XGBoost support to test wrapper, CEL validation, and CI publish workflow - Add XGBoostPolicy() method to MLPolicySourceWrapper for integration tests - Add has(self.xgboost) to MLPolicy CEL mutual-exclusivity validation rule - Regenerate CRD manifests with updated CEL rule - Add xgboost-runtime to build-and-push-images CI workflow - Add XGBoost image build step to E2E setup script (commented out due to Kind bug) Signed-off-by: krishna-kg732 <[email protected]> * docs: Add XGBoost distributed training example notebook Add example notebook demonstrating distributed XGBoost training using the Kubeflow Trainer SDK and the xgboost-distributed ClusterTrainingRuntime. The notebook uses the modern XGBoost Collective API with RabitTracker for distributed coordination and synthetic classification data. Signed-off-by: krishna-kg732 <[email protected]> * test: Skip XGBoost E2E test until runtime image is published Signed-off-by: krishna-kg732 <[email protected]> * chore : address review comments Signed-off-by: krishna-kg732 <[email protected]> * chore:resolve merge conflicts Signed-off-by: krishna-kg732 <[email protected]> --------- Signed-off-by: Krishna-kg732 <[email protected]> Signed-off-by: krishna-kg732 <[email protected]>	2026-03-10 10:04:42 +00:00
4 people	88a92a0483	feat: support for flux framework as hpc manager (#3188) ... * feat: support for flux framework as hpc manager Flux supports the majority of MPI flavors/variants, and can be used to bootstrap MPI as a plugin. It adds other features for scheduling and topology that can be used for simulations and ai/ml jobs. This changeset adds the plugin implementation, including the plugin module, tests, and an example with a small README to serve as documentation for the time being. Signed-off-by: vsoch <[email protected]> * review: slim down use of annotations We still need to put the original command in an annotation to retieve later, but others can be re-derived from the environment Signed-off-by: vsoch <[email protected]> * review: clean up tasks and remove extra FluxPolicySource Signed-off-by: vsoch <[email protected]> * review: ensure to unset trainer args Signed-off-by: vsoch <[email protected]> * review: better support for nproc per node and nodes Signed-off-by: vsoch <[email protected]> * review: add constants types for flux Signed-off-by: vsoch <[email protected]> * review: add gpu specification from resources Signed-off-by: vsoch <[email protected]> * Fix TrainJob NumProcPerNode Signed-off-by: Andrey Velichkevich <[email protected]> * Remove elasticPolicy from kubebuilder:validation Signed-off-by: Andrey Velichkevich <[email protected]> * chore: use go embed for long string templates flux plugin Signed-off-by: vsoch <[email protected]> * cleanup: removed un-used annotation Signed-off-by: vsoch <[email protected]> --------- Signed-off-by: vsoch <[email protected]> Signed-off-by: Andrey Velichkevich <[email protected]> Co-authored-by: vsoch <[email protected]> Co-authored-by: Andrey Velichkevich <[email protected]>	2026-03-10 03:19:05 +00:00
Andrey VelichkevichandGitHub	63d928468c	feat(api): BREAKING CHANGE: Remove numProcPerNode from Torch API (#3239) ... * feat(api): BREAKING CHANGE: Remove numProcPerNode from Torch API Signed-off-by: Andrey Velichkevich <[email protected]> * Remove trainer_v1alpha1_torch_elastic_policy.py from Py models Signed-off-by: Andrey Velichkevich <[email protected]> * Fix numNodes validation in torchtune Signed-off-by: Andrey Velichkevich <[email protected]> --------- Signed-off-by: Andrey Velichkevich <[email protected]>	2026-03-01 06:53:03 +00:00
Andrey VelichkevichandGitHub	60004df3d3	feat(api): BREAKING CHANGE – Remove ElasticPolicy API (#3235) ... Signed-off-by: Andrey Velichkevich <[email protected]>	2026-02-24 20:58:47 +00:00
Tomas TormoandGitHub	bb945b03dd	feat: Add the manager field to the podTemplateOverride object (#3020) ... Signed-off-by: Tomas Tormo <[email protected]>	2026-02-16 23:54:28 +00:00
Andrey VelichkevichandGitHub	ea044c39c4	feat(api): Fix immutability of the TrainJob APIs (#3157) ... * feat(api): Fix immutability for TrainJob APIs Signed-off-by: Andrey Velichkevich <[email protected]> * Only validate immutability of podTemplateOverride API Signed-off-by: Andrey Velichkevich <[email protected]> * Fix CEL rule Signed-off-by: Andrey Velichkevich <[email protected]> * Use listType atomic for PodTemplateOverrideTargetJob Signed-off-by: Andrey Velichkevich <[email protected]> * Remove PodTemplateOverrides CEL validation Signed-off-by: Andrey Velichkevich <[email protected]> * Test image immutability Signed-off-by: Andrey Velichkevich <[email protected]> --------- Signed-off-by: Andrey Velichkevich <[email protected]>	2026-02-09 19:23:13 +00:00
Tomas TormoandGitHub	62812f9ce3	feat(runtimes): Add JAX training runtime (#3151) ... Signed-off-by: mahdikhashan <[email protected]> Signed-off-by: Tomas Tormo <[email protected]>	2026-02-06 18:15:22 +00:00
Andrey VelichkevichandGitHub	cb04c3bd2e	chore(deps): Bump JobSet v0.11.0 and LWS v0.8.0 (#3144) ... Signed-off-by: Andrey Velichkevich <[email protected]>	2026-01-27 23:53:34 +00:00
Andrey VelichkevichandGitHub	7cdf7ce5a2	chore(deps): Bump Go 1.25, k8s v1.35, and controller-runtime v0.23.1 (#3127) ... * chore(deps): Bump Go 1.25, k8s to v1.35, and controller-runtime v0.23.1 Signed-off-by: Andrey Velichkevich <[email protected]> * Migrate to new events API with EventRecorder Signed-off-by: Andrey Velichkevich <[email protected]> * Fix RBAC for events Signed-off-by: Andrey Velichkevich <[email protected]> * Bump k8s version in tests Signed-off-by: Andrey Velichkevich <[email protected]> * Add type compliance check Signed-off-by: Andrey Velichkevich <[email protected]> * Use Reconciling action for event Signed-off-by: Andrey Velichkevich <[email protected]> --------- Signed-off-by: Andrey Velichkevich <[email protected]>	2026-01-27 19:50:34 +00:00
Sanskar GurdasaniandGitHub	b5532e5e33	feat(api): Add securityContext support to PodTemplateSpecOverride in TrainJob (#3066) ... Signed-off-by: Sanskarzz <[email protected]>	2026-01-15 15:30:18 +00:00
Antonin StefanuttiandGitHub	7755220575	chore(ci): Enable Kubernetes API Linter (#2858) ... * chore(ci): Enable Kubernetes API Linter Signed-off-by: Antonin Stefanutti <[email protected]> * lint files Signed-off-by: Antonin Stefanutti <[email protected]> * Generate Signed-off-by: Antonin Stefanutti <[email protected]> * Review Signed-off-by: Antonin Stefanutti <[email protected]> * Fix golangci-lint target Signed-off-by: Antonin Stefanutti <[email protected]> * Fix commands max length Signed-off-by: Antonin Stefanutti <[email protected]> * Re: Fix golangci-lint target Signed-off-by: Antonin Stefanutti <[email protected]> * Enable the integers linter Signed-off-by: Antonin Stefanutti <[email protected]> * Disable maxlength linter Signed-off-by: Antonin Stefanutti <[email protected]> * make generate Signed-off-by: Antonin Stefanutti <[email protected]> * Make fields optional Signed-off-by: Antonin Stefanutti <[email protected]> * Fix Suspended field in JobStatus Signed-off-by: Antonin Stefanutti <[email protected]> --------- Signed-off-by: Antonin Stefanutti <[email protected]>	2025-10-17 14:40:59 +00:00
Xigang WangandGitHub	1d01c440b7	feat(api): Add PodTemplateOverrides API into TrainJob (#2785) ... Signed-off-by: xigang <[email protected]>	2025-10-09 10:08:14 +00:00
Tomas TormoandGitHub	b1f1a4ab6f	fix: Allow multiple podSpec overrides to target the same TargetJob (#2880) ... Signed-off-by: Tomas Tormo <[email protected]>	2025-10-07 12:55:13 +00:00
5 people	9802f954ab	feat: KEP-2437 - PodGroup Creation for Volcano Scheduler (#2729) ... * feat: api for volcano scheduling plugin Signed-off-by: Xinmin Du <[email protected]> * feat: init volcano-plugin Signed-off-by: Xinmin Du <[email protected]> * feat: init test file Signed-off-by: Xinmin Du <[email protected]> * feat: register volcano plugin Signed-off-by: Xinmin Du <[email protected]> * feat: deal with minTaskMember, minMember, NetworkTopo Signed-off-by: Xinmin Du <[email protected]> * fix: calculate of minResource Signed-off-by: Xinmin Du <[email protected]> * test: build PodGroup test Signed-off-by: Xinmin Du <[email protected]> * refactor: separate to 2 prs（build&handler） Signed-off-by: Xinmin Du <[email protected]> * test: add test for new&reconcile_builder Signed-off-by: Xinmin Du <[email protected]> * fix: typo Signed-off-by: Xinmin Du <[email protected]> * fix: trainer/v2 import Signed-off-by: Xinmin Du <[email protected]> * fix: networktopo type Signed-off-by: Xinmin Du <[email protected]> * fix: OpenAPI validation errors Signed-off-by: Xinmin Du <[email protected]> * fix: remove minTaskMembers Signed-off-by: Xinmin Du <[email protected]> * test: test coverage 100% Signed-off-by: Xinmin Du <[email protected]> * feat: update apis Signed-off-by: Xinmin Du <[email protected]> * feat: replace testify Signed-off-by: Xinmin Du <[email protected]> * fix: registry Volcano CRDs to the scheme Signed-off-by: Xinmin Du <[email protected]> * fix: add volcano to scheme Signed-off-by: Xinmin Du <[email protected]> * fix: fix networktopo schema Signed-off-by: Xinmin Du <[email protected]> * fix: add networktopo spec in trainer Signed-off-by: Xinmin Du <[email protected]> * fix: unit test Signed-off-by: Xinmin Du <[email protected]> * feat: import networkTopo directly Signed-off-by: Xinmin Du <[email protected]> * fix: make generate Signed-off-by: Xinmin Du <[email protected]> * fix: make generate Signed-off-by: Xinmin Du <[email protected]> * fix: golangci-lint Signed-off-by: Xinmin Du <[email protected]> * fix: golangci-lint Signed-off-by: Xinmin Du <[email protected]> * feat: add volcano installation in integration test Signed-off-by: Xinmin Du <[email protected]> * fix: filter volcano api Signed-off-by: Xinmin Du <[email protected]> * fix: get volcano.podgroup with local version Signed-off-by: Xinmin Du <[email protected]> * fix: init test env with volcano podgroup installed Signed-off-by: Xinmin Du <[email protected]> * fix: check plugin in enforcePodgroupPolicy Signed-off-by: Xinmin Du <[email protected]> * fix: group-name label in unit test Signed-off-by: Xinmin Du <[email protected]> * fix: ReconcilerBuilders Signed-off-by: Xinmin Du <[email protected]> * feat: add PodGroupHandler Signed-off-by: Xinmin Du <[email protected]> * feat: unit test for handlers Signed-off-by: Xinmin Du <[email protected]> * fix: group name annotation Signed-off-by: Xinmin Du <[email protected]> * Update hack/swagger/main.go Co-authored-by: Andrey Velichkevich <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: no need to delete RBAC Signed-off-by: Xinmin Du <[email protected]> * Update pkg/runtime/framework/plugins/volcano/indexer.go Co-authored-by: Andrey Velichkevich <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: nil checking for trainjob Signed-off-by: Xinmin Du <[email protected]> * Update pkg/runtime/framework/plugins/volcano/volcano.go Co-authored-by: Andrey Velichkevich <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: make generate Signed-off-by: Xinmin Du <[email protected]> * fix: index conflict Signed-off-by: Xinmin Du <[email protected]> * Update pkg/runtime/framework/plugins/coscheduling/coscheduling.go Co-authored-by: Shao Wang <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: update volcano to v1.12.2 Signed-off-by: Xinmin Du <[email protected]> * feat: re-use indexer Signed-off-by: Xinmin Du <[email protected]> * feat: add validate Signed-off-by: Xinmin Du <[email protected]> * fix: no scheduler when coscheduling is nil Signed-off-by: Xinmin Du <[email protected]> * fix: put group-name in annotations Signed-off-by: Xinmin Du <[email protected]> * feat: validate if priorityClass installed Signed-off-by: Xinmin Du <[email protected]> * feat: propagate annotations to pod Signed-off-by: Xinmin Du <[email protected]> * feat: integration test for volcano Signed-off-by: Xinmin Du <[email protected]> * fix: golangci-lint check Signed-off-by: Xinmin Du <[email protected]> * feat: use shared indexer Signed-off-by: Xinmin Du <[email protected]> * feat: remove indexer to runtime/ Signed-off-by: Xinmin Du <[email protected]> * Update hack/swagger/main.go Co-authored-by: Yuki Iwai <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * Update hack/swagger/main.go Co-authored-by: Yuki Iwai <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: append owner reference & missing import Signed-off-by: Xinmin Du <[email protected]> * fix: rewrite volcano UT Signed-off-by: Xinmin Du <[email protected]> * feat: add copyright Signed-off-by: Xinmin Du <[email protected]> * fix: sync RBAC to Helm charts Signed-off-by: Xinmin Du <[email protected]> * fix: refactor UTs Signed-off-by: Xinmin Du <[email protected]> * fix: test validation separately Signed-off-by: Xinmin Du <[email protected]> * Update hack/swagger/main.go Co-authored-by: Yuki Iwai <[email protected]> Signed-off-by: Du Xinmin <[email protected]> * fix: refactor TestVolcano Signed-off-by: Xinmin Du <[email protected]> * fix: refactor TestValidate Signed-off-by: Xinmin Du <[email protected]> * Update pkg/runtime/framework/plugins/volcano/volcano_test.go Co-authored-by: Yuki Iwai <[email protected]> Signed-off-by: Du Xinmin <[email protected]> --------- Signed-off-by: Xinmin Du <[email protected]> Signed-off-by: Du Xinmin <[email protected]> Co-authored-by: Andrey Velichkevich <[email protected]> Co-authored-by: Shao Wang <[email protected]> Co-authored-by: Yuki Iwai <[email protected]>	2025-10-06 05:51:10 +00:00
Antonin StefanuttiandGitHub	afb208d86e	chore(operator): Upgrade JobSet to v0.10.1 (#2875) ... Signed-off-by: Antonin Stefanutti <[email protected]>	2025-10-03 10:08:08 +00:00
Andrey VelichkevichandGitHub	ca158302e7	chore(operator): Bump JobSet to v0.9.0 version (#2821) ... * chore(operator): Bump JobSet to v0.9.0 version Signed-off-by: Andrey Velichkevich <[email protected]> * Fix Go tests Signed-off-by: Andrey Velichkevich <[email protected]> --------- Signed-off-by: Andrey Velichkevich <[email protected]>	2025-09-09 06:55:36 +00:00
3 people	39e4b08a03	feat: support imagePullSecrets in TrainJob pod spec overrides (#2806) ... Signed-off-by: Tsubasa Nagasawa <[email protected]> Co-authored-by: Antonin Stefanutti <[email protected]>	2025-09-08 12:43:36 +00:00
Antonin StefanuttiandGitHub	b0bde2f602	chore(operator): Upgrade Kubernetes to v1.34 (#2804) ... Signed-off-by: Antonin Stefanutti <[email protected]>	2025-08-28 17:02:21 +00:00
Tsubasa NagasawaandGitHub	187a5de400	feat: support affinity in TrainJob pod spec overrides (#2796) ... Signed-off-by: Tsubasa Nagasawa <[email protected]>	2025-08-28 15:52:22 +00:00
3 people	e2ede1f1f6	Upgrade Kubernetes to v1.33 (#2756) ... Signed-off-by: Antonin Stefanutti <[email protected]> Co-authored-by: Andrey Velichkevich <[email protected]>	2025-07-28 15:18:49 +00:00
Antonin StefanuttiandGitHub	8dc6334ccb	feat: Add schedulingGates to PodSpecOverrides (#2700) ... * feat: Add schedulingGates to PodSpecOverrides Signed-off-by: Antonin Stefanutti <[email protected]> * Change desired job to target job in PodSpecOverrides comments Signed-off-by: Antonin Stefanutti <[email protected]> --------- Signed-off-by: Antonin Stefanutti <[email protected]>	2025-07-03 11:03:20 +00:00
5 people	434d953004	feat(controller): Implement PodSpecOverride API (#2614) ... * feat(controller): Implement PodSpecOverride API Signed-off-by: Andrey Velichkevich <[email protected]> * Update API comments Signed-off-by: Andrey Velichkevich <[email protected]> * Fix InitContainers override Signed-off-by: Andrey Velichkevich <[email protected]> * Fix generate Signed-off-by: Andrey Velichkevich <[email protected]> * Update pkg/apis/trainer/v1alpha1/trainjob_types.go Co-authored-by: Shao Wang <[email protected]> Signed-off-by: Andrey Velichkevich <[email protected]> * Change API to targetJob and add validation Signed-off-by: Andrey Velichkevich <[email protected]> * Add unit test for the JobSet builder Signed-off-by: Andrey Velichkevich <[email protected]> * Add integration tests Signed-off-by: Andrey Velichkevich <[email protected]> * Update pkg/apis/trainer/v1alpha1/trainjob_types.go Co-authored-by: Shao Wang <[email protected]> Signed-off-by: Andrey Velichkevich <[email protected]> * Update pkg/apis/trainer/v1alpha1/trainjob_types.go Co-authored-by: Shao Wang <[email protected]> Signed-off-by: Andrey Velichkevich <[email protected]> * Update pkg/apis/trainer/v1alpha1/trainjob_types.go Co-authored-by: Antonin Stefanutti <[email protected]> Signed-off-by: Andrey Velichkevich <[email protected]> * Merge PodSpecOverrides before the build phase Signed-off-by: Antonin Stefanutti <[email protected]> * Fix make generate Signed-off-by: Antonin Stefanutti <[email protected]> * Fix PodSpec SMT patch unmarshalling into JobSet template Signed-off-by: Antonin Stefanutti <[email protected]> * Add UT for volumes PodSpecOverrides Signed-off-by: Antonin Stefanutti <[email protected]> * Add UT for tolerations PodSpecOverrides Signed-off-by: Antonin Stefanutti <[email protected]> * Add UT for node selector PodSpecOverrides Signed-off-by: Antonin Stefanutti <[email protected]> * Revert back to using TargetJobs []PodSpecOverrideTargetJob Signed-off-by: Antonin Stefanutti <[email protected]> * Run make fmt Signed-off-by: Antonin Stefanutti <[email protected]> * Add target jobs unicity validation for PodSpecOverrides Signed-off-by: Antonin Stefanutti <[email protected]> * Update with code review from @tenzen-y Signed-off-by: Antonin Stefanutti <[email protected]> --------- Signed-off-by: Andrey Velichkevich <[email protected]> Signed-off-by: Antonin Stefanutti <[email protected]> Co-authored-by: Shao Wang <[email protected]> Co-authored-by: Antonin Stefanutti <[email protected]> Co-authored-by: Antonin Stefanutti <[email protected]>	2025-06-06 17:04:18 +00:00
Anya KramarandGitHub	4c25accaa0	Move generated Python models into kubeflow_trainer_api package (#2632) ... Signed-off-by: kramaranya <[email protected]>	2025-05-07 18:14:39 +00:00