141 Commits
Author SHA1 Message Date
Andrey VelichkevichandGitHub 5755f90531 feat(release): Update Katib release script to publish SDK models (#2584)
* feat(release): Update Katib release script to publish SDK models

Signed-off-by: Andrey Velichkevich <[email protected]>

* Fix comment

Signed-off-by: Andrey Velichkevich <[email protected]>

---------

Signed-off-by: Andrey Velichkevich <[email protected]>
2025-10-30 22:04:00 +00:00
Ayush GuptaandGitHub 1c19d3a6eb Fix Istio sidecar injection by moving from annotations to labels (#2527)
* Fix Istio sidecar injection by moving from annotations to labels

Signed-off-by: madmecodes <[email protected]>

* Update Istio sidecar injection from annotations to labels across the codebase
Replace annotations with labels for Istio sidecar injection according to Istio recommendations. Update conformance tests, examples, constants, composers, and utilities to use the new label-based approach consistently.

Signed-off-by: madmecodes <[email protected]>

* fix: Update SuggestionLabels function and composer implementation for Istio label injection

Signed-off-by: madmecodes <[email protected]>

* Fix linting issues in mpi-job-horovod.py

Signed-off-by: madmecodes <[email protected]>

* update: function moved from annotations to labels

Signed-off-by: madmecodes <[email protected]>

---------

Signed-off-by: madmecodes <[email protected]>
2025-05-09 17:52:41 +00:00
Mahdi KhashanandGitHub 19db2d47d4 [feature] move manifest image references to ghcr (#2529)
* move to ghcr

Signed-off-by: mahdikhashan <[email protected]>

* move images to ghcr

Signed-off-by: mahdikhashan <[email protected]>

* manifests

Signed-off-by: mahdikhashan <[email protected]>

* change registry in all path

Signed-off-by: mahdikhashan <[email protected]>

* update script

Signed-off-by: mahdikhashan <[email protected]>

* fix

Signed-off-by: mahdikhashan <[email protected]>

* fix

Signed-off-by: mahdikhashan <[email protected]>

* slight fix

Signed-off-by: mahdikhashan <[email protected]>

---------

Signed-off-by: mahdikhashan <[email protected]>
Signed-off-by: Mahdi Khashan <[email protected]>
2025-03-24 17:11:50 +00:00
Anish AsthanaandGitHub c06feb693c Add 'KEP Usage' KEP and template link (#2509)
Signed-off-by: Anish Asthana <[email protected]>
2025-02-14 23:07:37 +00:00
Mahdi KhashanandGitHub b48367dc1a [DOCS] move llm hyperparameter optimisation design image to the proposal directory and rename it (#2472)
- remove redundant folder

Signed-off-by: mahdikhashan <[email protected]>
2025-01-08 17:43:21 +00:00
Shao WangandGitHub d0847fa6eb fix: remove remaining MXNet dependency. (#2456)
Signed-off-by: Electronic-Waste <[email protected]>
2024-12-02 13:23:57 +00:00
Gonçalo Montalvão MarquesandGitHub 8dd2eac540 docs: remove katib workflow (#2443)
Signed-off-by: Gonçalo Montalvão Marques <[email protected]>
2024-10-15 15:14:18 +00:00
Andrey VelichkevichandGitHub 4f43d71c8f Update README and out-of-date docs (#2438)
Signed-off-by: Andrey Velichkevich <[email protected]>
2024-10-10 19:50:10 +00:00
Shashank MittalandGitHub 033d1b1034 [GSOC] Support for various Parameter distributions in Katib (#2334)
Signed-off-by: Shashank Mittal <[email protected]>

modified feasibleSpace

refactored proposal based on comments

comparison table updated

extra heading removed
2024-07-31 08:03:05 +00:00
5a7cae366d [GSoC] Create LLM Hyperparameters Optimization API Proposal (#2333)
* create llm hyperparameters tuning api proposal

Signed-off-by: helenxie-bit <[email protected]>

* update llm hyperparameters tuning api proposal

Signed-off-by: helenxie-bit <[email protected]>

* update proposal

Signed-off-by: helenxie-bit <[email protected]>

* fix some typos

Signed-off-by: helenxie-bit <[email protected]>

* update the path of image and delete parameter 'resouces_per_worker' from tune api

Signed-off-by: helenxie-bit <[email protected]>

* delete objective function and adjust the design of tune API

Signed-off-by: helenxie-bit <[email protected]>

* Update docs/proposals/llm-hyperparameter-optimization-api.md

Co-authored-by: Andrey Velichkevich <[email protected]>
Signed-off-by: helenxie-bit <[email protected]>

* Move 'Advanced Functionalities' to 'Non-Goals', and update 'Implementation' part

Signed-off-by: helenxie-bit <[email protected]>

* update 'pytorch_config'

Signed-off-by: helenxie-bit <[email protected]>

* change the name of 'pytorch_config' to 'resources_per_trial'

Signed-off-by: helenxie-bit <[email protected]>

* adjust format

Signed-off-by: helenxie-bit <[email protected]>

* adjust format

Signed-off-by: helenxie-bit <[email protected]>

* adjust format

Signed-off-by: helenxie-bit <[email protected]>

* update implementation part and the type of 'resources_per_trial'

Signed-off-by: helenxie-bit <[email protected]>

* update the example

Signed-off-by: helenxie-bit <[email protected]>

* update 'resources_per_trial'& add one more option for defining objective function

Signed-off-by: helenxie-bit <[email protected]>

* fix typo errors

Signed-off-by: helenxie-bit <[email protected]>

* delete 'WIP' tag

Signed-off-by: helenxie-bit <[email protected]>

* update example

Signed-off-by: helenxie-bit <[email protected]>

* update example

Signed-off-by: helenxie-bit <[email protected]>

* update example

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

---------

Signed-off-by: helenxie-bit <[email protected]>
Co-authored-by: Andrey Velichkevich <[email protected]>
2024-07-25 14:10:54 +00:00
AlexandGitHub 53f3c09d1c Begin enabling pre-commit hooks (#2242)
* Begin enabling pre-commit hooks

Signed-off-by: droctothorpe <[email protected]>

* Address PR feedback

Signed-off-by: droctothorpe <[email protected]>

---------

Signed-off-by: droctothorpe <[email protected]>
2024-07-18 17:04:58 +00:00
Ikko Eltociear AshimineandGitHub 7a42792f71 docs: update suggestion.md (#2387)
implmentation -> implementation

Signed-off-by: Ikko Ashimine <[email protected]>
2024-07-17 14:13:57 +00:00
Shao WangandGitHub 87acc02b58 [GSoC] KEP for Project 6: Push-based Metrics Collection for Katib (#2328)
* doc: initial commit of gsoc proposal(project 6).

Signed-off-by: Electronic-Waste <[email protected]>

* doc: complete KEP for gsoc proposal(Project 6).

Signed-off-by: Electronic-Waste <[email protected]>

* chore: add non-goals and examples.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: add .

Signed-off-by: Electronic-Waste <[email protected]>

* chore: add compatibility changes in trial controller.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: update architecture figure.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: update format.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: update doc after the review in 10th, June.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: add code link and remove namespace env variable.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: modify proposal after the review in 14th, June.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: delete WIP label.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: add timeout param into report_metrics.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: metrics_collector_config spelling.

Signed-off-by: Electronic-Waste <[email protected]>

---------

Signed-off-by: Electronic-Waste <[email protected]>
2024-06-28 22:54:42 +00:00
Hezhi XieandGitHub f32231caff Fix apple silicon rosetta error when building images from the source code (#2342)
Signed-off-by: helenxie-bit <[email protected]>
2024-06-05 11:59:03 +00:00
Andrey VelichkevichandGitHub ded506b93e Support ARM64 arch for release images (#2315)
* Support ARM arch for release images

Signed-off-by: Andrey Velichkevich <[email protected]>

* Update Developer Doc

Signed-off-by: Andrey Velichkevich <[email protected]>

---------

Signed-off-by: Andrey Velichkevich <[email protected]>
2024-04-24 22:48:44 +00:00
Yuki IwaiandGitHub adb937f4e4 Upgrade Go version to v1.22 (#2309)
Signed-off-by: Yuki Iwai <[email protected]>
2024-04-15 12:58:51 +00:00
Yuki IwaiandGitHub 105aea3d9e Upgrade Python version to 3.11 (#2278)
* Upgrade Python version to 3.11

Signed-off-by: Yuki Iwai <[email protected]>

* Upgrade the numpy version to 1.25.2

Signed-off-by: Yuki Iwai <[email protected]>

* Increase resource requests for the ENAS suggestion service

Signed-off-by: Yuki Iwai <[email protected]>

* Update pytest CI

Signed-off-by: Yuki Iwai <[email protected]>

* Prepare dedicated pytest for skopt

Signed-off-by: Yuki Iwai <[email protected]>

---------

Signed-off-by: Yuki Iwai <[email protected]>
2024-03-12 19:54:11 +00:00
Yuki IwaiandGitHub 38647e6989 Remove MXNet examples (#2267)
* UT: Replace MXNet example with PyTorch example

Signed-off-by: Yuki Iwai <[email protected]>

* CI: Replace MXNet examples with PyTorch examples

Signed-off-by: Yuki Iwai <[email protected]>

---------

Signed-off-by: Yuki Iwai <[email protected]>
2024-03-04 10:45:07 +00:00
Yuki IwaiandGitHub dc00614aec Add Technical and style guide to the contribution guide (#2250)
Signed-off-by: tenzen-y <[email protected]>
2024-01-04 14:41:12 +00:00
Andrey VelichkevichandGitHub d8f2e5fd03 Add Changelog for Katib v0.16.0-rc.0 (#2204) 2023-08-16 22:31:37 +00:00
Andrey VelichkevichandGitHub cb1426f9b4 Bump Katib Python SDK to 0.16.0rc0 version (#2205) 2023-08-15 15:18:03 +00:00
Yuki IwaiandGitHub f27491905e Sending an empty data to the certsReady channel (#2196)
Signed-off-by: Yuki Iwai <[email protected]>
2023-08-05 14:17:53 +00:00
Yuki IwaiandGitHub c6c49416d9 Consolidate the katib-cert-generator to the katib-controller (#2185)
* Consolidate the katib-cert-generator to the katib-controller

Signed-off-by: Yuki Iwai <[email protected]>

* Use deployed secret instead of creating a new secret when the cert-generator saves certs on secret

Signed-off-by: Yuki Iwai <[email protected]>

* Rename secretName with webhookSecretName in the .init.certGenerator

Signed-off-by: Yuki Iwai <[email protected]>

* Fix manifests

Signed-off-by: Yuki Iwai <[email protected]>

* Remove unneeded comments

Signed-off-by: Yuki Iwai <[email protected]>

* Restore unintentionally deleted log

Signed-off-by: Yuki Iwai <[email protected]>

* Rename package cert-generator with certgenerator

Signed-off-by: Yuki Iwai <[email protected]>

* Add test cases to check if the enable is set to true when the webhookServiceName or webhookSecretName is set

Signed-off-by: Yuki Iwai <[email protected]>

* Update the developer guide

Signed-off-by: Yuki Iwai <[email protected]>

* Swap livness probe and readiness probe

Signed-off-by: Yuki Iwai <[email protected]>

* Introduce SSA to the cert-generator

Signed-off-by: Yuki Iwai <[email protected]>

* Use the same member names between CertGenerator and KatibConfig

Signed-off-by: Yuki Iwai <[email protected]>

* Disable leader election on the cert-generator

Signed-off-by: Yuki Iwai <[email protected]>

* Drop unneeded fields from SSA patches

Signed-off-by: Yuki Iwai <[email protected]>

---------

Signed-off-by: Yuki Iwai <[email protected]>
2023-08-04 19:31:20 +00:00
Yuki IwaiandGitHub cc41442971 Upgrade Go version to v1.20 (#2190)
Signed-off-by: Yuki Iwai <[email protected]>
2023-08-03 11:20:19 +00:00
Yuki IwaiandGitHub 71ee0464c0 Implement KatibConfig API (#2176)
* Implement KatibConfig API

Signed-off-by: Yuki Iwai <[email protected]>

* Replace 'collectorKind' with 'kind'

Signed-off-by: Yuki Iwai <[email protected]>

* Replace 'metricsCollectorSidecars' with 'metricsCollectors'

Signed-off-by: Yuki Iwai <[email protected]>

* Fix a typo

Signed-off-by: Yuki Iwai <[email protected]>

* Make the init.controller.leaderElection non-pointer

Signed-off-by: Yuki Iwai <[email protected]>

* Make the init.controller.injectSecurityContext non-pointer

Signed-off-by: Yuki Iwai <[email protected]>

* Update a comment for the future works

Signed-off-by: Yuki Iwai <[email protected]>

* Update manifest for the katib-leader-election

Signed-off-by: Yuki Iwai <[email protected]>

* Fix a comment for the KatibConfig API

Signed-off-by: Yuki Iwai <[email protected]>

* Replace 'configapi' with 'configv1beta1'

Signed-off-by: Yuki Iwai <[email protected]>

* Remove debug code

Signed-off-by: Yuki Iwai <[email protected]>

* Put constant for the default KatibConfig value on /pkg/apis/config/v1beta1

Signed-off-by: Yuki Iwai <[email protected]>

* Use 'sigs.k8s.io/yaml' instead of 'github.com/ghodss/yaml'

Signed-off-by: Yuki Iwai <[email protected]>

* Avoid to depend on k8s.io/utils directly

Signed-off-by: Yuki Iwai <[email protected]>

* Fix a typo

Signed-off-by: Yuki Iwai <[email protected]>

* Refactor katib-config using kustomize vars

Signed-off-by: Yuki Iwai <[email protected]>

* Fix a typo

Signed-off-by: Yuki Iwai <[email protected]>

* Put KatibConfig on every install

Signed-off-by: Yuki Iwai <[email protected]>

* Remove configMapGenerator from the katib-with-kubeflow

Signed-off-by: Yuki Iwai <[email protected]>

---------

Signed-off-by: Yuki Iwai <[email protected]>
2023-08-01 18:45:08 +00:00
Andrey VelichkevichandGitHub 7d16660c2c [UI] Remove Deprecated Katib UI (#2179)
* [UI] Remove Deprecated Katib UI

* Fix UI Developer doc
2023-07-25 09:53:29 +00:00
Andrey VelichkevichandGitHub f9f16ddf1a Add Changelog for Katib v0.15.0-rc.0 (#2106)
* Add Changelog for Katib v0.15.0-rc.0

* Move Optuna Grid Algorithm to the Core

* Add Breaking and Major Changes
2023-02-23 15:53:24 +00:00
Andrey VelichkevichandGitHub cfe5b1c2ea Bump Katib Python SDK to 0.15.0rc0 version (#2105) 2023-01-28 00:10:02 +00:00
Andrey VelichkevichandGitHub 9fd7af4b05 Use Never Resume Policy as Default (#2102) 2023-01-25 14:11:53 +00:00
Yuki IwaiandGitHub 5b00c20c6f Upgrade Python version to 3.10 (#2057)
Signed-off-by: tenzen-y <[email protected]>
Signed-off-by: Yuki Iwai <[email protected]>

Signed-off-by: tenzen-y <[email protected]>
Signed-off-by: Yuki Iwai <[email protected]>
2023-01-06 09:03:56 +00:00
Yuki IwaiandGitHub 856fb5c3e7 Remove Chocolate Suggestion Service (#2071)
Signed-off-by: Yuki Iwai <[email protected]>

Signed-off-by: Yuki Iwai <[email protected]>
2023-01-05 18:40:11 +00:00
Andrey VelichkevichandGitHub e6935ad6d4 Add Conformance Program Doc for AutoML and Training WG (#2048)
* Add Conformance Program Doc for AutoML and Training WG

* Address Review Comments
2022-12-08 13:35:10 +00:00
Andrey VelichkevichandGitHub 0928090850 Remove Certificate Chain from Cert Generator (#2045)
* Remove Certificate Chain from Cert Generator

* Update Cert Generator Doc
2022-12-05 15:47:09 +00:00
Andrey VelichkevichandGitHub 621bf761ca Add More Katib Presentations 2022 (#2009) 2022-11-10 23:33:50 +00:00
Yuki IwaiandGitHub 7f843315df [chore] Upgrade Go version to v1.19 (#1995)
Signed-off-by: tenzen-y <[email protected]>

Signed-off-by: tenzen-y <[email protected]>
2022-11-03 03:02:04 +00:00
Jaeyeon Kim(김재연)andGitHub 8aa003da73 [feat] health check for katib-controller (#1934)
* [feat]: add health check endpoint

* remove time sleep in github action test script

* add error check

* update docs
2022-10-11 16:50:26 +00:00
Yuki IwaiandGitHub 05f892713b Add --connect-timeout flag to katib-db-manager (#1937) 2022-08-22 11:28:30 +00:00
Yuki IwaiandGitHub 1f407138d6 Introduce Automatic platform ARGs (#1935)
* introduce Automatic platform ARGs
Automatic platform ARGs: https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope

* use BuildKit in actions test for charmed-katib

* use docker buildx in scripts/v1beta1/build.sh
2022-08-21 02:43:29 +00:00
Yuki IwaiandGitHub 360565afc9 [chore] Upgrade Go version to v1.18 (#1925) 2022-08-17 04:35:50 +00:00
Yuki IwaiandGitHub 4a730d6e5c Add the pytorch-mnist with GPU support container image (#1916) 2022-07-16 17:39:05 +00:00
a9pandGitHub 3fd8502567 Population based training (#1833)
* docs: update new algorithm service details

* feat: trial augmentation strategy

* feat: pbt suggestion service

* feat: PbtTemplate and associated test image

* feat: introduce annotation field to trial specifications

* feat: trial assignment changes to support annotations from suggestion

- Add new Annotation types to suggestion_types.go
- Add Annotation object and update Trial parser in trial.py

* feat: update pbt suggestion to use new Annotation api

- Suggestion uses exact match to track spawned trials
- Trials that get transmitted, but not created (or added to experiment) are added back to the respawn pool (population_size consistency)

* chore: gofmt and black run across PBT changes

* feedback: remove tf summary export, change default print unit, reduce range to be percentage compatible.

* feedback: move PBT template to example.

* feedback: changes to inject_webhook and utils.

- Rename mutateVolume to mutateMetricsCollectorVolume
- Add addContainerVolumeMount
- Add getPrimaryContainerIndex

* feedback: change suggestion mutation mount variable name and add to consts

* feedback: Add trial_names to GetSuggestionsReply and change suggestion path to <experiment>/<trial>

* feedback: removed unnecessary checks and moved to async pbt implementation

* feedback: update trial name override location and change annotations override to labels.

* feedback: add pbt to github workflow

* feedback: move labels to ParameterAssignments in GetSuggestionsReply and cleanup pbt.yaml.

* feedback: remove operator changes

* feedback: GHA updates

* feedback: new formatting changes

* feedback: add suggestion-pbt to gh-actions build-load.sh.

* fix: missing pbt->simple-pbt name changes, add simple-pbt to update-images.sh update yaml function (causing failing gha).

* feedback: add pointer to website from main readme for pbt
2022-06-21 15:35:34 +00:00
Yuki IwaiandGitHub 2ea34d2d4a Migrate kubeflow-katib-presubmit to GitHub Actions (#1882)
* migrate test-infra to GitHub Actions

* change python base image to python:3.9-slim

* move from minikube to kind

* separate darts container images by device type

* run e2e test with multi kubernetes version

* disble to deploy katib-ui by default

* change kind kubernetes cluster version

* fix update-images.sh

* fix shellcheck

* fix script to setup katib

* split enas, darts and tf-mnist-with-summaries with trial images

* specicy experiments in pytorch-mnist-e2e-test

* reduce storage size for mysql

* fix trial image name for enas and darts

* fix trial image name for file-metrics-collector-with-json-format

* change kubernetes versions

* do not run e2e test on push master branch

* remove backoffLimit field in examples
2022-06-06 04:50:29 +00:00
Andrey VelichkevichandGitHub b46384808b Add Argo Workflows and Katib presentation (#1818) 2022-02-17 18:11:35 +00:00
Andrey VelichkevichandGitHub eb4a1bee1d Fix default label for Training Operators (#1808)
* Fix default label for Training Operators

* Fix version comment

* Change the docs

* Change git command
2022-02-15 07:19:40 +00:00
0f8895c4ee Bump Python version to 3.9 (#1731)
* bump Python to 3.9

* modify script to build container image

* fix example for enas

* update scripts to modify image name in ci

* review: change docker build command

* review: use new tf-mnist-with-example in Ci for tfjob

* review: refactor tf-mnist-with-summaries

* review: remove Dockerfile.ppc64le for new-ui

* review: update docs related tf-mnist-with-summaries

* TFEventMetricsCollector supports TF>=2.0 and stop supporting TF <=1.x

* review: add help command to scripts/v1beta1/build.sh

Co-authored-by: Andrey Velichkevich <[email protected]>

* fix unit test for tfevent-metricscollector

* review: generate tf event files on CI

* add test command to Makefile

* update publish-trial-images

* update update-images.sh

* reduce batch size

Co-authored-by: Andrey Velichkevich <[email protected]>
2021-12-10 12:33:49 +00:00
Andrey VelichkevichandGitHub 2cbf4ec74a Use release tags for Trial images (#1757)
* Update all Trial image tags to latest

* Modify tags in Notebooks

* Rename script

* Script changes

* Few changes

* Add other images

* Modify test script

* Finish test script

* Modify release script

* Finish release script

* Few changes
2021-12-10 02:11:48 +00:00
Yuan TangandGitHub b9c3ea4c95 docs: Add my presentations that include Katib (#1753) 2021-11-30 17:37:30 +00:00
9ba993d7b5 Implement validation for early stopping (#1709)
* implement validation for early stopping

* fix some documents

* fix error messages

* implement gRPC API to verify parameters for early stopping

* review: use early_stopping as gRPC API

Co-authored-by: Andrey Velichkevich <[email protected]>

* review: fix error description

Co-authored-by: Andrey Velichkevich <[email protected]>

* review: remove t.Run

* review: remove condition to verify algorithmName for early stopping

* remove description about updating gRPC API docs in kubeflow website

Co-authored-by: Andrey Velichkevich <[email protected]>
2021-11-26 15:47:54 +00:00
Andrey VelichkevichandGitHub d00acdecf5 Change namespace label for metrics collector injection (#1740)
* Change namespace label for metrics collector injection

* Fix var name
2021-11-23 02:34:05 +00:00
Andrey VelichkevichandGitHub 4f7c77b357 Modify gRPC API with Current Request Number (#1728)
* Modify API to current_request_number

* Changes after review

* Add request_number deprecated API

* Fix test
2021-11-04 18:03:31 -07:00