hanzo-ml/katib
1
0
Fork 0
mirror of https://github.com/hanzo-ml/katib.git synced 2026-09-24 00:31:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,381 Commits 1 Branch 0 Tags
master
Commit Graph
16 Commits
Author SHA1 Message Date
Andrey VelichkevichandGitHub 96f3ed1b5f [SDK] Support PyTorchJob as a Trial Worker (#2512) 
* [SDK] Support PyTorchJob as Trial Worker

Signed-off-by: Andrey Velichkevich <[email protected]>

* Fix pod spec for Job

Signed-off-by: Andrey Velichkevich <[email protected]>

* Set default restart_policy to Never

Signed-off-by: Andrey Velichkevich <[email protected]>

* Fix primary_container_name for PyTorchJob

Signed-off-by: Andrey Velichkevich <[email protected]>

* Add unit tests for PyTorchJob as Trial

Signed-off-by: Andrey Velichkevich <[email protected]>

* Add e2e test for PyTorchJob as Trial

Signed-off-by: Andrey Velichkevich <[email protected]>

* Bump kubeflow-training SDK

Signed-off-by: Andrey Velichkevich <[email protected]>

* Deploy Training Operator with server side apply

Signed-off-by: Andrey Velichkevich <[email protected]>

* Decrease CPUs for E2E

Signed-off-by: Andrey Velichkevich <[email protected]>

* Install Training Operator for tune workflow

Signed-off-by: Andrey Velichkevich <[email protected]>

* Fix comments

Signed-off-by: Andrey Velichkevich <[email protected]>

---------

Signed-off-by: Andrey Velichkevich <[email protected]>
 2025-02-13 11:10:36 +00:00
Shao WangandGitHub d0847fa6eb fix: remove remaining MXNet dependency. (#2456) 
Signed-off-by: Electronic-Waste <[email protected]>
 2024-12-02 13:23:57 +00:00
Hezhi (Helen) XieandGitHub 05b6be71bb [GSoC] Update tune API for LLM hyperparameters optimization (#2393) 
* update tune api for llm hyperparameters optimization

Signed-off-by: helenxie-bit <[email protected]>

* resolve conflict

Signed-off-by: helenxie-bit <[email protected]>

* fix the problem of dependency

Signed-off-by: helenxie-bit <[email protected]>

* fix the format of import statement

Signed-off-by: helenxie-bit <[email protected]>

* adjust the blank lines

Signed-off-by: helenxie-bit <[email protected]>

* delete the trainer to reuse it in Training Operator

Signed-off-by: helenxie-bit <[email protected]>

* update constants

Signed-off-by: helenxie-bit <[email protected]>

* update metrics format

Signed-off-by: helenxie-bit <[email protected]>

* update the type of  and

Signed-off-by: helenxie-bit <[email protected]>

* update the message of 'ImportError'

Signed-off-by: helenxie-bit <[email protected]>

* add TODO of PVC creation

Signed-off-by: helenxie-bit <[email protected]>

* update the name of pvc

Signed-off-by: helenxie-bit <[email protected]>

* reuse constants from Training Operator

Signed-off-by: helenxie-bit <[email protected]>

* keep 'parameters' and update validation

Signed-off-by: helenxie-bit <[email protected]>

* update for test

Signed-off-by: helenxie-bit <[email protected]>

* reuse 'get_container_spec' and 'get_pod_template_spec' from Training Operator

Signed-off-by: helenxie-bit <[email protected]>

* format with black

Signed-off-by: helenxie-bit <[email protected]>

* fix Lint error

Signed-off-by: helenxie-bit <[email protected]>

* fix Lint errors

Signed-off-by: helenxie-bit <[email protected]>

* delete types

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* update format

Signed-off-by: helenxie-bit <[email protected]>

* update format

Signed-off-by: helenxie-bit <[email protected]>

* fix e2e test error

Signed-off-by: helenxie-bit <[email protected]>

* add TODO

Signed-off-by: helenxie-bit <[email protected]>

* format with max line length

Signed-off-by: helenxie-bit <[email protected]>

* format docstring

Signed-off-by: helenxie-bit <[email protected]>

* update format

Signed-off-by: helenxie-bit <[email protected]>

* add helper functions

Signed-off-by: helenxie-bit <[email protected]>

* update format

Signed-off-by: helenxie-bit <[email protected]>

* update format

Signed-off-by: helenxie-bit <[email protected]>

* run test again

Signed-off-by: helenxie-bit <[email protected]>

* run test again

Signed-off-by: helenxie-bit <[email protected]>

* run test again

Signed-off-by: helenxie-bit <[email protected]>

* fix dict substitution in training_parameters

Signed-off-by: helenxie-bit <[email protected]>

* fix typo

Signed-off-by: helenxie-bit <[email protected]>

* resolve conflicts and add check for case of no parameters

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* fix flake8 error

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* update isort file to black and fix typo

Signed-off-by: helenxie-bit <[email protected]>

* modify the set of metrics format

Signed-off-by: helenxie-bit <[email protected]>

* update tune API

Signed-off-by: helenxie-bit <[email protected]>

* add types.TrainerResources class

Signed-off-by: helenxie-bit <[email protected]>

* fix flake8 error

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* resolve conflict

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* delete properties of 'TrainerResources'

Signed-off-by: helenxie-bit <[email protected]>

* fix format error

Signed-off-by: helenxie-bit <[email protected]>

* update types

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* add import of 'TrainerResources' in '__init__.py' of katib

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* revert changes and rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* check pvc and pv status of katib deployments

Signed-off-by: helenxie-bit <[email protected]>

* check pvc and pv status of katib deployments

Signed-off-by: helenxie-bit <[email protected]>

* recommit changes

Signed-off-by: helenxie-bit <[email protected]>

* update minikube version when setup

Signed-off-by: helenxie-bit <[email protected]>

* delete the code that disables formatting for the tune function

Signed-off-by: helenxie-bit <[email protected]>

* update according to andrey's feedback

Signed-off-by: helenxie-bit <[email protected]>

* add helper function in utils

Signed-off-by: helenxie-bit <[email protected]>

* fix format

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* move metrics_collector_spec back & update helper functions & add return type for helper functions

Signed-off-by: helenxie-bit <[email protected]>

* rerun tests

Signed-off-by: helenxie-bit <[email protected]>

* fix some typos

Signed-off-by: helenxie-bit <[email protected]>

* simplify the definition of 'TrainerResources'

Signed-off-by: helenxie-bit <[email protected]>

---------

Signed-off-by: helenxie-bit <[email protected]>
 2024-09-03 11:35:14 +00:00
Shao WangandGitHub 62c20c8893 [SDK] fix grpc related bugs in Python SDK (#2398) 
* fix: fix bugs in report_metrics.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: fix bugs in tune.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: fix bugs in get_trial_metrics.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: update .gitignore and setup.py.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: update Makefile.

Signed-off-by: Electronic-Waste <[email protected]>

* feat: add report_metrics_test.py.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: fix lint error.

Signed-off-by: Electronic-Waste <[email protected]>

* feat: add UTs for get_trial_metrics.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: update post_gen.py.

Signed-off-by: Electronic-Waste <[email protected]>

* refactor: rebase to master.

Signed-off-by: Electronic-Waste <[email protected]>

* test(sdk): use single katib_client.

Signed-off-by: Electronic-Waste <[email protected]>

* fix(sdk): add TODO for import rewrite.

Signed-off-by: Electronic-Waste <[email protected]>

* fix(sdk): fix lint error with black.

Signed-off-by: Electronic-Waste <[email protected]>

* fix(sdk): fix lint error with isort.

Signed-off-by: Electronic-Waste <[email protected]>

* fix(sdk): reformat import in katib_client_test.py.

Signed-off-by: Electronic-Waste <[email protected]>

---------

Signed-off-by: Electronic-Waste <[email protected]>
 2024-08-23 11:12:58 +00:00
Ignas BaranauskasandGitHub bcb199ddc3 Enhance pre-commit hooks with flake8 and black (#2407) 
* Add black formater and flake8 linter to pre-commit

Also add's the flake8 config file

Signed-off-by: Ignas Baranauskas <[email protected]>

* Fixes black formating

Signed-off-by: Ignas Baranauskas <[email protected]>

* Fixes flake8 linting errors

Signed-off-by: Ignas Baranauskas <[email protected]>

---------

Signed-off-by: Ignas Baranauskas <[email protected]>
 2024-08-16 10:13:28 +00:00
Shao WangandGitHub 1e06458e1f [GSoC] Add New Parameter in tune (#2369) 
* chore: add metrics_collector_config in tune function.

Signed-off-by: Electronic-Waste <[email protected]>

* rebase: rebase feat/new-param-tune to master.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: add metrics collector kind list in comment.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: always pass Trial name to the training container.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: delete passing env variable logics in katib_client.py

Signed-off-by: Electronic-Waste <[email protected]>

* fix: passing env variable KATIB_TRIAL_NAME in the webhook of pod.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: pass env variable KATIB_TRIAL_NAME only to the primary container.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: add report_metrics in post_gen.py.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: change nil error to allErrs(deleted by accident).

Signed-off-by: Electronic-Waste <[email protected]>

* fix: fix lint error in inject_webhook.go.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: wrap env variables passing logics into mutatePodEnv.

Signed-off-by: Electronic-Waste <[email protected]>

* chore: add unit tests for mutatePodEnv.

Signed-off-by: Electronic-Waste <[email protected]>

* fix: delete protocmp.

Signed-off-by: Electronic-Waste <[email protected]>

---------

Signed-off-by: Electronic-Waste <[email protected]>
 2024-07-18 17:51:57 +00:00
Andrey VelichkevichandGitHub 00faa70d8e SDK: Import all Kubernetes Models (#2148) 2023-04-20 16:56:40 +00:00
Andrey VelichkevichGitHubYuki Iwai
 		b96916f6fc Create Tune API in the Katib SDK (#1951) 
* Create Tune API in the Katib SDK

* Add Final to consts
Modify packages_to_install doc
Create validate objective function

* Add GPU TF Image
Change k8s version package

* Create search module

* Fix link in README

Co-authored-by: Yuki Iwai <[email protected]>

* Fix licence date

Co-authored-by: Yuki Iwai <[email protected]>
 2022-10-04 21:19:22 +00:00
Yuki IwaiandGitHub 152f19185e Add shellcheck (#1857) 
* add shellcheck

* fix /test/e2e/v1beta1/scripts/run-e2e-experiment.sh

* fix test/e2e/v1beta1/scripts/setup-katib.sh

* fix pkg/apis/manager/[v1beta1|health]/build.sh

* fix scripts/v1beta1/deploy.sh

* fix scripts/v1beta1/release.sh

* fix scripts/v1beta1/update-images.sh

* fix scripts/v1beta1/undeploy.sh

* fix hack/update-openapigen.sh

* update boilerplate

* fix hack/gen-python-sdk/gen-sdk.sh

* fix hack/update-codegen.sh

* fix hack/update-mockgen.sh

* use /usr/bin/env bash

* add path to update-boilerplate.sh

* fix script to update boilerplate

* fix comment
 2022-05-03 01:15:00 +00:00
Jaeyeon KimGitHubAndrey Velichkevich
 		e09ea1f155 SDK: change list apis to return objects as default (#1630) 
* SDK: change list apis to return objects as default

- change list_trials, list_experiments to return list of objects as
 a default
- also, give 'in_short' parameter for who wants only name and status
 as before

* [enh]: change return type from List[dict] to List[V1beta1Experiment]

* [enh]: deserialize dict to katib's custom class

* [docs]: refactor KatibClient docs

* change deserialize method location to utils

* remove useless import

* Add objects necessary to deserilization in swagger

* use fakeresponse rather than duplicating codes

* Update sdk/python/v1beta1/kubeflow/katib/utils/utils.py

Co-authored-by: Andrey Velichkevich <[email protected]>

* Update sdk/python/v1beta1/kubeflow/katib/utils/utils.py

Co-authored-by: Andrey Velichkevich <[email protected]>

Co-authored-by: Andrey Velichkevich <[email protected]>
 2021-08-30 22:42:03 -07:00
Andrey VelichkevichandGitHub a59cd47f1d Update Katib SDK with OpenAPI generator (#1572) 
* Use openapi JAR

* Remove test from SDK

* Use k8s>=12.0 version
 2021-07-29 08:35:19 -07:00
Andrey VelichkevichandGitHub 558e1c5aeb Add script to update boilerplate (#1491) 
* Update boilerplate for clients

* Modify not format boilerplate

* Modify boilerplate script for go files

* Update boilerplate manually for go files

* Modify script

* Generate boilerplate for Go files

* Add script for Python files

* Generate boilerplate for the Py files

* Include shell

* Generate boilerplate for shell scripts

* Add to makefile

* Fix comments

* Change comments
 2021-03-19 23:48:17 -07:00
Andrey VelichkevichandGitHub 2e226de7e5 Remove v1alpha3 version (#1396) 
* Remove v1alpha3 files

* Modify SDK

* Change dict() to object
 2020-11-30 06:48:50 -08:00
Andrey VelichkevichandGitHub 2992c5706f Upload python SDK version (#1335) 
* Upload 0.0.4 SDK version

* Fix doc links

* Fix links in README

* Modify tables

* Fix link

* Remove

* Modify client and gen script

* Update version to 0.0.5

* Run CI

* Add Katib client to init
 2020-09-15 02:42:07 -07:00
Andrey VelichkevichandGitHub f200a2828d Update generate script with SDK (#1323) 
* Update generate script

* Capitalise API

* Remove verbose
 2020-09-04 13:47:42 -07:00
Xu XiaoandGitHub 6c07d26b78 [python sdk] add v1beta1 models (#1252) 
* [python sdk] add v1beta1 models

* upgrade version of python SDK to 0.0.3

* remove v1Alpha3 python sdk

* add some python models manually: v1Time and V1UnstructuredUnstructured

* bring back v1alpha3

* create separate python sdk for v1alpha3 and v1beta1

* move on

* release pkg on pypi.org

* remove dist files

* refine
 2020-08-18 03:11:32 -07:00