mirror of
https://github.com/hanzo-ml/katib.git
synced 2026-09-24 00:31:27 +00:00
feat: brand-neutral rebrand (consume @<org>/brand contract)
5 new root markdown files. UPSTREAM_README.md preserves original. No upstream code modified. Branding consumed at runtime from a brand package per BRAND.md (@hanzo/brand, @luxfi/brand, @zooai/brand, etc.). Multi-tenant via JWT tenant claim per MULTI_TENANT.md. Apache-2.0 preserved; NOTICE attributes upstream.
This commit is contained in:
@@ -0,0 +1,59 @@
|
||||
# Brand package contract
|
||||
|
||||
This fork (and every fork in the `hanzo-ml` estate) consumes brand
|
||||
configuration from a runtime brand package conforming to a shared
|
||||
schema.
|
||||
|
||||
## Schema
|
||||
|
||||
A brand package ships:
|
||||
|
||||
```
|
||||
@<org>/brand/
|
||||
brand.json # source of truth: brand, chains, iam, rpc, walletConnect
|
||||
dist/
|
||||
index.{js,mjs,d.ts} # exports: loadBrand(), brand singleton, theme tokens
|
||||
loader.{js,mjs,d.ts} # loadBrand() runtime
|
||||
assets/
|
||||
logo/logo.svg
|
||||
logo/favicon.png
|
||||
fonts/
|
||||
DESIGN.md # design spec (theming, typography, density)
|
||||
```
|
||||
|
||||
## `brand.json` top-level keys
|
||||
|
||||
| Key | Purpose |
|
||||
|---|---|
|
||||
| `brand` | name, title, domains, social links, theme tokens (light/dark), logoUrl, faviconUrl, supportEmail |
|
||||
| `chains` | supported chain IDs, default chain |
|
||||
| `iam` | issuer, jwksUrl, tenantClaim, tenantHeader |
|
||||
| `rpc` | RPC endpoint matrix per chain/env |
|
||||
| `walletConnect` | WalletConnect project ID + metadata |
|
||||
| `api` | API endpoint matrix per env |
|
||||
|
||||
## Runtime contract
|
||||
|
||||
```typescript
|
||||
import { brand, loadBrand } from '@<org>/brand/loader'
|
||||
|
||||
await loadBrand() // hydrates the singleton from /brand.json
|
||||
console.log(brand.name) // e.g. "Lux Exchange" / "Hanzo AI" / "Zoo"
|
||||
console.log(brand.appDomain)
|
||||
console.log(brand.iam.jwksUrl)
|
||||
```
|
||||
|
||||
## Reseller deployment
|
||||
|
||||
A reseller takes this fork, publishes their own brand package
|
||||
(`@<their-org>/brand`) with their assets + `brand.json`, sets:
|
||||
|
||||
```bash
|
||||
BRAND_PACKAGE=@<their-org>/brand
|
||||
```
|
||||
|
||||
The fork deploys under their brand with zero source changes.
|
||||
|
||||
## Available brand packages
|
||||
|
||||
See README.md table.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Design
|
||||
|
||||
The design system is defined in the brand package's `DESIGN.md`. Each
|
||||
brand publishes its own design spec; this fork delegates to whichever
|
||||
brand package is configured via `BRAND_PACKAGE`.
|
||||
|
||||
## Layered design system
|
||||
|
||||
| Layer | Where |
|
||||
|---|---|
|
||||
| Tokens (colors, typography, spacing, density) | brand package `brand.json` + `DESIGN.md` |
|
||||
| Components (buttons, forms, cards) | this fork's frontend (where applicable) |
|
||||
| Brand assets (logos, favicons, fonts) | brand package `assets/` |
|
||||
| Theme application | brand package `loadBrand()` + framework integration |
|
||||
|
||||
## Why no design system in this fork
|
||||
|
||||
The design system lives one level up so all forks in the lifecycle
|
||||
estate share consistent components and tokens without duplication. A
|
||||
reseller swaps the brand package; design tokens update without code
|
||||
changes.
|
||||
|
||||
## See also
|
||||
|
||||
- The configured brand package's `DESIGN.md` (e.g. `@luxfi/brand/DESIGN.md`)
|
||||
- BRAND.md in this fork
|
||||
@@ -0,0 +1,24 @@
|
||||
# Changes from upstream
|
||||
|
||||
- Forked into the `hanzo-ml` GitHub organization
|
||||
- Brand-neutral README + BRAND.md + DESIGN.md + MULTI_TENANT.md +
|
||||
HANZO_CHANGES.md + NOTICE added at repo root
|
||||
- `UPSTREAM_README.md` preserves the original project documentation
|
||||
- No upstream code modified; `merge-upstream` keeps this current
|
||||
|
||||
## Brand-neutral guarantee
|
||||
|
||||
This fork hardcodes NO product brand. All branding is consumed at
|
||||
runtime from a `@<org>/brand` package per the contract in BRAND.md.
|
||||
Any reseller can deploy this fork under their own brand with zero
|
||||
source changes.
|
||||
|
||||
## Hanzo control plane reference
|
||||
|
||||
The canonical control plane for the ML lifecycle estate is the Rust
|
||||
operator at [`hanzoai/operator`](https://github.com/hanzoai/operator).
|
||||
This fork is either:
|
||||
- the shipped frontend (for `pipelines`)
|
||||
- a reference implementation for the Rust operator's reconciler
|
||||
patterns (for `kubeflow`, `kuberay`, `trainer`, `katib`,
|
||||
`spark-operator`, `hub`, `kserve`)
|
||||
@@ -0,0 +1,34 @@
|
||||
# Multi-tenant isolation contract
|
||||
|
||||
This fork is multi-tenant via a Hanzo-compatible IAM JWT.
|
||||
|
||||
## Tenant scope
|
||||
|
||||
Every request carries `Authorization: Bearer <jwt>`. The JWT is
|
||||
validated against the brand package's `iam.jwksUrl`. The `iam.tenantClaim`
|
||||
(default `org_id`) determines the tenant scope for the request.
|
||||
|
||||
## Header propagation
|
||||
|
||||
After validation, the gateway/middleware strips any client-supplied
|
||||
identity headers and emits the validated tenant via
|
||||
`iam.tenantHeader` (default `X-Org-Id`). Downstream services trust
|
||||
only the gateway-minted header.
|
||||
|
||||
## Storage isolation
|
||||
|
||||
Per-tenant data lives at `${DATA_DIR}/orgs/${tenant_id}/...`.
|
||||
Cross-tenant access is forbidden by default. Services check the JWT
|
||||
tenant matches the resource owner before any read/write.
|
||||
|
||||
## Configurability
|
||||
|
||||
`iam.tenantClaim` and `iam.tenantHeader` are brand-package-configurable
|
||||
so resellers using a non-standard JWT shape can override the defaults
|
||||
without code changes.
|
||||
|
||||
## Reference
|
||||
|
||||
- HIP-0026 — Identity and Access Management Standard
|
||||
- HIP-0106 — Unified Hanzo Cloud Binary (tenant-routing section)
|
||||
- HIP-0302 — Per-tenant SQLite + ZapDB Durability
|
||||
@@ -0,0 +1,12 @@
|
||||
This work is a fork of an upstream open-source project licensed under
|
||||
the Apache License, Version 2.0 (the "License"). The full upstream
|
||||
license text and copyright notices are preserved in LICENSE and
|
||||
throughout the source tree.
|
||||
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Modifications by the hanzo-ml community are documented in
|
||||
HANZO_CHANGES.md. The fork is brand-neutral; branding is consumed at
|
||||
runtime from a `@<org>/brand` package per the contract in BRAND.md.
|
||||
@@ -1,224 +1,95 @@
|
||||
# Kubeflow Katib
|
||||
# Sweep (HPO/AutoML)
|
||||
|
||||
[](https://github.com/kubeflow/katib/actions/workflows/test-go.yaml?branch=master)
|
||||
[](https://coveralls.io/github/kubeflow/katib?branch=master)
|
||||
[](https://goreportcard.com/report/github.com/kubeflow/katib)
|
||||
[](https://github.com/kubeflow/katib/releases)
|
||||
[](https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels)
|
||||
[](https://www.bestpractices.dev/projects/9941)
|
||||
[](https://app.fossa.com/projects/git%2Bgithub.com%2Fkubeflow%2Fkatib?ref=badge_shield)
|
||||
Hyperparameter tuning, AutoML, early stopping, neural architecture search.
|
||||
|
||||
<h1 align="center">
|
||||
<img src="./docs/images/logo-title.png" alt="logo" width="200">
|
||||
<br>
|
||||
</h1>
|
||||
A brand-neutral fork in the [hanzo-ml](https://github.com/hanzo-ml)
|
||||
organization — the open-source ML lifecycle estate. Branding is
|
||||
consumed at runtime from a brand package via the
|
||||
[`@<org>/brand`](#brand-package) contract; the same code deploys under
|
||||
any white-label brand with zero source changes.
|
||||
|
||||
Kubeflow Katib is a Kubernetes-native project for automated machine learning (AutoML).
|
||||
Katib supports
|
||||
[Hyperparameter Tuning](https://en.wikipedia.org/wiki/Hyperparameter_optimization),
|
||||
[Early Stopping](https://en.wikipedia.org/wiki/Early_stopping) and
|
||||
[Neural Architecture Search](https://en.wikipedia.org/wiki/Neural_architecture_search).
|
||||
## Brand package
|
||||
|
||||
Katib is the project which is agnostic to machine learning (ML) frameworks.
|
||||
It can tune hyperparameters of applications written in any language of the
|
||||
users’ choice and natively supports many ML frameworks, such as
|
||||
[TensorFlow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), [XGBoost](https://xgboost.readthedocs.io/en/latest/), and others.
|
||||
This fork imports brand configuration from a runtime brand package
|
||||
following the contract used across the open-source ML estate. Set
|
||||
`BRAND_PACKAGE` to the npm package name; the fork's frontend (where
|
||||
applicable) calls `loadBrand()` at boot to hydrate the singleton from
|
||||
that package's `brand.json`.
|
||||
|
||||
Katib can perform training jobs using any Kubernetes
|
||||
[Custom Resources](https://www.kubeflow.org/docs/components/katib/trial-template/)
|
||||
with out of the box support for [Kubeflow Training Operator](https://github.com/kubeflow/training-operator),
|
||||
[Argo Workflows](https://github.com/argoproj/argo-workflows), [Tekton Pipelines](https://github.com/tektoncd/pipeline)
|
||||
and many more.
|
||||
Available brand packages:
|
||||
|
||||
Katib stands for `secretary` in Arabic.
|
||||
| Package | Brand |
|
||||
|---|---|
|
||||
| `@hanzo/brand` | default / Hanzo AI |
|
||||
| `@luxfi/brand` | Lux Finance |
|
||||
| `@zooai/brand` | Zoo Labs |
|
||||
| `@osage/brand` | Osage |
|
||||
| `@parsdao/brand` | Pars DAO |
|
||||
| `@cyrusdao/brand` | Cyrus DAO |
|
||||
| `@onyx-plus/brand` | Onyx Plus |
|
||||
| `@migaprotocol/brand` | Miga Protocol |
|
||||
| `@vccross/brand` | VC Cross |
|
||||
| `@mlc/brand` | MLC |
|
||||
| `@zenlm/brand` | Zen LM |
|
||||
|
||||
## Search Algorithms
|
||||
Or any custom reseller package conforming to the same schema.
|
||||
|
||||
Katib supports several search algorithms. Follow the
|
||||
[Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/user-guides/hp-tuning/configure-algorithm/#hp-tuning-algorithms)
|
||||
to know more about each algorithm and check the
|
||||
[this guide](https://www.kubeflow.org/docs/components/katib/user-guides/hp-tuning/configure-algorithm/#use-custom-algorithm-in-katib)
|
||||
to implement your custom algorithm.
|
||||
## Multi-tenant via IAM
|
||||
|
||||
<table>
|
||||
<tbody>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<b>Hyperparameter Tuning</b>
|
||||
</td>
|
||||
<td>
|
||||
<b>Neural Architecture Search</b>
|
||||
</td>
|
||||
<td>
|
||||
<b>Early Stopping</b>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#random-search">Random Search</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#neural-architecture-search-based-on-enas">ENAS</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/early-stopping/#median-stopping-rule">Median Stop</a>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#grid-search">Grid Search</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#differentiable-architecture-search-darts">DARTS</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#bayesian-optimization">Bayesian Optimization</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#tree-of-parzen-estimators-tpe">TPE</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#multivariate-tpe">Multivariate TPE</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#covariance-matrix-adaptation-evolution-strategy-cma-es">CMA-ES</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#sobols-quasirandom-sequence">Sobol's Quasirandom Sequence</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#hyperband">HyperBand</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#pbt">Population Based Training</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
Every request carries a JWT. The brand package's `iam` block specifies:
|
||||
|
||||
To perform the above algorithms Katib supports the following frameworks:
|
||||
- `issuer` — JWT issuer URL
|
||||
- `jwksUrl` — JWKS endpoint
|
||||
- `tenantClaim` — JWT claim with the org/tenant ID (default `org_id`)
|
||||
- `tenantHeader` — HTTP header that propagates the validated tenant
|
||||
ID (default `X-Org-Id`)
|
||||
|
||||
- [Goptuna](https://github.com/c-bata/goptuna)
|
||||
- [Hyperopt](https://github.com/hyperopt/hyperopt)
|
||||
- [Optuna](https://github.com/optuna/optuna)
|
||||
- [Scikit Optimize](https://github.com/scikit-optimize/scikit-optimize)
|
||||
The tenant ID scopes all storage, queries, and resource ownership.
|
||||
Cross-tenant access is forbidden by default.
|
||||
|
||||
## Prerequisites
|
||||
## Quick start (reseller deployment)
|
||||
|
||||
Please check [the official Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/installation/#prerequisites)
|
||||
for prerequisites to install Katib.
|
||||
|
||||
## Installation
|
||||
|
||||
Please follow [the Kubeflow Katib guide](https://www.kubeflow.org/docs/components/katib/installation/#installing-katib)
|
||||
for the detailed instructions on how to install Katib.
|
||||
|
||||
### Installing the Control Plane
|
||||
|
||||
Run the following command to install the latest stable release of Katib control plane:
|
||||
|
||||
```
|
||||
kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=v0.17.0"
|
||||
```bash
|
||||
export BRAND_PACKAGE="@<your-org>/brand" # e.g. @luxfi/brand
|
||||
```
|
||||
|
||||
Run the following command to install the latest changes of Katib control plane:
|
||||
The frontend (this fork's case: `pipelines` ships the React DAG UI;
|
||||
the other 7 are read-only reference forks) loads the brand at boot.
|
||||
The backend reads the brand package's `iam` block for JWKS + tenant
|
||||
configuration.
|
||||
|
||||
```
|
||||
kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=master"
|
||||
## Role
|
||||
|
||||
Read-only reference for the Rust operator's `Sweep` CRD reconciler.
|
||||
|
||||
The canonical control plane for the ML lifecycle estate is the Rust
|
||||
operator at [`hanzoai/operator`](https://github.com/hanzoai/operator).
|
||||
See [HIP-0109](https://github.com/hanzoai/HIPs/blob/main/HIPs/hip-0109-hanzo-ml-cloud-toolkit.md)
|
||||
for the lifecycle CRD set.
|
||||
|
||||
## Upstream sync
|
||||
|
||||
This fork stays current with upstream via the GitHub merge-upstream
|
||||
API. No upstream code is modified in this fork; only the 5 markdown
|
||||
files at root are added.
|
||||
|
||||
```bash
|
||||
gh api -X POST /repos/hanzo-ml/katib/merge-upstream \
|
||||
-f branch=master
|
||||
```
|
||||
|
||||
For the Katib Experiments check the [complete examples list](./examples/v1beta1).
|
||||
See [UPSTREAM_README.md](./UPSTREAM_README.md) for the original
|
||||
project documentation.
|
||||
|
||||
### Installing the Python SDK
|
||||
## License
|
||||
|
||||
Katib implements [a Python SDK](https://pypi.org/project/kubeflow-katib/) to simplify creation of
|
||||
hyperparameter tuning jobs for Data Scientists.
|
||||
Apache-2.0. See [NOTICE](./NOTICE) for attribution.
|
||||
|
||||
Run the following command to install the latest stable release of Katib SDK:
|
||||
## See also
|
||||
|
||||
```sh
|
||||
pip install -U kubeflow-katib
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
|
||||
Please refer to [the getting started guide](https://www.kubeflow.org/docs/components/katib/getting-started/#getting-started-with-katib-python-sdk)
|
||||
to quickly create your first hyperparameter tuning Experiment using the Python SDK.
|
||||
|
||||
## Community
|
||||
|
||||
The following links provide information on how to get involved in the community:
|
||||
|
||||
- Attend [the bi-weekly AutoML and Training Working Group](https://bit.ly/2PWVCkV)
|
||||
community meeting.
|
||||
- Join our [`#kubeflow-katib`](https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels)
|
||||
Slack channel.
|
||||
- Check out [who is using Katib](ADOPTERS.md) and [presentations about Katib project](docs/presentations.md).
|
||||
|
||||
## Contributing
|
||||
|
||||
Please refer to the [CONTRIBUTING guide](CONTRIBUTING.md).
|
||||
|
||||
## Citation
|
||||
|
||||
If you use Katib in a scientific publication, we would appreciate
|
||||
citations to the following paper:
|
||||
|
||||
[A Scalable and Cloud-Native Hyperparameter Tuning System](https://arxiv.org/abs/2006.02085), George _et al._, arXiv:2006.02085, 2020.
|
||||
|
||||
Bibtex entry:
|
||||
|
||||
```
|
||||
@misc{george2020katib,
|
||||
title={A Scalable and Cloud-Native Hyperparameter Tuning System},
|
||||
author={Johnu George and Ce Gao and Richard Liu and Hou Gang Liu and Yuan Tang and Ramdoot Pydipaty and Amit Kumar Saha},
|
||||
year={2020},
|
||||
eprint={2006.02085},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.DC}
|
||||
}
|
||||
```
|
||||
- [DESIGN.md](./DESIGN.md) — design system reference (mirrors the brand
|
||||
package's design spec)
|
||||
- [BRAND.md](./BRAND.md) — brand package contract and reseller guide
|
||||
- [MULTI_TENANT.md](./MULTI_TENANT.md) — tenant isolation contract
|
||||
- [HANZO_CHANGES.md](./HANZO_CHANGES.md) — divergence from upstream
|
||||
- [HIP-0109 Hanzo ML Cloud Toolkit](https://github.com/hanzoai/HIPs/blob/main/HIPs/hip-0109-hanzo-ml-cloud-toolkit.md)
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
# Kubeflow Katib
|
||||
|
||||
[](https://github.com/kubeflow/katib/actions/workflows/test-go.yaml?branch=master)
|
||||
[](https://coveralls.io/github/kubeflow/katib?branch=master)
|
||||
[](https://goreportcard.com/report/github.com/kubeflow/katib)
|
||||
[](https://github.com/kubeflow/katib/releases)
|
||||
[](https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels)
|
||||
[](https://www.bestpractices.dev/projects/9941)
|
||||
[](https://app.fossa.com/projects/git%2Bgithub.com%2Fkubeflow%2Fkatib?ref=badge_shield)
|
||||
|
||||
<h1 align="center">
|
||||
<img src="./docs/images/logo-title.png" alt="logo" width="200">
|
||||
<br>
|
||||
</h1>
|
||||
|
||||
Kubeflow Katib is a Kubernetes-native project for automated machine learning (AutoML).
|
||||
Katib supports
|
||||
[Hyperparameter Tuning](https://en.wikipedia.org/wiki/Hyperparameter_optimization),
|
||||
[Early Stopping](https://en.wikipedia.org/wiki/Early_stopping) and
|
||||
[Neural Architecture Search](https://en.wikipedia.org/wiki/Neural_architecture_search).
|
||||
|
||||
Katib is the project which is agnostic to machine learning (ML) frameworks.
|
||||
It can tune hyperparameters of applications written in any language of the
|
||||
users’ choice and natively supports many ML frameworks, such as
|
||||
[TensorFlow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), [XGBoost](https://xgboost.readthedocs.io/en/latest/), and others.
|
||||
|
||||
Katib can perform training jobs using any Kubernetes
|
||||
[Custom Resources](https://www.kubeflow.org/docs/components/katib/trial-template/)
|
||||
with out of the box support for [Kubeflow Training Operator](https://github.com/kubeflow/training-operator),
|
||||
[Argo Workflows](https://github.com/argoproj/argo-workflows), [Tekton Pipelines](https://github.com/tektoncd/pipeline)
|
||||
and many more.
|
||||
|
||||
Katib stands for `secretary` in Arabic.
|
||||
|
||||
## Search Algorithms
|
||||
|
||||
Katib supports several search algorithms. Follow the
|
||||
[Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/user-guides/hp-tuning/configure-algorithm/#hp-tuning-algorithms)
|
||||
to know more about each algorithm and check the
|
||||
[this guide](https://www.kubeflow.org/docs/components/katib/user-guides/hp-tuning/configure-algorithm/#use-custom-algorithm-in-katib)
|
||||
to implement your custom algorithm.
|
||||
|
||||
<table>
|
||||
<tbody>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<b>Hyperparameter Tuning</b>
|
||||
</td>
|
||||
<td>
|
||||
<b>Neural Architecture Search</b>
|
||||
</td>
|
||||
<td>
|
||||
<b>Early Stopping</b>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#random-search">Random Search</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#neural-architecture-search-based-on-enas">ENAS</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/early-stopping/#median-stopping-rule">Median Stop</a>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#grid-search">Grid Search</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#differentiable-architecture-search-darts">DARTS</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#bayesian-optimization">Bayesian Optimization</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#tree-of-parzen-estimators-tpe">TPE</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#multivariate-tpe">Multivariate TPE</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#covariance-matrix-adaptation-evolution-strategy-cma-es">CMA-ES</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#sobols-quasirandom-sequence">Sobol's Quasirandom Sequence</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#hyperband">HyperBand</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>
|
||||
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#pbt">Population Based Training</a>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
<td>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
To perform the above algorithms Katib supports the following frameworks:
|
||||
|
||||
- [Goptuna](https://github.com/c-bata/goptuna)
|
||||
- [Hyperopt](https://github.com/hyperopt/hyperopt)
|
||||
- [Optuna](https://github.com/optuna/optuna)
|
||||
- [Scikit Optimize](https://github.com/scikit-optimize/scikit-optimize)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Please check [the official Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/installation/#prerequisites)
|
||||
for prerequisites to install Katib.
|
||||
|
||||
## Installation
|
||||
|
||||
Please follow [the Kubeflow Katib guide](https://www.kubeflow.org/docs/components/katib/installation/#installing-katib)
|
||||
for the detailed instructions on how to install Katib.
|
||||
|
||||
### Installing the Control Plane
|
||||
|
||||
Run the following command to install the latest stable release of Katib control plane:
|
||||
|
||||
```
|
||||
kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=v0.17.0"
|
||||
```
|
||||
|
||||
Run the following command to install the latest changes of Katib control plane:
|
||||
|
||||
```
|
||||
kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=master"
|
||||
```
|
||||
|
||||
For the Katib Experiments check the [complete examples list](./examples/v1beta1).
|
||||
|
||||
### Installing the Python SDK
|
||||
|
||||
Katib implements [a Python SDK](https://pypi.org/project/kubeflow-katib/) to simplify creation of
|
||||
hyperparameter tuning jobs for Data Scientists.
|
||||
|
||||
Run the following command to install the latest stable release of Katib SDK:
|
||||
|
||||
```sh
|
||||
pip install -U kubeflow-katib
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
|
||||
Please refer to [the getting started guide](https://www.kubeflow.org/docs/components/katib/getting-started/#getting-started-with-katib-python-sdk)
|
||||
to quickly create your first hyperparameter tuning Experiment using the Python SDK.
|
||||
|
||||
## Community
|
||||
|
||||
The following links provide information on how to get involved in the community:
|
||||
|
||||
- Attend [the bi-weekly AutoML and Training Working Group](https://bit.ly/2PWVCkV)
|
||||
community meeting.
|
||||
- Join our [`#kubeflow-katib`](https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels)
|
||||
Slack channel.
|
||||
- Check out [who is using Katib](ADOPTERS.md) and [presentations about Katib project](docs/presentations.md).
|
||||
|
||||
## Contributing
|
||||
|
||||
Please refer to the [CONTRIBUTING guide](CONTRIBUTING.md).
|
||||
|
||||
## Citation
|
||||
|
||||
If you use Katib in a scientific publication, we would appreciate
|
||||
citations to the following paper:
|
||||
|
||||
[A Scalable and Cloud-Native Hyperparameter Tuning System](https://arxiv.org/abs/2006.02085), George _et al._, arXiv:2006.02085, 2020.
|
||||
|
||||
Bibtex entry:
|
||||
|
||||
```
|
||||
@misc{george2020katib,
|
||||
title={A Scalable and Cloud-Native Hyperparameter Tuning System},
|
||||
author={Johnu George and Ce Gao and Richard Liu and Hou Gang Liu and Yuan Tang and Ramdoot Pydipaty and Amit Kumar Saha},
|
||||
year={2020},
|
||||
eprint={2006.02085},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.DC}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user