mirror of
https://github.com/hanzoai/k8s.git
synced 2026-10-02 15:40:05 +00:00
initial commit, on zip v1.20.0
The tree was staged but never committed — this is the repo's first commit, so it carries both the existing content and the zip v1.18.8 -> v1.20.0 migration. Migration: Prepare() is replaced by Build() error, and both test harnesses assert the verdict rather than discarding it. main.go:59's Use call needed no change — the three middleware constructors already return zip.Handler, which satisfies the closed Component interface. The committed projections (k8s.plugin.json, openapi.json, ops/zipdoc_gen.go) are regenerated: v1.20.0 splits an op's prose into summary (first line) + description and zipdoc now emits per-field docs, so all three artifacts move with the bump. Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
Hanzo Dev
@@ -0,0 +1,9 @@
|
|||||||
|
name: CI/CD
|
||||||
|
on:
|
||||||
|
push: { branches: [main], tags: ["v*"] }
|
||||||
|
pull_request:
|
||||||
|
workflow_dispatch:
|
||||||
|
jobs:
|
||||||
|
cicd:
|
||||||
|
uses: hanzoai/ci/.github/workflows/build.yml@v1
|
||||||
|
secrets: inherit
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
k8s
|
||||||
@@ -0,0 +1,203 @@
|
|||||||
|
Copyright (c) 2026 Hanzo AI Inc.
|
||||||
|
|
||||||
|
Apache License
|
||||||
|
Version 2.0, January 2004
|
||||||
|
http://www.apache.org/licenses/
|
||||||
|
|
||||||
|
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||||
|
|
||||||
|
1. Definitions.
|
||||||
|
|
||||||
|
"License" shall mean the terms and conditions for use, reproduction,
|
||||||
|
and distribution as defined by Sections 1 through 9 of this document.
|
||||||
|
|
||||||
|
"Licensor" shall mean the copyright owner or entity authorized by
|
||||||
|
the copyright owner that is granting the License.
|
||||||
|
|
||||||
|
"Legal Entity" shall mean the union of the acting entity and all
|
||||||
|
other entities that control, are controlled by, or are under common
|
||||||
|
control with that entity. For the purposes of this definition,
|
||||||
|
"control" means (i) the power, direct or indirect, to cause the
|
||||||
|
direction or management of such entity, whether by contract or
|
||||||
|
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||||
|
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||||
|
|
||||||
|
"You" (or "Your") shall mean an individual or Legal Entity
|
||||||
|
exercising permissions granted by this License.
|
||||||
|
|
||||||
|
"Source" form shall mean the preferred form for making modifications,
|
||||||
|
including but not limited to software source code, documentation
|
||||||
|
source, and configuration files.
|
||||||
|
|
||||||
|
"Object" form shall mean any form resulting from mechanical
|
||||||
|
transformation or translation of a Source form, including but
|
||||||
|
not limited to compiled object code, generated documentation,
|
||||||
|
and conversions to other media types.
|
||||||
|
|
||||||
|
"Work" shall mean the work of authorship, whether in Source or
|
||||||
|
Object form, made available under the License, as indicated by a
|
||||||
|
copyright notice that is included in or attached to the work
|
||||||
|
(an example is provided in the Appendix below).
|
||||||
|
|
||||||
|
"Derivative Works" shall mean any work, whether in Source or Object
|
||||||
|
form, that is based on (or derived from) the Work and for which the
|
||||||
|
editorial revisions, annotations, elaborations, or other modifications
|
||||||
|
represent, as a whole, an original work of authorship. For the purposes
|
||||||
|
of this License, Derivative Works shall not include works that remain
|
||||||
|
separable from, or merely link (or bind by name) to the interfaces of,
|
||||||
|
the Work and Derivative Works thereof.
|
||||||
|
|
||||||
|
"Contribution" shall mean any work of authorship, including
|
||||||
|
the original version of the Work and any modifications or additions
|
||||||
|
to that Work or Derivative Works thereof, that is intentionally
|
||||||
|
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||||
|
or by an individual or Legal Entity authorized to submit on behalf of
|
||||||
|
the copyright owner. For the purposes of this definition, "submitted"
|
||||||
|
means any form of electronic, verbal, or written communication sent
|
||||||
|
to the Licensor or its representatives, including but not limited to
|
||||||
|
communication on electronic mailing lists, source code control systems,
|
||||||
|
and issue tracking systems that are managed by, or on behalf of, the
|
||||||
|
Licensor for the purpose of discussing and improving the Work, but
|
||||||
|
excluding communication that is conspicuously marked or otherwise
|
||||||
|
designated in writing by the copyright owner as "Not a Contribution."
|
||||||
|
|
||||||
|
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||||
|
on behalf of whom a Contribution has been received by Licensor and
|
||||||
|
subsequently incorporated within the Work.
|
||||||
|
|
||||||
|
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||||
|
this License, each Contributor hereby grants to You a perpetual,
|
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||||
|
copyright license to reproduce, prepare Derivative Works of,
|
||||||
|
publicly display, publicly perform, sublicense, and distribute the
|
||||||
|
Work and such Derivative Works in Source or Object form.
|
||||||
|
|
||||||
|
3. Grant of Patent License. Subject to the terms and conditions of
|
||||||
|
this License, each Contributor hereby grants to You a perpetual,
|
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||||
|
(except as stated in this section) patent license to make, have made,
|
||||||
|
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||||
|
where such license applies only to those patent claims licensable
|
||||||
|
by such Contributor that are necessarily infringed by their
|
||||||
|
Contribution(s) alone or by combination of their Contribution(s)
|
||||||
|
with the Work to which such Contribution(s) was submitted. If You
|
||||||
|
institute patent litigation against any entity (including a
|
||||||
|
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||||
|
or a Contribution incorporated within the Work constitutes direct
|
||||||
|
or contributory patent infringement, then any patent licenses
|
||||||
|
granted to You under this License for that Work shall terminate
|
||||||
|
as of the date such litigation is filed.
|
||||||
|
|
||||||
|
4. Redistribution. You may reproduce and distribute copies of the
|
||||||
|
Work or Derivative Works thereof in any medium, with or without
|
||||||
|
modifications, and in Source or Object form, provided that You
|
||||||
|
meet the following conditions:
|
||||||
|
|
||||||
|
(a) You must give any other recipients of the Work or
|
||||||
|
Derivative Works a copy of this License; and
|
||||||
|
|
||||||
|
(b) You must cause any modified files to carry prominent notices
|
||||||
|
stating that You changed the files; and
|
||||||
|
|
||||||
|
(c) You must retain, in the Source form of any Derivative Works
|
||||||
|
that You distribute, all copyright, patent, trademark, and
|
||||||
|
attribution notices from the Source form of the Work,
|
||||||
|
excluding those notices that do not pertain to any part of
|
||||||
|
the Derivative Works; and
|
||||||
|
|
||||||
|
(d) If the Work includes a "NOTICE" text file as part of its
|
||||||
|
distribution, then any Derivative Works that You distribute must
|
||||||
|
include a readable copy of the attribution notices contained
|
||||||
|
within such NOTICE file, excluding those notices that do not
|
||||||
|
pertain to any part of the Derivative Works, in at least one
|
||||||
|
of the following places: within a NOTICE text file distributed
|
||||||
|
as part of the Derivative Works; within the Source form or
|
||||||
|
documentation, if provided along with the Derivative Works; or,
|
||||||
|
within a display generated by the Derivative Works, if and
|
||||||
|
wherever such third-party notices normally appear. The contents
|
||||||
|
of the NOTICE file are for informational purposes only and
|
||||||
|
do not modify the License. You may add Your own attribution
|
||||||
|
notices within Derivative Works that You distribute, alongside
|
||||||
|
or as an addendum to the NOTICE text from the Work, provided
|
||||||
|
that such additional attribution notices cannot be construed
|
||||||
|
as modifying the License.
|
||||||
|
|
||||||
|
You may add Your own copyright statement to Your modifications and
|
||||||
|
may provide additional or different license terms and conditions
|
||||||
|
for use, reproduction, or distribution of Your modifications, or
|
||||||
|
for any such Derivative Works as a whole, provided Your use,
|
||||||
|
reproduction, and distribution of the Work otherwise complies with
|
||||||
|
the conditions stated in this License.
|
||||||
|
|
||||||
|
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||||
|
any Contribution intentionally submitted for inclusion in the Work
|
||||||
|
by You to the Licensor shall be under the terms and conditions of
|
||||||
|
this License, without any additional terms or conditions.
|
||||||
|
Notwithstanding the above, nothing herein shall supersede or modify
|
||||||
|
the terms of any separate license agreement you may have executed
|
||||||
|
with Licensor regarding such Contributions.
|
||||||
|
|
||||||
|
6. Trademarks. This License does not grant permission to use the trade
|
||||||
|
names, trademarks, service marks, or product names of the Licensor,
|
||||||
|
except as required for reasonable and customary use in describing the
|
||||||
|
origin of the Work and reproducing the content of the NOTICE file.
|
||||||
|
|
||||||
|
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||||
|
agreed to in writing, Licensor provides the Work (and each
|
||||||
|
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||||
|
implied, including, without limitation, any warranties or conditions
|
||||||
|
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||||
|
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||||
|
appropriateness of using or redistributing the Work and assume any
|
||||||
|
risks associated with Your exercise of permissions under this License.
|
||||||
|
|
||||||
|
8. Limitation of Liability. In no event and under no legal theory,
|
||||||
|
whether in tort (including negligence), contract, or otherwise,
|
||||||
|
unless required by applicable law (such as deliberate and grossly
|
||||||
|
negligent acts) or agreed to in writing, shall any Contributor be
|
||||||
|
liable to You for damages, including any direct, indirect, special,
|
||||||
|
incidental, or consequential damages of any character arising as a
|
||||||
|
result of this License or out of the use or inability to use the
|
||||||
|
Work (including but not limited to damages for loss of goodwill,
|
||||||
|
work stoppage, computer failure or malfunction, or any and all
|
||||||
|
other commercial damages or losses), even if such Contributor
|
||||||
|
has been advised of the possibility of such damages.
|
||||||
|
|
||||||
|
9. Accepting Warranty or Additional Liability. While redistributing
|
||||||
|
the Work or Derivative Works thereof, You may choose to offer,
|
||||||
|
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||||
|
or other liability obligations and/or rights consistent with this
|
||||||
|
License. However, in accepting such obligations, You may act only
|
||||||
|
on Your own behalf and on Your sole responsibility, not on behalf
|
||||||
|
of any other Contributor, and only if You agree to indemnify,
|
||||||
|
defend, and hold each Contributor harmless for any liability
|
||||||
|
incurred by, or claims asserted against, such Contributor by reason
|
||||||
|
of your accepting any such warranty or additional liability.
|
||||||
|
|
||||||
|
END OF TERMS AND CONDITIONS
|
||||||
|
|
||||||
|
APPENDIX: How to apply the Apache License to your work.
|
||||||
|
|
||||||
|
To apply the Apache License to your work, attach the following
|
||||||
|
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||||
|
replaced with your own identifying information. (Don't include
|
||||||
|
the brackets!) The text should be enclosed in the appropriate
|
||||||
|
comment syntax for the file format. We also recommend that a
|
||||||
|
file or class name and description of purpose be included on the
|
||||||
|
same "printed page" as the copyright notice for easier
|
||||||
|
identification within third-party archives.
|
||||||
|
|
||||||
|
Copyright (c) 2026 Hanzo AI Inc.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
you may not use this file except in compliance with the License.
|
||||||
|
You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software
|
||||||
|
distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
See the License for the specific language governing permissions and
|
||||||
|
limitations under the License.
|
||||||
@@ -0,0 +1,190 @@
|
|||||||
|
# hanzoai/k8s — deep notes
|
||||||
|
|
||||||
|
The one Kubernetes owner in the fleet. Read this before changing anything here; the README
|
||||||
|
is the product surface, this is the reasoning.
|
||||||
|
|
||||||
|
## Why the repo exists (and why a library would not have worked)
|
||||||
|
|
||||||
|
Nine places in `hanzoai/cloud` built a Kubernetes client. Measured, not estimated:
|
||||||
|
|
||||||
|
| Holder | Credential | Cluster it assumed |
|
||||||
|
|---|---|---|
|
||||||
|
| `apps/platform` ×2, `apps/deploy` ×2, `apps/provisioning`, `apps/validators`, `apps/cron` | in-cluster ServiceAccount → `KUBECONFIG` | one ambient |
|
||||||
|
| `apps/ml` | in-cluster SA, re-scoped to a token file; federated via `fleet.DynForOrg` | ambient **or** the org's BYO |
|
||||||
|
| `apps/fleet` | **KMS-sealed kubeconfig** per org+project | N registered BYO |
|
||||||
|
| `apps/admin/infra` | **`DO_API_TOKEN` → a cluster-admin kubeconfig per cluster** | N DOKS clusters |
|
||||||
|
| `hanzoai/ai/cluster` | **kubeconfig in a database row**, fed to an apply-any-YAML function | one global mutable singleton |
|
||||||
|
|
||||||
|
Two of those are the reason this is a service and not a shared package:
|
||||||
|
|
||||||
|
1. **`ai/cluster/client.go:210 deployResource`** decoded arbitrary YAML, resolved the GVK
|
||||||
|
through a discovery `RESTMapper`, and create-or-updated **any kind the cluster served**.
|
||||||
|
With a DB-sourced kubeconfig, one edited row reached any object on any cluster. It is
|
||||||
|
DELETED, not ported. Its callers go through `resources.apply` with a named kind.
|
||||||
|
2. **`admin/infra`** held cluster-admin on every cluster on the house account to perform
|
||||||
|
nine operations. `nodes/proxy get` alone is arbitrary kubelet access.
|
||||||
|
|
||||||
|
A library cannot fix either, because the defect is not the code — it is that N binaries
|
||||||
|
hold cluster credentials. One owner, named ops, credentials in KMS, consumers ASK.
|
||||||
|
|
||||||
|
The **deciding** argument, though, is neither of those. It is that **lux runs its own
|
||||||
|
`lux-k8s`**: an org's resources may live on a different cluster from Hanzo's, plus BYO for
|
||||||
|
customers. A single-cluster client, however cleanly factored, cannot express that and would
|
||||||
|
have been rewritten. Hence multi-cluster from the first line.
|
||||||
|
|
||||||
|
## The model
|
||||||
|
|
||||||
|
```
|
||||||
|
Cluster{Name, Org, Provider, Endpoint, Nodes, NvidiaGPU, AmdGPU, Namespaces, Registered, Default}
|
||||||
|
```
|
||||||
|
|
||||||
|
- **No Kubeconfig field and no Credential field.** The ref is DERIVED —
|
||||||
|
`orgs/<org>/k8s/clusters/<name>/kubeconfig` — because a ref stored as data could be
|
||||||
|
edited to name another org's material. That is the same defect as letting a caller name
|
||||||
|
its own org, one level down.
|
||||||
|
- **`Org` is set from the validated caller**, never from an argument, and appears only on
|
||||||
|
replies (where it is a property of the thing described).
|
||||||
|
- **`Namespaces`** bounds a registration made with a scoped credential on a shared cluster.
|
||||||
|
Empty means the whole cluster, which is what an org registering its own cluster gets.
|
||||||
|
|
||||||
|
`registry.Reach(ctx, name) (*Bound, error)` is the only door:
|
||||||
|
|
||||||
|
1. `Tenant(ctx)` — zip's full rule (validated user claim, non-empty org, under
|
||||||
|
`MaxOrgLen`) plus one thing zip cannot know: the org becomes a KMS ref segment, so a
|
||||||
|
value containing `/` or `..` is refused here.
|
||||||
|
2. the name is looked up in **that org's index only** → `ErrNoCluster` (404) otherwise.
|
||||||
|
3. the credential is resolved from KMS over the call plane, carrying the caller, so KMS
|
||||||
|
re-applies its own `orgs/<org>/` rule.
|
||||||
|
4. `SafeRESTConfig` → typed + dynamic + metrics clients, cached per `org/name`.
|
||||||
|
|
||||||
|
`Bound.Scope(ns)` / `Bound.One(ns)` / `Bound.WholeCluster()` are the namespace gate, and
|
||||||
|
every namespaced op goes through one of them. A named namespace outside the bound is
|
||||||
|
REFUSED rather than silently narrowed — a caller that asked for the wrong thing must be
|
||||||
|
told, or it will believe an empty answer.
|
||||||
|
|
||||||
|
## What a "grant" is
|
||||||
|
|
||||||
|
An org may reach a cluster it **owns**. There is no grant table, and that is a decision
|
||||||
|
rather than an omission: cross-org credential sharing is refused by KMS by construction (a
|
||||||
|
call acting for B asking for `orgs/A/…` is forbidden), so a grant table would have been a
|
||||||
|
second authority that could only ever disagree with the first.
|
||||||
|
|
||||||
|
Granting org B access to a cluster A owns is therefore: seal a scoped ServiceAccount
|
||||||
|
kubeconfig **under B**, and register it in B's own scope with a `namespaces` bound. That is
|
||||||
|
one existing verb, no new concept, and it is honest — to reach a cluster you must hold a
|
||||||
|
credential for it, sealed in your own org's KMS.
|
||||||
|
|
||||||
|
The platform's own path for a tenant on the shared cluster is the same call, made by the
|
||||||
|
provisioning job with `zip.WithCaller` stating the tenant it acts for.
|
||||||
|
|
||||||
|
## The 36 ops
|
||||||
|
|
||||||
|
`plane/plane.go` is the whole contract. Groups: `clusters` (5), `nodes` (3), `namespaces`
|
||||||
|
(4), `pods` (2), `volumes` (4), `workloads` (8), `jobs` (3), `resources` (5), `access` (1),
|
||||||
|
`metrics` (1).
|
||||||
|
|
||||||
|
Each is registered TWICE from ONE handler: on the app (HTTP, which projects OpenAPI + an
|
||||||
|
MCP tool + a CLI command) and on `app.Peer()` (the fleet's 0600 socket, invoked by op
|
||||||
|
token through `zip.Ask`). The registrations are written out as direct `zip.Get` /
|
||||||
|
`zip.Post` calls with literal paths **deliberately** — zipdoc reads the AST for exactly
|
||||||
|
that shape, so a registration wrapped in a helper is a registration whose doc comment
|
||||||
|
reaches neither the OpenAPI description nor the MCP tool description.
|
||||||
|
|
||||||
|
Design notes worth keeping:
|
||||||
|
|
||||||
|
- **`resources.watch` is a bounded read, not a stream.** A stream cannot cross the call
|
||||||
|
plane as a value; rendering one (SSE, a websocket) is the edge's concern. It returns the
|
||||||
|
changes within `waitSeconds` and the cursor to resume from, and `expired` is the field
|
||||||
|
that matters — a too-old cursor rendered as an empty page is how a watch-backed board
|
||||||
|
silently stops updating.
|
||||||
|
- **`nodes.cordon` is one op over three calls** (patch `spec.unschedulable`, list the
|
||||||
|
node's pods, evict each). There is no generic node patch, because one would let a caller
|
||||||
|
write any field of any node's spec. The drain evicts rather than deletes, so a
|
||||||
|
PodDisruptionBudget refusal is the system working.
|
||||||
|
- **`volumes.expandClaim` takes `gi`, not a patch.** Growing the claim is the one way that
|
||||||
|
leaves claim, PV, device and filesystem agreeing. Nothing deletes a claim.
|
||||||
|
- **`namespaces.delete` is separately gated** and refused for a bounded registration: it
|
||||||
|
garbage-collects every object in the namespace, claims included.
|
||||||
|
- **`nodes.volumeStats` is its own op** so `nodes/proxy` can be dropped from every other
|
||||||
|
caller's grant. It is the only source of volume fill we have (no cloud provider exposes
|
||||||
|
it; metrics-server is absent from most of our clusters) and it reports `nodesRead` /
|
||||||
|
`nodesTotal`, because a claim with no row is UNMEASURED, which is a different fact from
|
||||||
|
empty.
|
||||||
|
- **`metrics.pods` reports `available`.** A cluster with no metrics-server rendered as
|
||||||
|
zeroes tells an operator a busy namespace is idle.
|
||||||
|
- **`clusters.status` probes the version endpoint**, not a namespace list: a registration
|
||||||
|
bounded to one namespace would fail a namespace list and look dead while being healthy.
|
||||||
|
|
||||||
|
## Secrets — the decision
|
||||||
|
|
||||||
|
There is **no secrets op**, and the absence is the answer to the question the measurement
|
||||||
|
report left open.
|
||||||
|
|
||||||
|
Three consumers wrote cluster Secrets (`platform/secrets.go`,
|
||||||
|
`provisioning/addon_inject.go`, `ai/cluster.ensureHfTokenSecret`) and one read one back.
|
||||||
|
An RPC that reads a tenant Secret is a credential-exfiltration endpoint however carefully
|
||||||
|
it is gated, and the standing rule is that credentials live in KMS.
|
||||||
|
|
||||||
|
The mechanism already exists in-cluster: **`KMSSecret`** (`secrets.lux.network/v1alpha1`)
|
||||||
|
names a KMS ref, and the kms-operator resolves it into a mounted Secret. It is in the
|
||||||
|
closed kind set, so `resources.apply` with `kind: KMSSecret` is the write-only projection
|
||||||
|
the report asked for — no new op, nothing returned, and the material never enters this
|
||||||
|
process. `platform/secrets.go`'s read-back must be re-sourced from KMS.
|
||||||
|
|
||||||
|
## Package weight, stated plainly
|
||||||
|
|
||||||
|
```
|
||||||
|
binary 630 packages zip 257 · k8s.io 276 · sigs.k8s.io 10 · own 4 · rest stdlib/shared
|
||||||
|
plane contract 1 package stdlib only
|
||||||
|
goja / esbuild 0 zip >= v1.18.7 makes the JS toolchain opt-in
|
||||||
|
OTLP 0 telemetry rides middleware.Telemetry, nil-safe
|
||||||
|
kustomize 0 rendering is a deploy concern; it never enters here
|
||||||
|
```
|
||||||
|
|
||||||
|
**630 is not ~260 and never could be.** client-go's typed clientset alone is ~279 packages,
|
||||||
|
and the typed client is required: `EvictV1` (a PDB-respecting drain), `GetLogs`, and the
|
||||||
|
kubelet `nodes/proxy` read have no dynamic-client equivalent. The measurement report
|
||||||
|
predicted 560–600 and said to say so up front rather than discover it at the end; the
|
||||||
|
measured answer is 630.
|
||||||
|
|
||||||
|
What this repo does control is that **nobody else pays it**. The same ~286 packages were
|
||||||
|
linked into four binaries; the consumer-facing cost is now one package.
|
||||||
|
|
||||||
|
## Operational notes
|
||||||
|
|
||||||
|
- **RBAC**: `rbac.yaml` is the union ceiling, gated against `ops.Kinds()` by the suite. On
|
||||||
|
a shared cluster do NOT bind it to a tenant — bind a namespaced Role and register the
|
||||||
|
cluster with a `namespaces` bound.
|
||||||
|
- **Migration from `apps/fleet`**: fleet sealed BYO kubeconfigs at
|
||||||
|
`<org>/fleet/clusters/<name>/kubeconfig` (no `orgs/` prefix, sharded by project). This
|
||||||
|
repo derives `orgs/<org>/k8s/clusters/<name>/kubeconfig` and does not read the old
|
||||||
|
shape — deriving the ref is what makes a row unable to name foreign material, so a
|
||||||
|
compatibility branch would give that property up. Migration is therefore a one-time
|
||||||
|
re-registration per BYO cluster (or a one-shot copy old-ref → new-ref inside KMS), and it
|
||||||
|
must happen in the change that removes fleet's registry.
|
||||||
|
- **Project sub-scope is gone on purpose.** fleet sharded by org+project with a
|
||||||
|
backward-compatibility seam that kept default-project keys byte-identical. This is a new
|
||||||
|
ref namespace with no existing key to stay identical with, and a cluster belongs to an
|
||||||
|
org, so there is one key shape and no seam.
|
||||||
|
- **`K8S_ALLOW_PRIVATE_HOSTS`** disables the non-routable-apiserver rejection. Tests only.
|
||||||
|
- **`plane.KMS`/`KMSGet`/`KMSPut`** are the only tokens here this repo does not own. They
|
||||||
|
are declared identically in cloud's own plane package; when HIP-0106 migration step 2
|
||||||
|
extracts `github.com/hanzoai/plane`, both collapse into that module and these three
|
||||||
|
declarations are deleted in the same change. One place to move.
|
||||||
|
|
||||||
|
## Open, and needing z's call
|
||||||
|
|
||||||
|
1. **DOKS provision/destroy stays with visor.** `POST`/`DELETE /v1/k8s/clusters` in visor
|
||||||
|
today provisions and destroys DigitalOcean clusters on the house account — a
|
||||||
|
cloud-provider spend action, and providers are visor's job. So `/v1/k8s/clusters` here
|
||||||
|
is the REGISTRY (list/get/register/deregister); visor keeps provider-side creation under
|
||||||
|
`/v1/compute/…` and calls `clusters.register` when a cluster comes up. This is the one
|
||||||
|
place "`/v1/k8s/clusters` survives and moves" and "cloud PROVIDERS are visor's job"
|
||||||
|
pull in opposite directions, and it is flagged rather than guessed.
|
||||||
|
2. **`/v1/clusters` removal is six routes, not a rename.** visor registers `GET`, `POST`
|
||||||
|
(BYO attach), `DELETE /:id` (BYO detach), `POST /:clusterId/pools`,
|
||||||
|
`POST /:clusterId/pools/:poolId/scale`, `DELETE /:clusterId/pools/:poolId`. The node-pool
|
||||||
|
trio is provider work and follows (1). `POST /v1/clusters` — the BYO kubeconfig attach —
|
||||||
|
is the single most important operation in the whole set and lands here as
|
||||||
|
`clusters.register`. Every caller (console, `hanzo clusters …`, SDKs, docs) moves in the
|
||||||
|
same change; forward-only, no alias.
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# One entry point. `make test` runs every gate HIP-0106 §8 requires, and hanzo.yml's
|
||||||
|
# test: block invokes exactly this target — a gate CI does not call is a comment.
|
||||||
|
GO ?= go
|
||||||
|
export GOWORK = off
|
||||||
|
export GOPRIVATE = github.com/hanzoai/*
|
||||||
|
|
||||||
|
.PHONY: all build generate test vet clean
|
||||||
|
|
||||||
|
all: build
|
||||||
|
|
||||||
|
build:
|
||||||
|
$(GO) build -trimpath -o k8s .
|
||||||
|
|
||||||
|
# The projections. Both are FILES written by the binary from its own live router, and
|
||||||
|
# both are committed — the suite regenerates and fails on any diff, so a committed
|
||||||
|
# artifact is never a golden nothing forces back to source.
|
||||||
|
generate: build
|
||||||
|
$(GO) generate ./...
|
||||||
|
./k8s declare k8s.plugin.json
|
||||||
|
./k8s openapi openapi.json
|
||||||
|
|
||||||
|
vet:
|
||||||
|
$(GO) vet ./...
|
||||||
|
|
||||||
|
test: vet
|
||||||
|
$(GO) test ./... -count=1
|
||||||
|
|
||||||
|
clean:
|
||||||
|
rm -f k8s
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
# hanzoai/k8s
|
||||||
|
|
||||||
|
The one Hanzo Kubernetes service. `/v1/k8s` is the whole API, and it is a **cluster
|
||||||
|
registry** plus a scoped manager over whichever cluster an org's resources live on.
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /v1/k8s/clusters every cluster the calling org may reach
|
||||||
|
POST /v1/k8s/clusters register one (kubeconfig -> validated -> sealed in KMS)
|
||||||
|
GET /v1/k8s/clusters/:cluster one registration
|
||||||
|
DELETE /v1/k8s/clusters/:cluster deregister, and destroy the sealed credential
|
||||||
|
GET /v1/k8s/clusters/:cluster/status
|
||||||
|
GET /v1/k8s/nodes?cluster=… and 30 more named, typed operations
|
||||||
|
```
|
||||||
|
|
||||||
|
## Multi-cluster is the shape, not a feature
|
||||||
|
|
||||||
|
lux runs `lux-k8s`. zoo runs `zoo-k8s`. Customers bring their own. So an org's resources
|
||||||
|
may live on a different cluster from Hanzo's, and "our cluster" is not a thing this
|
||||||
|
program can mean.
|
||||||
|
|
||||||
|
**Every operation names its cluster.** The name is resolved through the registry against
|
||||||
|
the tenant the call acts for, and `registry.Reach` is the only constructor of a Kubernetes
|
||||||
|
client in the module — so no function can talk to a cluster it did not name, and there is
|
||||||
|
no ambient in-cluster fallback to inherit. `hanzo-k8s`, `lux-k8s` and a customer's own
|
||||||
|
cluster are three rows and one code path.
|
||||||
|
|
||||||
|
## The credential lives in KMS and is resolved at use
|
||||||
|
|
||||||
|
A registration validates its kubeconfig ([`registry.SafeRESTConfig`](registry/config.go) —
|
||||||
|
no exec credential plugins, no non-routable apiserver), uses it once to prove the cluster
|
||||||
|
answers, seals it in KMS under `orgs/<org>/k8s/clusters/<name>/kubeconfig`, and drops it.
|
||||||
|
Nothing is written to disk, nothing is put in an environment variable, nothing is logged,
|
||||||
|
and no operation returns it.
|
||||||
|
|
||||||
|
The ref is **derived** from the owning org and the cluster name rather than stored, which
|
||||||
|
is what lets the KMS service apply its own tenant rule to every resolution — a second
|
||||||
|
refusal, in a second process, of a cross-org read.
|
||||||
|
|
||||||
|
## Cross-org access is impossible, and there is a test that proves it
|
||||||
|
|
||||||
|
- The registry index is **per-org**. A cluster another org registered answers **404**, not
|
||||||
|
403: a distinguishable refusal would be an oracle for enumerating other tenants' cluster
|
||||||
|
names.
|
||||||
|
- Two orgs may register the same NAME. They resolve to different endpoints with different
|
||||||
|
credentials.
|
||||||
|
- A registration on a shared cluster carries a **namespace bound**. A namespace outside it
|
||||||
|
is refused before the apiserver is dialed, and refused again by the credential's RBAC.
|
||||||
|
- Cluster-scoped reads (nodes, PersistentVolumes, the kubelet sweep) are refused outright
|
||||||
|
for a bounded registration.
|
||||||
|
|
||||||
|
[`registry/authz_test.go`](registry/authz_test.go) is written from the attacker's side, and
|
||||||
|
each guard was verified by REMOVING it and observing the red:
|
||||||
|
|
||||||
|
| Guard removed | Tests that fail |
|
||||||
|
|---|---|
|
||||||
|
| the org segment in `indexRef` | `TestForeignClusterNameIsNotFound`, `TestSameNameInTwoOrgsAreTwoClusters`, `TestRefsAreOrgPrefixed`, `TestDeregisterCannotReachAcrossOrgs` |
|
||||||
|
| the bound check in `Bound.Scope` | `TestForeignNamespaceIsRefused`, `TestBoundedListDoesNotWiden` |
|
||||||
|
| the validated-user test in `Tenant` | `TestOrgHeaderWithoutPrincipalIsRefused` |
|
||||||
|
|
||||||
|
[`ops/ops_test.go`](ops/ops_test.go) proves the same refusals arrive on the real wire, with
|
||||||
|
the headers a gateway attaches.
|
||||||
|
|
||||||
|
## No generic passthrough
|
||||||
|
|
||||||
|
Every operation is named and typed, including the custom-resource quartet, whose kinds come
|
||||||
|
from a **closed table** in [`ops/resources.go`](ops/resources.go). The function this
|
||||||
|
replaces decoded arbitrary YAML, resolved its kind through a discovery `RESTMapper`, and
|
||||||
|
create-or-updated any kind the cluster served — with a kubeconfig read out of a database
|
||||||
|
row. With that, no ClusterRole smaller than "everything" could be written.
|
||||||
|
|
||||||
|
With this set, the union of privilege is a finite list, and it is in
|
||||||
|
[`rbac.yaml`](rbac.yaml) — the first written statement of it anywhere in the fleet.
|
||||||
|
`""/secrets` is deliberately absent: material reaches a namespace as a `KMSSecret` the
|
||||||
|
kms-operator resolves, and nothing here reads a Secret back.
|
||||||
|
|
||||||
|
## Consumers ask; they do not link
|
||||||
|
|
||||||
|
[`github.com/hanzoai/k8s/plane`](plane/plane.go) is the call contract: op tokens and their
|
||||||
|
In/Out types, **one package, stdlib only**. A consumer links it and calls
|
||||||
|
`zip.Ask[plane.PodsIn, plane.Pods](ctx, "k8s", plane.PodsList, in)`. It never links
|
||||||
|
client-go, and it never holds a cluster credential.
|
||||||
|
|
||||||
|
```
|
||||||
|
this binary 630 packages (zip 257 + the k8s.io/sigs.k8s.io family 286)
|
||||||
|
plane contract 1 package
|
||||||
|
```
|
||||||
|
|
||||||
|
630 is above the ~260 a conforming plugin measures, and the reason is not this repo's
|
||||||
|
code: client-go's typed clientset alone is ~279 packages, and the typed client is required
|
||||||
|
— `EvictV1` for a PDB-respecting drain, `GetLogs`, and the kubelet `nodes/proxy` read have
|
||||||
|
no dynamic-client equivalent. What this repo does control is that **nobody else pays it**:
|
||||||
|
the same 286 packages used to be linked into four binaries, and the consumer-facing cost is
|
||||||
|
now one package.
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
```
|
||||||
|
make build # the binary
|
||||||
|
make generate # zipdoc + k8s.plugin.json + openapi.json, all from the live router
|
||||||
|
make test # go vet + every gate HIP-0106 §8 requires
|
||||||
|
```
|
||||||
|
|
||||||
|
Conforms to [HIP-0106](../hips/HIPs/hip-0106-hanzo-plugin-contract.md) and HIP-0119.
|
||||||
|
Deep documentation for agents: [`LLM.md`](LLM.md).
|
||||||
+627
@@ -0,0 +1,627 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"go/ast"
|
||||||
|
"go/parser"
|
||||||
|
"go/token"
|
||||||
|
"io/fs"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/ops"
|
||||||
|
"github.com/hanzoai/k8s/registry"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The gates HIP-0106 §8 requires, each named for the failure it catches. Two properties
|
||||||
|
// hold throughout, because they are the two ways a green tick lies:
|
||||||
|
//
|
||||||
|
// - NON-VACUITY. A gate that examined nothing FAILS and says how much it examined.
|
||||||
|
// Every route to zero examinations is a defect elsewhere that would otherwise arrive
|
||||||
|
// as a pass.
|
||||||
|
// - A SOURCE ON ONE SIDE. Never two derived artifacts compared with each other. Here
|
||||||
|
// the live router is the source: the declaration and the OpenAPI subset are both
|
||||||
|
// projected from it inside the test, and the committed files are compared against
|
||||||
|
// that projection rather than against one another.
|
||||||
|
|
||||||
|
// app builds the composition this binary serves, exactly as run() does. Nothing is
|
||||||
|
// stubbed: the registry is the real one over a store that is never reached, because a
|
||||||
|
// projection is a function of the code and must not need a peer.
|
||||||
|
func app(t *testing.T) *zip.App {
|
||||||
|
t.Helper()
|
||||||
|
a := zip.New(zip.Config{AppName: name})
|
||||||
|
ops.Ops{Reg: registry.New(registry.Sealed{})}.Mount(a)
|
||||||
|
if err := a.Build(); err != nil {
|
||||||
|
t.Fatalf("Build: %v", err)
|
||||||
|
}
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
|
||||||
|
// sh executes a tool and returns its combined output, failing the test on error.
|
||||||
|
func sh(t *testing.T, args ...string) string {
|
||||||
|
t.Helper()
|
||||||
|
cmd := exec.Command(args[0], args[1:]...)
|
||||||
|
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=-mod=mod")
|
||||||
|
out, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: %v\n%s", strings.Join(args, " "), err, out)
|
||||||
|
}
|
||||||
|
return string(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
// §8.1 — the plugin is a LEAF.
|
||||||
|
func TestPluginIsALeaf(t *testing.T) {
|
||||||
|
// A grep over source is NOT this gate: the dangerous break is the one that still
|
||||||
|
// compiles, so the check is over the import graph a build actually produces.
|
||||||
|
out := sh(t, "go", "list", "-deps", "./...")
|
||||||
|
deps := strings.Fields(out)
|
||||||
|
if len(deps) < 100 {
|
||||||
|
t.Fatalf("this gate proved nothing: the dependency graph came back as %d packages", len(deps))
|
||||||
|
}
|
||||||
|
for _, d := range deps {
|
||||||
|
if strings.HasPrefix(d, "github.com/hanzoai/cloud") || strings.HasPrefix(d, "github.com/hanzoai/base") {
|
||||||
|
t.Errorf("this plugin links the host package %s — that is the 316-package coupling the contract removes", d)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.Logf("%d packages, none of them a host's", len(deps))
|
||||||
|
}
|
||||||
|
|
||||||
|
// §1.2 — the measured floor, reported rather than asserted at a number.
|
||||||
|
func TestPackageWeightIsReported(t *testing.T) {
|
||||||
|
binary := strings.Fields(sh(t, "go", "list", "-deps", "."))
|
||||||
|
leaf := strings.Fields(sh(t, "go", "list", "-deps", "./plane"))
|
||||||
|
var k8sFamily int
|
||||||
|
for _, d := range binary {
|
||||||
|
if strings.HasPrefix(d, "k8s.io/") || strings.HasPrefix(d, "sigs.k8s.io/") {
|
||||||
|
k8sFamily++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The plane leaf is the number that matters to a CONSUMER: it is what an app links
|
||||||
|
// to ask this service a question, and it must stay at one package. A consumer that
|
||||||
|
// linked client-go to make an RPC would have gained nothing from the split.
|
||||||
|
if len(leaf) != 1 {
|
||||||
|
t.Fatalf("the plane contract is %d packages; a consumer must be able to link it without linking a Kubernetes client:\n%s",
|
||||||
|
len(leaf), strings.Join(leaf, "\n"))
|
||||||
|
}
|
||||||
|
// kustomize is a RENDERING concern and belongs to whoever renders. Worth 74-84
|
||||||
|
// packages on its own, and it has no business in a cluster client.
|
||||||
|
for _, d := range binary {
|
||||||
|
if strings.Contains(d, "kustomize") {
|
||||||
|
t.Errorf("%s: rendering belongs to the deploy side, not to the cluster client", d)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.Logf("binary %d packages (k8s family %d), plane contract %d package", len(binary), k8sFamily, len(leaf))
|
||||||
|
}
|
||||||
|
|
||||||
|
// §8.3 — the committed declaration is current, and the OpenAPI subset is a subset of it.
|
||||||
|
func TestDeclarationIsCurrent(t *testing.T) {
|
||||||
|
live := app(t).Declaration()
|
||||||
|
committed := readJSON[zip.Declaration](t, "k8s.plugin.json")
|
||||||
|
if diff := jsonDiff(t, live, committed); diff != "" {
|
||||||
|
t.Fatalf("k8s.plugin.json is not what the router serves; run `make generate`:\n%s", diff)
|
||||||
|
}
|
||||||
|
if len(live.Routes) == 0 || len(live.Ops) == 0 {
|
||||||
|
t.Fatal("this gate proved nothing: the declaration is empty")
|
||||||
|
}
|
||||||
|
|
||||||
|
// openapi ⊆ declaration. The relation is one-way on purpose: a served-but-unpublished
|
||||||
|
// route must still be DECLARED or a host will not route it, which is the outage class
|
||||||
|
// where every product beacon in the fleet answered 405 for want of a table row.
|
||||||
|
spec := readJSON[struct {
|
||||||
|
Paths map[string]map[string]any `json:"paths"`
|
||||||
|
}](t, "openapi.json")
|
||||||
|
declared := map[string]bool{}
|
||||||
|
for _, r := range live.Routes {
|
||||||
|
declared[openAPIPath(r.Pattern)] = true
|
||||||
|
}
|
||||||
|
for p := range spec.Paths {
|
||||||
|
if !declared[p] {
|
||||||
|
t.Errorf("the OpenAPI document publishes %s, which the declaration does not carry — a host will not route it", p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.Logf("%d routes, %d ops, %d published paths", len(live.Routes), len(live.Ops), len(spec.Paths))
|
||||||
|
}
|
||||||
|
|
||||||
|
// §8.4 — the committed OpenAPI subset is current.
|
||||||
|
func TestOpenAPIIsCurrent(t *testing.T) {
|
||||||
|
live := app(t).OpenAPISpec()
|
||||||
|
committed := readJSON[map[string]any](t, "openapi.json")
|
||||||
|
if diff := jsonDiff(t, live, committed); diff != "" {
|
||||||
|
t.Fatalf("openapi.json is not what the router serves; run `make generate`:\n%s", diff)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// §8.5 — zipdoc is wired, current, and every op actually carries prose.
|
||||||
|
func TestProseReachesBothProjections(t *testing.T) {
|
||||||
|
// The directive is the ONLY way a doc comment reaches a document: Go drops comments
|
||||||
|
// at compile time and reflection sees types, never prose. A package without it can
|
||||||
|
// never get a description into the spec or into an MCP tool, however correct
|
||||||
|
// everything else is.
|
||||||
|
for _, dir := range typedOpDirs(t) {
|
||||||
|
src, err := os.ReadFile(filepath.Join(dir, "zipdoc_gen.go"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s registers typed ops but has no committed zipdoc_gen.go: %v", dir, err)
|
||||||
|
}
|
||||||
|
if len(src) == 0 {
|
||||||
|
t.Fatalf("%s/zipdoc_gen.go is empty", dir)
|
||||||
|
}
|
||||||
|
if !strings.Contains(readFile(t, filepath.Join(dir, "ops.go")), "//go:generate go run github.com/zap-proto/zip/cmd/zipdoc") {
|
||||||
|
t.Fatalf("%s is missing the zipdoc directive, so its prose reaches neither projection", dir)
|
||||||
|
}
|
||||||
|
sh(t, "go", "run", "github.com/zap-proto/zip/cmd/zipdoc", "-check", "./"+dir)
|
||||||
|
}
|
||||||
|
|
||||||
|
// And the prose is actually THERE, in both projections, for every op. A directive
|
||||||
|
// that runs over handlers with no comments is a gate that proves nothing.
|
||||||
|
a := app(t)
|
||||||
|
spec, err := json.Marshal(a.OpenAPISpec())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var doc struct {
|
||||||
|
Paths map[string]map[string]struct {
|
||||||
|
OperationID string `json:"operationId"`
|
||||||
|
Description string `json:"description"`
|
||||||
|
} `json:"paths"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(spec, &doc); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
described := 0
|
||||||
|
for p, methods := range doc.Paths {
|
||||||
|
for m, op := range methods {
|
||||||
|
if op.OperationID == "" {
|
||||||
|
t.Errorf("%s %s has no operationId, so it is invisible to the plane, the MCP tool list and the CLI", m, p)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(op.Description) == "" {
|
||||||
|
t.Errorf("%s %s has no description: an agent choosing whether to call it would read nothing", m, p)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
described++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tools := a.MCPTools()
|
||||||
|
for _, tool := range tools {
|
||||||
|
if d, _ := tool["description"].(string); strings.TrimSpace(d) == "" {
|
||||||
|
t.Errorf("MCP tool %v has no description", tool["name"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if described == 0 || len(tools) == 0 {
|
||||||
|
t.Fatal("this gate proved nothing: no described operations and no tools")
|
||||||
|
}
|
||||||
|
if len(tools) != described {
|
||||||
|
t.Errorf("%d described operations but %d MCP tools — one projection is losing ops", described, len(tools))
|
||||||
|
}
|
||||||
|
t.Logf("%d operations carry prose in both the OpenAPI document and the MCP tool list", described)
|
||||||
|
}
|
||||||
|
|
||||||
|
// §3.4 — the shape a host will accept.
|
||||||
|
func TestEveryRouteIsRoutable(t *testing.T) {
|
||||||
|
live := app(t).Declaration()
|
||||||
|
if live.Name != name {
|
||||||
|
t.Fatalf("the declaration names %q but the binary is %q — one name, no mapping table", live.Name, name)
|
||||||
|
}
|
||||||
|
if live.Eager {
|
||||||
|
t.Error("this app is request-driven: it owns no listener, consumer or background loop, so a host must be free to start it lazily")
|
||||||
|
}
|
||||||
|
for _, r := range live.Routes {
|
||||||
|
switch {
|
||||||
|
case !strings.HasPrefix(r.Pattern, "/v1/k8s"):
|
||||||
|
t.Errorf("%s %s is outside this app's own prefix, so it would claim a path another plugin owns", r.Method, r.Pattern)
|
||||||
|
case strings.Contains(r.Pattern, ".."):
|
||||||
|
t.Errorf("%s %s contains a parent reference", r.Method, r.Pattern)
|
||||||
|
case strings.Contains(r.Pattern, "*"):
|
||||||
|
t.Errorf("%s %s is a catch-all: it would claim paths other plugins own and paths that do not exist yet", r.Method, r.Pattern)
|
||||||
|
case strings.Contains(r.Pattern, "/api/"):
|
||||||
|
t.Errorf("%s %s carries an /api/ prefix", r.Method, r.Pattern)
|
||||||
|
case strings.Contains(r.Pattern, "/v2"):
|
||||||
|
t.Errorf("%s %s is a v2", r.Method, r.Pattern)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(live.Routes) < 30 {
|
||||||
|
t.Fatalf("this gate examined %d routes; the measured demand is 36 ops", len(live.Routes))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The op set and the plane contract are one set, read from two sources.
|
||||||
|
func TestEveryPlaneConstantIsAnOpAndViceVersa(t *testing.T) {
|
||||||
|
// One side is the LIVE ROUTER, the other is the plane package's own const block. A
|
||||||
|
// constant nobody registered is an op a consumer will call and get nothing from; an
|
||||||
|
// op whose token is not declared is a name a consumer cannot spell without copying a
|
||||||
|
// string. Both fail here rather than at the first call.
|
||||||
|
registered := app(t).Declaration().Ops
|
||||||
|
declared := planeConstants(t)
|
||||||
|
if len(registered) == 0 || len(declared) == 0 {
|
||||||
|
t.Fatal("this gate proved nothing: one of the two sets is empty")
|
||||||
|
}
|
||||||
|
in := func(set []string, v string) bool {
|
||||||
|
for _, s := range set {
|
||||||
|
if s == v {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, tok := range declared {
|
||||||
|
if !in(registered, tok) {
|
||||||
|
t.Errorf("plane declares the op token %q but nothing registers it", tok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, tok := range registered {
|
||||||
|
if !in(declared, tok) {
|
||||||
|
t.Errorf("the router serves the op token %q but plane does not declare it, so a consumer cannot spell it", tok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.Logf("%d op tokens, declared and served", len(declared))
|
||||||
|
}
|
||||||
|
|
||||||
|
// §8.7 — tenancy, read off the published document so every input is covered.
|
||||||
|
func TestNoOperationTakesATenant(t *testing.T) {
|
||||||
|
spec, err := json.Marshal(app(t).OpenAPISpec())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Walk the WHOLE document rather than a list of types: a parameter name, a schema
|
||||||
|
// property, a $ref'd component — any of them naming a tenant is the defect, and
|
||||||
|
// walking everything means a new one cannot slip in beside a check that only looked
|
||||||
|
// at inputs it knew about.
|
||||||
|
var doc any
|
||||||
|
if err := json.Unmarshal(spec, &doc); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
banned := map[string]bool{"org": true, "orgId": true, "organization": true, "tenant": true, "tenantId": true}
|
||||||
|
seen := 0
|
||||||
|
var walk func(path string, v any)
|
||||||
|
walk = func(path string, v any) {
|
||||||
|
switch t2 := v.(type) {
|
||||||
|
case map[string]any:
|
||||||
|
for k, sub := range t2 {
|
||||||
|
seen++
|
||||||
|
// An org on a RESPONSE schema is a property of the thing described. The
|
||||||
|
// path tells them apart: only request bodies and parameters are inputs.
|
||||||
|
if banned[k] && (strings.Contains(path, "requestBody") || strings.Contains(path, "parameters")) {
|
||||||
|
t.Errorf("%s names %q: an org in the argument is an org the caller chose", path, k)
|
||||||
|
}
|
||||||
|
walk(path+"."+k, sub)
|
||||||
|
}
|
||||||
|
case []any:
|
||||||
|
for i, sub := range t2 {
|
||||||
|
if m, ok := sub.(map[string]any); ok {
|
||||||
|
if n, isStr := m["name"].(string); isStr && banned[n] && strings.Contains(path, "parameters") {
|
||||||
|
t.Errorf("%s[%d] is a %q parameter: a caller that can name the org can read another tenant", path, i, n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
walk(path, sub)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
walk("", doc)
|
||||||
|
if seen < 200 {
|
||||||
|
t.Fatalf("this gate walked only %d nodes of the document; it did not read the spec it is protecting", seen)
|
||||||
|
}
|
||||||
|
t.Logf("walked %d document nodes", seen)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every op that reaches a cluster NAMES it. This is the multi-cluster invariant, gated.
|
||||||
|
func TestEveryClusterOpNamesItsCluster(t *testing.T) {
|
||||||
|
spec, err := json.Marshal(app(t).OpenAPISpec())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var doc struct {
|
||||||
|
Paths map[string]map[string]struct {
|
||||||
|
OperationID string `json:"operationId"`
|
||||||
|
Parameters []struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
} `json:"parameters"`
|
||||||
|
RequestBody struct {
|
||||||
|
Content map[string]struct {
|
||||||
|
Schema struct {
|
||||||
|
Properties map[string]any `json:"properties"`
|
||||||
|
Ref string `json:"$ref"`
|
||||||
|
} `json:"schema"`
|
||||||
|
} `json:"content"`
|
||||||
|
} `json:"requestBody"`
|
||||||
|
} `json:"paths"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(spec, &doc); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// The two ops that legitimately do not name a cluster: listing the registry, and
|
||||||
|
// registering a cluster (which is where a name comes to exist, so it takes `name`).
|
||||||
|
exempt := map[string]bool{"k8s_clusters_list": true, "k8s_clusters_register": true}
|
||||||
|
checked := 0
|
||||||
|
for p, methods := range doc.Paths {
|
||||||
|
for m, op := range methods {
|
||||||
|
if exempt[op.OperationID] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
checked++
|
||||||
|
named := false
|
||||||
|
for _, param := range op.Parameters {
|
||||||
|
if param.Name == "cluster" {
|
||||||
|
named = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, c := range op.RequestBody.Content {
|
||||||
|
if _, ok := c.Schema.Properties["cluster"]; ok {
|
||||||
|
named = true
|
||||||
|
}
|
||||||
|
// A $ref'd body carries the field through its component schema, which the
|
||||||
|
// plane package's own gate covers; a ref is therefore accepted here.
|
||||||
|
if c.Schema.Ref != "" {
|
||||||
|
named = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !named {
|
||||||
|
t.Errorf("%s %s (%s) does not name a cluster — there is no ambient cluster, so this op would read whichever apiserver the process was started next to",
|
||||||
|
m, p, op.OperationID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if checked < 30 {
|
||||||
|
t.Fatalf("this gate examined %d operations; the measured demand is 36", checked)
|
||||||
|
}
|
||||||
|
t.Logf("%d operations, every one naming its cluster", checked)
|
||||||
|
}
|
||||||
|
|
||||||
|
// §8.2 — the binary builds and mounts its whole surface alone.
|
||||||
|
func TestBuildsAndMountsAlone(t *testing.T) {
|
||||||
|
bin := filepath.Join(t.TempDir(), name)
|
||||||
|
sh(t, "go", "build", "-o", bin, ".")
|
||||||
|
// `declare` exercises construct → register → project without a host, a store or a
|
||||||
|
// peer, which is exactly what "mounts alone" means for a request-driven app: if any
|
||||||
|
// of that needed something else to have initialised it first, this exits non-zero.
|
||||||
|
out := filepath.Join(t.TempDir(), "d.json")
|
||||||
|
sh(t, bin, "declare", out)
|
||||||
|
d := readJSON[zip.Declaration](t, out)
|
||||||
|
if len(d.Routes) == 0 {
|
||||||
|
t.Fatal("the binary mounted no routes when run with no host")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// §8.8 — the suite is not empty, and every package contributes.
|
||||||
|
func TestEveryPackageHasTests(t *testing.T) {
|
||||||
|
pkgs := strings.Fields(sh(t, "go", "list", "./..."))
|
||||||
|
if len(pkgs) < 4 {
|
||||||
|
t.Fatalf("this gate proved nothing: %d packages", len(pkgs))
|
||||||
|
}
|
||||||
|
for _, p := range pkgs {
|
||||||
|
dir := "."
|
||||||
|
if rel := strings.TrimPrefix(p, "github.com/hanzoai/k8s"); rel != "" {
|
||||||
|
dir = "." + rel
|
||||||
|
}
|
||||||
|
matches, err := filepath.Glob(filepath.Join(dir, "*_test.go"))
|
||||||
|
if err != nil || len(matches) == 0 {
|
||||||
|
t.Errorf("%s has no test file: a package with no executed test is a package the suite does not cover", p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// §8.9 — CI invokes the gate. Without this every gate above protects nobody.
|
||||||
|
func TestCIInvokesMakeTest(t *testing.T) {
|
||||||
|
y := readFile(t, "hanzo.yml")
|
||||||
|
if !strings.Contains(y, "make test") {
|
||||||
|
t.Fatal("hanzo.yml's test: block does not run `make test`, so every gate in this file is a comment")
|
||||||
|
}
|
||||||
|
w := readFile(t, ".github/workflows/cicd.yml")
|
||||||
|
if !strings.Contains(w, "hanzoai/ci/.github/workflows/build.yml@v1") {
|
||||||
|
t.Fatal("the workflow does not import the canonical hanzoai/ci reusable, so this repo has its own build logic")
|
||||||
|
}
|
||||||
|
mk := readFile(t, "Makefile")
|
||||||
|
for _, target := range []string{"generate:", "test:"} {
|
||||||
|
if !strings.Contains(mk, target) {
|
||||||
|
t.Fatalf("the Makefile has no %s target", target)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The regeneration gates above are worth nothing if `make test` does not run them.
|
||||||
|
if !strings.Contains(mk, "test ./...") {
|
||||||
|
t.Fatal("`make test` does not run the Go suite")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- helpers ---------------------------------------------------------------
|
||||||
|
|
||||||
|
func readFile(t *testing.T, path string) string {
|
||||||
|
t.Helper()
|
||||||
|
raw, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: %v", path, err)
|
||||||
|
}
|
||||||
|
return string(raw)
|
||||||
|
}
|
||||||
|
|
||||||
|
func readJSON[T any](t *testing.T, path string) T {
|
||||||
|
t.Helper()
|
||||||
|
var out T
|
||||||
|
if err := json.Unmarshal([]byte(readFile(t, path)), &out); err != nil {
|
||||||
|
t.Fatalf("%s: %v", path, err)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// jsonDiff compares two values by their CANONICAL JSON.
|
||||||
|
//
|
||||||
|
// Both sides go through a marshal → unmarshal → marshal round trip, which is not
|
||||||
|
// ceremony: the live spec carries authored examples as json.RawMessage, whose field
|
||||||
|
// order is preserved verbatim, while the committed file re-parses into maps, whose keys
|
||||||
|
// marshal sorted. Comparing one against the other directly reports a diff on every
|
||||||
|
// example in the document and would have made this gate permanently red for no reason.
|
||||||
|
func jsonDiff(t *testing.T, live, committed any) string {
|
||||||
|
t.Helper()
|
||||||
|
a, err := canonical(live)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
b, err := canonical(committed)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if string(a) == string(b) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
al, bl := strings.Split(string(a), "\n"), strings.Split(string(b), "\n")
|
||||||
|
var out []string
|
||||||
|
for i := 0; i < len(al) || i < len(bl); i++ {
|
||||||
|
var x, y string
|
||||||
|
if i < len(al) {
|
||||||
|
x = al[i]
|
||||||
|
}
|
||||||
|
if i < len(bl) {
|
||||||
|
y = bl[i]
|
||||||
|
}
|
||||||
|
if x != y {
|
||||||
|
out = append(out, "live: "+x, "committed: "+y)
|
||||||
|
if len(out) >= 12 {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return strings.Join(out, "\n")
|
||||||
|
}
|
||||||
|
|
||||||
|
// canonical renders a value as JSON with every map key sorted and every nested document
|
||||||
|
// re-parsed, so two spellings of one document compare equal.
|
||||||
|
func canonical(v any) ([]byte, error) {
|
||||||
|
raw, err := json.Marshal(v)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var norm any
|
||||||
|
if err := json.Unmarshal(raw, &norm); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return json.MarshalIndent(norm, "", " ")
|
||||||
|
}
|
||||||
|
|
||||||
|
// openAPIPath rewrites a router pattern into the document's spelling: ":id" is the
|
||||||
|
// router's, "{id}" is OpenAPI's, and the two must be compared in one vocabulary.
|
||||||
|
func openAPIPath(pattern string) string {
|
||||||
|
parts := strings.Split(pattern, "/")
|
||||||
|
for i, p := range parts {
|
||||||
|
if strings.HasPrefix(p, ":") {
|
||||||
|
parts[i] = "{" + p[1:] + "}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return strings.Join(parts, "/")
|
||||||
|
}
|
||||||
|
|
||||||
|
// typedOpDirs returns every directory in this module that registers a typed op, read
|
||||||
|
// from the AST rather than from a list, so a new package is covered by §8.5 the moment
|
||||||
|
// it registers its first op.
|
||||||
|
func typedOpDirs(t *testing.T) []string {
|
||||||
|
t.Helper()
|
||||||
|
verbs := map[string]bool{"Get": true, "Post": true, "Put": true, "Patch": true, "Delete": true}
|
||||||
|
found := map[string]bool{}
|
||||||
|
err := filepath.WalkDir(".", func(path string, d fs.DirEntry, err error) error {
|
||||||
|
if err != nil || d.IsDir() || !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
f, perr := parser.ParseFile(token.NewFileSet(), path, nil, 0)
|
||||||
|
if perr != nil {
|
||||||
|
return perr
|
||||||
|
}
|
||||||
|
ast.Inspect(f, func(n ast.Node) bool {
|
||||||
|
call, ok := n.(*ast.CallExpr)
|
||||||
|
if !ok {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
sel, ok := ast.Unparen(call.Fun).(*ast.SelectorExpr)
|
||||||
|
if !ok {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
pkg, ok := sel.X.(*ast.Ident)
|
||||||
|
if ok && pkg.Name == "zip" && verbs[sel.Sel.Name] && len(call.Args) >= 3 {
|
||||||
|
found[filepath.Dir(path)] = true
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
})
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("walk: %v", err)
|
||||||
|
}
|
||||||
|
if len(found) == 0 {
|
||||||
|
t.Fatal("this gate proved nothing: no package registers a typed op")
|
||||||
|
}
|
||||||
|
out := make([]string, 0, len(found))
|
||||||
|
for d := range found {
|
||||||
|
out = append(out, d)
|
||||||
|
}
|
||||||
|
sort.Strings(out)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// planeConstants reads the op tokens out of the plane package's own const declarations,
|
||||||
|
// so the comparison against the router is source against source.
|
||||||
|
func planeConstants(t *testing.T) []string {
|
||||||
|
t.Helper()
|
||||||
|
f, err := parser.ParseFile(token.NewFileSet(), "plane/plane.go", nil, 0)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parse plane: %v", err)
|
||||||
|
}
|
||||||
|
var out []string
|
||||||
|
ast.Inspect(f, func(n ast.Node) bool {
|
||||||
|
vs, ok := n.(*ast.ValueSpec)
|
||||||
|
if !ok || len(vs.Values) != 1 {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
lit, ok := vs.Values[0].(*ast.BasicLit)
|
||||||
|
if !ok || lit.Kind != token.STRING {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
v := strings.Trim(lit.Value, `"`)
|
||||||
|
if strings.HasPrefix(v, "k8s_") {
|
||||||
|
out = append(out, v)
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
})
|
||||||
|
sort.Strings(out)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// The RBAC ceiling and the code are one set, read from two sources.
|
||||||
|
func TestRBACCoversExactlyTheClosedKindSet(t *testing.T) {
|
||||||
|
// A kind added to the code with no rule here 403s in production; a rule here with no
|
||||||
|
// kind in the code is privilege nothing uses. Both are failures, and neither is
|
||||||
|
// visible without this comparison — which is why the grant lives in the repo that
|
||||||
|
// performs the verbs rather than in a chart nobody diffs against the code.
|
||||||
|
rbac := readFile(t, "rbac.yaml")
|
||||||
|
kinds := ops.Kinds()
|
||||||
|
if len(kinds) == 0 {
|
||||||
|
t.Fatal("this gate proved nothing: the kind set is empty")
|
||||||
|
}
|
||||||
|
for _, kind := range kinds {
|
||||||
|
// The rule names the plural RESOURCE, so the kind's own lower-cased plural is
|
||||||
|
// what has to appear. The two spellings differ only by pluralisation, and every
|
||||||
|
// kind in this app's table pluralises by adding "s".
|
||||||
|
resource := strings.ToLower(kind) + "s"
|
||||||
|
if !strings.Contains(rbac, resource) {
|
||||||
|
t.Errorf("rbac.yaml grants nothing for %s (%s): the op would 403 in production", kind, resource)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The absence of a secrets grant is a design decision, so it is asserted rather than
|
||||||
|
// left to be quietly re-added by someone who needed one read.
|
||||||
|
// The comparison is on EXACT resource tokens, not on substrings: "kmssecrets" is the
|
||||||
|
// CR that exists precisely so no core Secret is ever read, and a substring check
|
||||||
|
// would flag the solution as the problem.
|
||||||
|
for _, line := range strings.Split(rbac, "\n") {
|
||||||
|
trimmed := strings.TrimSpace(line)
|
||||||
|
if !strings.HasPrefix(trimmed, "resources:") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
list := strings.Trim(strings.TrimPrefix(trimmed, "resources:"), " []")
|
||||||
|
for _, item := range strings.Split(list, ",") {
|
||||||
|
if strings.TrimSpace(item) == "secrets" {
|
||||||
|
t.Errorf("rbac.yaml grants core secrets (%q) — material reaches a namespace as a KMSSecret, and nothing here reads a Secret back", trimmed)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.Logf("%d custom-resource kinds, each with a rule", len(kinds))
|
||||||
|
}
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
module github.com/hanzoai/k8s
|
||||||
|
|
||||||
|
go 1.26.5
|
||||||
|
|
||||||
|
require (
|
||||||
|
github.com/zap-proto/zip v1.20.0
|
||||||
|
k8s.io/api v0.35.3
|
||||||
|
k8s.io/apimachinery v0.35.3
|
||||||
|
k8s.io/client-go v0.35.3
|
||||||
|
k8s.io/metrics v0.35.3
|
||||||
|
)
|
||||||
|
|
||||||
|
require (
|
||||||
|
github.com/andybalholm/brotli v1.2.1 // indirect
|
||||||
|
github.com/davecgh/go-spew v1.1.1 // indirect
|
||||||
|
github.com/emicklei/go-restful/v3 v3.12.2 // indirect
|
||||||
|
github.com/fxamacker/cbor/v2 v2.9.1 // indirect
|
||||||
|
github.com/go-logr/logr v1.4.3 // indirect
|
||||||
|
github.com/go-openapi/jsonpointer v0.21.0 // indirect
|
||||||
|
github.com/go-openapi/jsonreference v0.20.2 // indirect
|
||||||
|
github.com/go-openapi/swag v0.23.0 // indirect
|
||||||
|
github.com/gofiber/schema v1.7.1 // indirect
|
||||||
|
github.com/gofiber/utils/v2 v2.0.4 // indirect
|
||||||
|
github.com/google/gnostic-models v0.7.0 // indirect
|
||||||
|
github.com/google/uuid v1.6.0 // indirect
|
||||||
|
github.com/josharian/intern v1.0.0 // indirect
|
||||||
|
github.com/json-iterator/go v1.1.12 // indirect
|
||||||
|
github.com/klauspost/compress v1.18.5 // indirect
|
||||||
|
github.com/luxfi/log v1.4.3 // indirect
|
||||||
|
github.com/mailru/easyjson v0.7.7 // indirect
|
||||||
|
github.com/mattn/go-colorable v0.1.14 // indirect
|
||||||
|
github.com/mattn/go-isatty v0.0.21 // indirect
|
||||||
|
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
|
||||||
|
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
|
||||||
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
||||||
|
github.com/philhofer/fwd v1.2.0 // indirect
|
||||||
|
github.com/spf13/pflag v1.0.9 // indirect
|
||||||
|
github.com/tinylib/msgp v1.6.4 // indirect
|
||||||
|
github.com/valyala/bytebufferpool v1.0.0 // indirect
|
||||||
|
github.com/valyala/fasthttp v1.70.0 // indirect
|
||||||
|
github.com/x448/float16 v0.8.4 // indirect
|
||||||
|
github.com/zap-proto/fiber/v3 v3.2.1 // indirect
|
||||||
|
github.com/zap-proto/go v1.3.0 // indirect
|
||||||
|
github.com/zap-proto/http v0.3.1 // indirect
|
||||||
|
go.yaml.in/yaml/v2 v2.4.3 // indirect
|
||||||
|
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
||||||
|
golang.org/x/crypto v0.50.0 // indirect
|
||||||
|
golang.org/x/net v0.53.0 // indirect
|
||||||
|
golang.org/x/oauth2 v0.30.0 // indirect
|
||||||
|
golang.org/x/sys v0.43.0 // indirect
|
||||||
|
golang.org/x/term v0.42.0 // indirect
|
||||||
|
golang.org/x/text v0.36.0 // indirect
|
||||||
|
golang.org/x/time v0.9.0 // indirect
|
||||||
|
google.golang.org/protobuf v1.36.8 // indirect
|
||||||
|
gopkg.in/evanphx/json-patch.v4 v4.13.0 // indirect
|
||||||
|
gopkg.in/inf.v0 v0.9.1 // indirect
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||||
|
k8s.io/klog/v2 v2.130.1 // indirect
|
||||||
|
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912 // indirect
|
||||||
|
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4 // indirect
|
||||||
|
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 // indirect
|
||||||
|
sigs.k8s.io/randfill v1.0.0 // indirect
|
||||||
|
sigs.k8s.io/structured-merge-diff/v6 v6.3.0 // indirect
|
||||||
|
sigs.k8s.io/yaml v1.6.0 // indirect
|
||||||
|
)
|
||||||
@@ -0,0 +1,169 @@
|
|||||||
|
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
|
||||||
|
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
|
||||||
|
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
|
||||||
|
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
|
||||||
|
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
|
||||||
|
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/emicklei/go-restful/v3 v3.12.2 h1:DhwDP0vY3k8ZzE0RunuJy8GhNpPL6zqLkDf9B/a0/xU=
|
||||||
|
github.com/emicklei/go-restful/v3 v3.12.2/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
|
||||||
|
github.com/fxamacker/cbor/v2 v2.9.1 h1:2rWm8B193Ll4VdjsJY28jxs70IdDsHRWgQYAI80+rMQ=
|
||||||
|
github.com/fxamacker/cbor/v2 v2.9.1/go.mod h1:vM4b+DJCtHn+zz7h3FFp/hDAI9WNWCsZj23V5ytsSxQ=
|
||||||
|
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
|
||||||
|
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||||
|
github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs=
|
||||||
|
github.com/go-openapi/jsonpointer v0.21.0 h1:YgdVicSA9vH5RiHs9TZW5oyafXZFc6+2Vc1rr/O9oNQ=
|
||||||
|
github.com/go-openapi/jsonpointer v0.21.0/go.mod h1:IUyH9l/+uyhIYQ/PXVA41Rexl+kOkAPDdXEYns6fzUY=
|
||||||
|
github.com/go-openapi/jsonreference v0.20.2 h1:3sVjiK66+uXK/6oQ8xgcRKcFgQ5KXa2KvnJRumpMGbE=
|
||||||
|
github.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k=
|
||||||
|
github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14=
|
||||||
|
github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE=
|
||||||
|
github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=
|
||||||
|
github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
|
||||||
|
github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
|
||||||
|
github.com/gofiber/schema v1.7.1 h1:oSJBKdgP8JeIME4TQSAqlNKTU2iBB+2RNmKi8Nsc+TI=
|
||||||
|
github.com/gofiber/schema v1.7.1/go.mod h1:A/X5Ffyru4p9eBdp99qu+nzviHzQiZ7odLT+TwxWhbk=
|
||||||
|
github.com/gofiber/utils/v2 v2.0.4 h1:WwAxUA7L4MW2DjdEHF234lfqvBqd2vYYuBtA9TJq2ec=
|
||||||
|
github.com/gofiber/utils/v2 v2.0.4/go.mod h1:GGERKU3Vhj5z6hS8YKvxL99A54DjOvTFZ0cjZnG4Lj4=
|
||||||
|
github.com/google/gnostic-models v0.7.0 h1:qwTtogB15McXDaNqTZdzPJRHvaVJlAl+HVQnLmJEJxo=
|
||||||
|
github.com/google/gnostic-models v0.7.0/go.mod h1:whL5G0m6dmc5cPxKc5bdKdEN3UjI7OUGxBlw57miDrQ=
|
||||||
|
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||||
|
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||||
|
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
|
||||||
|
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 h1:BHT72Gu3keYf3ZEu2J0b1vyeLSOYI8bm5wbJM/8yDe8=
|
||||||
|
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
|
||||||
|
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||||
|
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||||
|
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
|
||||||
|
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
|
||||||
|
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
|
||||||
|
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
|
||||||
|
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
|
||||||
|
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||||
|
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
|
||||||
|
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
||||||
|
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
|
||||||
|
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
|
||||||
|
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
|
||||||
|
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
||||||
|
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
||||||
|
github.com/luxfi/log v1.4.3 h1:xkUKRWvQ4ZwvlUC2e0/RTtHYZOYSMvSQ9W9lbjwBmiI=
|
||||||
|
github.com/luxfi/log v1.4.3/go.mod h1:myIkufyiQomSQH34K981kbz6cG4WUoerRUh7F4XhlQI=
|
||||||
|
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
|
||||||
|
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
|
||||||
|
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
|
||||||
|
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||||
|
github.com/mattn/go-isatty v0.0.21 h1:xYae+lCNBP7QuW4PUnNG61ffM4hVIfm+zUzDuSzYLGs=
|
||||||
|
github.com/mattn/go-isatty v0.0.21/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4=
|
||||||
|
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||||
|
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
|
||||||
|
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||||
|
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
|
||||||
|
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee h1:W5t00kpgFdJifH4BDsTlE89Zl93FEloxaWZfGcifgq8=
|
||||||
|
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
|
||||||
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
|
||||||
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
||||||
|
github.com/onsi/ginkgo/v2 v2.27.2 h1:LzwLj0b89qtIy6SSASkzlNvX6WktqurSHwkk2ipF/Ns=
|
||||||
|
github.com/onsi/ginkgo/v2 v2.27.2/go.mod h1:ArE1D/XhNXBXCBkKOLkbsb2c81dQHCRcF5zwn/ykDRo=
|
||||||
|
github.com/onsi/gomega v1.38.2 h1:eZCjf2xjZAqe+LeWvKb5weQ+NcPwX84kqJ0cZNxok2A=
|
||||||
|
github.com/onsi/gomega v1.38.2/go.mod h1:W2MJcYxRGV63b418Ai34Ud0hEdTVXq9NW9+Sx6uXf3k=
|
||||||
|
github.com/philhofer/fwd v1.2.0 h1:e6DnBTl7vGY+Gz322/ASL4Gyp1FspeMvx1RNDoToZuM=
|
||||||
|
github.com/philhofer/fwd v1.2.0/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
|
||||||
|
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
|
||||||
|
github.com/shamaton/msgpack/v3 v3.1.0 h1:jsk0vEAqVvvS9+fTZ5/EcQ9tz860c9pWxJ4Iwecz8gU=
|
||||||
|
github.com/shamaton/msgpack/v3 v3.1.0/go.mod h1:DcQG8jrdrQCIxr3HlMYkiXdMhK+KfN2CitkyzsQV4uc=
|
||||||
|
github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY=
|
||||||
|
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||||
|
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||||
|
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||||
|
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||||
|
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
|
||||||
|
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
|
||||||
|
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||||
|
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||||
|
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||||
|
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
||||||
|
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||||
|
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||||
|
github.com/tinylib/msgp v1.6.4 h1:mOwYbyYDLPj35mkA2BjjYejgJk9BuHxDdvRnb6v2ZcQ=
|
||||||
|
github.com/tinylib/msgp v1.6.4/go.mod h1:RSp0LW9oSxFut3KzESt5Voq4GVWyS+PSulT77roAqEA=
|
||||||
|
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
|
||||||
|
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
|
||||||
|
github.com/valyala/fasthttp v1.70.0 h1:LAhMGcWk13QZWm85+eg8ZBNbrq5mnkWFGbHMUJHIdXA=
|
||||||
|
github.com/valyala/fasthttp v1.70.0/go.mod h1:oDZEHHkJ/Buyklg6uURmYs19442zFSnCIfX3j1FY3pE=
|
||||||
|
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
|
||||||
|
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
|
||||||
|
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
|
||||||
|
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
|
||||||
|
github.com/zap-proto/fiber/v3 v3.2.1 h1:k45oKyTwySPtGt8sPz2Ao8OUHc7pDEhai8Np2Ym6Jbg=
|
||||||
|
github.com/zap-proto/fiber/v3 v3.2.1/go.mod h1:eDm2z+ufJrkuE4MeX0Mea4oc/p7/HpXiZjVB+BXCKOA=
|
||||||
|
github.com/zap-proto/go v1.3.0 h1:S3rMoawwhH/BbSZ4G8zG05hJoQnMSMDPzIq75diCTqE=
|
||||||
|
github.com/zap-proto/go v1.3.0/go.mod h1:914SNGTH6Rv3Yu1MweWJBPEN8FZlo5C39QyhaB0C7Q0=
|
||||||
|
github.com/zap-proto/http v0.3.1 h1:A2rCPWYCX866eAsdiWuns0dvWnBmViZtGm4pwX7jwlY=
|
||||||
|
github.com/zap-proto/http v0.3.1/go.mod h1:UYfGhDDCetgxs65XSev8Lpf65COg5vKQK+cWwZGh4zQ=
|
||||||
|
github.com/zap-proto/zip v1.20.0 h1:BMS/DKw4rwLSAK0ZTqtxaihaoIMjxS/t6i421O39nC4=
|
||||||
|
github.com/zap-proto/zip v1.20.0/go.mod h1:EKMmUX9wCPvpkhpMBQRqa17YVXNXRYEjj7X+65Y+J9E=
|
||||||
|
go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
|
||||||
|
go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
|
||||||
|
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
|
||||||
|
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
|
||||||
|
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
|
||||||
|
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
|
||||||
|
golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM=
|
||||||
|
golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU=
|
||||||
|
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
|
||||||
|
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
|
||||||
|
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
|
||||||
|
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
|
||||||
|
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
|
||||||
|
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||||
|
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
|
||||||
|
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||||
|
golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY=
|
||||||
|
golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY=
|
||||||
|
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
|
||||||
|
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
|
||||||
|
golang.org/x/time v0.9.0 h1:EsRrnYcQiGH+5FfbgvV4AP7qEZstoyrHB0DzarOQ4ZY=
|
||||||
|
golang.org/x/time v0.9.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
|
||||||
|
golang.org/x/tools v0.44.0 h1:UP4ajHPIcuMjT1GqzDWRlalUEoY+uzoZKnhOjbIPD2c=
|
||||||
|
golang.org/x/tools v0.44.0/go.mod h1:KA0AfVErSdxRZIsOVipbv3rQhVXTnlU6UhKxHd1seDI=
|
||||||
|
google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc=
|
||||||
|
google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
|
||||||
|
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
|
||||||
|
gopkg.in/evanphx/json-patch.v4 v4.13.0 h1:czT3CmqEaQ1aanPc5SdlgQrrEIb8w/wwCvWWnfEbYzo=
|
||||||
|
gopkg.in/evanphx/json-patch.v4 v4.13.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M=
|
||||||
|
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
|
||||||
|
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
|
||||||
|
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
|
k8s.io/api v0.35.3 h1:pA2fiBc6+N9PDf7SAiluKGEBuScsTzd2uYBkA5RzNWQ=
|
||||||
|
k8s.io/api v0.35.3/go.mod h1:9Y9tkBcFwKNq2sxwZTQh1Njh9qHl81D0As56tu42GA4=
|
||||||
|
k8s.io/apimachinery v0.35.3 h1:MeaUwQCV3tjKP4bcwWGgZ/cp/vpsRnQzqO6J6tJyoF8=
|
||||||
|
k8s.io/apimachinery v0.35.3/go.mod h1:jQCgFZFR1F4Ik7hvr2g84RTJSZegBc8yHgFWKn//hns=
|
||||||
|
k8s.io/client-go v0.35.3 h1:s1lZbpN4uI6IxeTM2cpdtrwHcSOBML1ODNTCCfsP1pg=
|
||||||
|
k8s.io/client-go v0.35.3/go.mod h1:RzoXkc0mzpWIDvBrRnD+VlfXP+lRzqQjCmKtiwZ8Q9c=
|
||||||
|
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
|
||||||
|
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
|
||||||
|
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912 h1:Y3gxNAuB0OBLImH611+UDZcmKS3g6CthxToOb37KgwE=
|
||||||
|
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912/go.mod h1:kdmbQkyfwUagLfXIad1y2TdrjPFWp2Q89B3qkRwf/pQ=
|
||||||
|
k8s.io/metrics v0.35.3 h1:WonA18pEwrtb7a6XfhFg1ZY1Le0RFkcEw7CFApMTZos=
|
||||||
|
k8s.io/metrics v0.35.3/go.mod h1:/O8UBb5QVyAekR2QvL/WWxskpdV1wVSEl4MSLAy4Ql4=
|
||||||
|
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4 h1:SjGebBtkBqHFOli+05xYbK8YF1Dzkbzn+gDM4X9T4Ck=
|
||||||
|
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
|
||||||
|
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 h1:IpInykpT6ceI+QxKBbEflcR5EXP7sU1kvOlxwZh5txg=
|
||||||
|
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730/go.mod h1:mdzfpAEoE6DHQEN0uh9ZbOCuHbLK5wOm7dK4ctXE9Tg=
|
||||||
|
sigs.k8s.io/randfill v1.0.0 h1:JfjMILfT8A6RbawdsK2JXGBR5AQVfd+9TbzrlneTyrU=
|
||||||
|
sigs.k8s.io/randfill v1.0.0/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY=
|
||||||
|
sigs.k8s.io/structured-merge-diff/v6 v6.3.0 h1:jTijUJbW353oVOd9oTlifJqOGEkUw2jB/fXCbTiQEco=
|
||||||
|
sigs.k8s.io/structured-merge-diff/v6 v6.3.0/go.mod h1:M3W8sfWvn2HhQDIbGWj3S099YozAsymCo/wrT5ohRUE=
|
||||||
|
sigs.k8s.io/yaml v1.6.0 h1:G8fkbMSAFqgEFgh4b1wmtzDnioxFCUgTZhlbj5P9QYs=
|
||||||
|
sigs.k8s.io/yaml v1.6.0/go.mod h1:796bPqUfzR/0jLAl6XjHl3Ck7MiyVv8dbTdyT3/pMf4=
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
# Canonical CI/CD config for hanzoai/k8s — read by the hanzoai/ci reusable workflow
|
||||||
|
# (.github/workflows/cicd.yml) and by platform.hanzo.ai.
|
||||||
|
#
|
||||||
|
# This is a zip PLUGIN, so it ships as a BINARY a running host installs and verifies by
|
||||||
|
# digest, not as an image a cluster runs. There is no images: lane and no deploy: lane:
|
||||||
|
# the composition that serves /v1/k8s is hanzoai/cloud's, and it installs this artifact
|
||||||
|
# by URL + SHA-256 from the index published below.
|
||||||
|
binaries:
|
||||||
|
- name: k8s
|
||||||
|
main: .
|
||||||
|
platforms: [linux/amd64, linux/arm64]
|
||||||
|
|
||||||
|
test:
|
||||||
|
- name: go-test
|
||||||
|
# `make test`, never a bare `go test`: the target carries `go vet` and the four
|
||||||
|
# regeneration gates (declaration, OpenAPI, zipdoc, the op/plane bijection), and a
|
||||||
|
# bare run would skip every one of them while still exiting 0.
|
||||||
|
run: |
|
||||||
|
set -e
|
||||||
|
export GOPRIVATE='github.com/hanzoai/*' GOWORK=off
|
||||||
|
make test
|
||||||
+223
@@ -0,0 +1,223 @@
|
|||||||
|
{
|
||||||
|
"name": "k8s",
|
||||||
|
"routes": [
|
||||||
|
{
|
||||||
|
"method": "POST",
|
||||||
|
"pattern": "/v1/k8s/access/can",
|
||||||
|
"op": "k8s_access_can"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/clusters",
|
||||||
|
"op": "k8s_clusters_list"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "POST",
|
||||||
|
"pattern": "/v1/k8s/clusters",
|
||||||
|
"op": "k8s_clusters_register"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "DELETE",
|
||||||
|
"pattern": "/v1/k8s/clusters/:cluster",
|
||||||
|
"op": "k8s_clusters_deregister"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/clusters/:cluster",
|
||||||
|
"op": "k8s_clusters_get"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/clusters/:cluster/status",
|
||||||
|
"op": "k8s_clusters_status"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/jobs",
|
||||||
|
"op": "k8s_jobs_list"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "POST",
|
||||||
|
"pattern": "/v1/k8s/jobs",
|
||||||
|
"op": "k8s_jobs_create"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/jobs/:namespace/:name",
|
||||||
|
"op": "k8s_jobs_get"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/metrics/pods",
|
||||||
|
"op": "k8s_metrics_pods"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/namespaces",
|
||||||
|
"op": "k8s_namespaces_list"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "POST",
|
||||||
|
"pattern": "/v1/k8s/namespaces",
|
||||||
|
"op": "k8s_namespaces_ensure"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "DELETE",
|
||||||
|
"pattern": "/v1/k8s/namespaces/:namespace",
|
||||||
|
"op": "k8s_namespaces_delete"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/namespaces/:namespace",
|
||||||
|
"op": "k8s_namespaces_get"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/nodes",
|
||||||
|
"op": "k8s_nodes_list"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "POST",
|
||||||
|
"pattern": "/v1/k8s/nodes/:node/cordon",
|
||||||
|
"op": "k8s_nodes_cordon"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/nodes/volume-stats",
|
||||||
|
"op": "k8s_nodes_volume_stats"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/pods",
|
||||||
|
"op": "k8s_pods_list"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/pods/:namespace/:pod/logs",
|
||||||
|
"op": "k8s_pods_logs"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/resources",
|
||||||
|
"op": "k8s_resources_list"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "PUT",
|
||||||
|
"pattern": "/v1/k8s/resources",
|
||||||
|
"op": "k8s_resources_apply"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "DELETE",
|
||||||
|
"pattern": "/v1/k8s/resources/:kind/:namespace/:name",
|
||||||
|
"op": "k8s_resources_delete"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/resources/:kind/:namespace/:name",
|
||||||
|
"op": "k8s_resources_get"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/resources/watch",
|
||||||
|
"op": "k8s_resources_watch"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/volumes/claims",
|
||||||
|
"op": "k8s_volumes_list_claims"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "POST",
|
||||||
|
"pattern": "/v1/k8s/volumes/claims",
|
||||||
|
"op": "k8s_volumes_ensure_claim"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "POST",
|
||||||
|
"pattern": "/v1/k8s/volumes/claims/:namespace/:name/expand",
|
||||||
|
"op": "k8s_volumes_expand_claim"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/volumes/persistent",
|
||||||
|
"op": "k8s_volumes_list_persistent"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/workloads/configmaps",
|
||||||
|
"op": "k8s_workloads_list_configmaps"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/workloads/deployments",
|
||||||
|
"op": "k8s_workloads_list_deployments"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/workloads/deployments/:namespace/:name",
|
||||||
|
"op": "k8s_workloads_get_deployment"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/workloads/events",
|
||||||
|
"op": "k8s_workloads_list_events"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/workloads/ingresses",
|
||||||
|
"op": "k8s_workloads_list_ingresses"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/workloads/replicasets",
|
||||||
|
"op": "k8s_workloads_list_replicasets"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/workloads/services",
|
||||||
|
"op": "k8s_workloads_list_services"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"method": "GET",
|
||||||
|
"pattern": "/v1/k8s/workloads/statefulsets",
|
||||||
|
"op": "k8s_workloads_list_statefulsets"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"ops": [
|
||||||
|
"k8s_access_can",
|
||||||
|
"k8s_clusters_deregister",
|
||||||
|
"k8s_clusters_get",
|
||||||
|
"k8s_clusters_list",
|
||||||
|
"k8s_clusters_register",
|
||||||
|
"k8s_clusters_status",
|
||||||
|
"k8s_jobs_create",
|
||||||
|
"k8s_jobs_get",
|
||||||
|
"k8s_jobs_list",
|
||||||
|
"k8s_metrics_pods",
|
||||||
|
"k8s_namespaces_delete",
|
||||||
|
"k8s_namespaces_ensure",
|
||||||
|
"k8s_namespaces_get",
|
||||||
|
"k8s_namespaces_list",
|
||||||
|
"k8s_nodes_cordon",
|
||||||
|
"k8s_nodes_list",
|
||||||
|
"k8s_nodes_volume_stats",
|
||||||
|
"k8s_pods_list",
|
||||||
|
"k8s_pods_logs",
|
||||||
|
"k8s_resources_apply",
|
||||||
|
"k8s_resources_delete",
|
||||||
|
"k8s_resources_get",
|
||||||
|
"k8s_resources_list",
|
||||||
|
"k8s_resources_watch",
|
||||||
|
"k8s_volumes_ensure_claim",
|
||||||
|
"k8s_volumes_expand_claim",
|
||||||
|
"k8s_volumes_list_claims",
|
||||||
|
"k8s_volumes_list_persistent",
|
||||||
|
"k8s_workloads_get_deployment",
|
||||||
|
"k8s_workloads_list_configmaps",
|
||||||
|
"k8s_workloads_list_deployments",
|
||||||
|
"k8s_workloads_list_events",
|
||||||
|
"k8s_workloads_list_ingresses",
|
||||||
|
"k8s_workloads_list_replicasets",
|
||||||
|
"k8s_workloads_list_services",
|
||||||
|
"k8s_workloads_list_statefulsets"
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
// Command k8s is Hanzo's Kubernetes service: the ONE place in the fleet that holds a
|
||||||
|
// cluster credential and the ONE API — /v1/k8s — through which anything reaches a
|
||||||
|
// cluster.
|
||||||
|
//
|
||||||
|
// It builds as its own binary. A host composes it at run time; run directly it is an
|
||||||
|
// ordinary HIP-0119 service on its own port. Same binary, no second code path.
|
||||||
|
//
|
||||||
|
// # Why this is a service and not a library
|
||||||
|
//
|
||||||
|
// A cluster client linked into N apps means N binaries holding cluster credentials.
|
||||||
|
// Before this repo existed, nine places in the fleet built one: two with the ambient
|
||||||
|
// in-cluster ServiceAccount, one with a cluster-admin kubeconfig minted on demand for
|
||||||
|
// every cluster on the house account, and one with a kubeconfig read out of a database
|
||||||
|
// row and fed to a function that applied arbitrary YAML. That last one is the shape of
|
||||||
|
// the whole problem: with it, no ClusterRole smaller than "everything" could be
|
||||||
|
// written, so nothing was least-privileged and no reviewer could bound a blast radius.
|
||||||
|
//
|
||||||
|
// One owner, named ops, credentials in KMS. Consumers ASK.
|
||||||
|
//
|
||||||
|
// # Multi-cluster from the first line
|
||||||
|
//
|
||||||
|
// lux runs lux-k8s. zoo runs zoo-k8s. Customers run their own. So an org's resources
|
||||||
|
// may live on a different cluster from Hanzo's, and "our cluster" is not a thing this
|
||||||
|
// program can mean. Every op names its cluster, resolved through the registry against
|
||||||
|
// the calling org — and there is no ambient client to fall back to, because
|
||||||
|
// registry.Reach is the only client constructor in the module and it takes a name.
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
"github.com/zap-proto/zip/middleware"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/ops"
|
||||||
|
"github.com/hanzoai/k8s/registry"
|
||||||
|
)
|
||||||
|
|
||||||
|
// name is this app's ONE name: the binary, the socket stem, the Declaration name and
|
||||||
|
// the <org>-<app> IAM segment. No mapping table.
|
||||||
|
const name = "k8s"
|
||||||
|
|
||||||
|
// defaultAddr is where this serves when run DIRECTLY rather than by a host. Under a
|
||||||
|
// host, zip.Addr ignores it and uses the private socket the host created — which is the
|
||||||
|
// whole plugin side of the transport contract, and the reason a fixed bind here would
|
||||||
|
// make every child but the first die on "address already in use".
|
||||||
|
const defaultAddr = ":9663"
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
if err := run(); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "%s: %v\n", name, err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func run() error {
|
||||||
|
app := zip.New(zip.Config{AppName: name})
|
||||||
|
app.Use(middleware.Recover(), middleware.RequestID(), middleware.Logger(app.Logger()))
|
||||||
|
|
||||||
|
// The store is the KMS peer over the call plane. It is constructed, not dialed:
|
||||||
|
// zip.Ask dials lazily on first use, so nothing here touches a peer before the
|
||||||
|
// describe check below — which is required, because a projection is a function of
|
||||||
|
// the code and a describe run must not need a running fleet.
|
||||||
|
ops.Ops{Reg: registry.New(registry.Sealed{})}.Mount(app)
|
||||||
|
|
||||||
|
if done, err := app.Described(); done {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return app.Listen(zip.Addr(defaultAddr))
|
||||||
|
}
|
||||||
+3058
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,57 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
authv1 "k8s.io/api/authorization/v1"
|
||||||
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// can asks a cluster whether this app's own credential may perform one act.
|
||||||
|
//
|
||||||
|
// It is a SelfSubjectAccessReview, which means the answer comes from the cluster's
|
||||||
|
// RBAC rather than from a grant this app asserts about itself. That distinction is the
|
||||||
|
// reason the op exists: the union ClusterRole this deployment needs is granted OUT of
|
||||||
|
// tree, so a drifted or half-applied grant is a real and undetectable condition — and
|
||||||
|
// asking beats asserting, because a refusal a caller can read is a feature it can
|
||||||
|
// degrade gracefully on instead of a 403 arriving in the middle of a mutation.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","verb":"get","resource":"resourcequotas","namespace":"acme-prod"}
|
||||||
|
// Response: {"allowed":true}
|
||||||
|
func (o Ops) can(ctx context.Context, in *plane.CanIn) (*plane.Can, error) {
|
||||||
|
if err := need("verb", in.Verb); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := need("resource", in.Resource); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
// A namespace, when given, is bounded like any other: asking about a namespace a
|
||||||
|
// registration may not address would let a tenant probe another tenant's RBAC.
|
||||||
|
ns := ""
|
||||||
|
if in.Namespace != "" {
|
||||||
|
ns, err = b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
} else if err := b.WholeCluster(); err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
review := &authv1.SelfSubjectAccessReview{
|
||||||
|
Spec: authv1.SelfSubjectAccessReviewSpec{
|
||||||
|
ResourceAttributes: &authv1.ResourceAttributes{
|
||||||
|
Verb: in.Verb, Group: in.Group, Resource: in.Resource, Namespace: ns,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
got, err := b.Typed.AuthorizationV1().SelfSubjectAccessReviews().Create(ctx, review, metav1.CreateOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
return &plane.Can{Allowed: got.Status.Allowed, Reason: got.Status.Reason}, nil
|
||||||
|
}
|
||||||
+149
@@ -0,0 +1,149 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// listClusters returns every Kubernetes cluster the calling org may reach.
|
||||||
|
//
|
||||||
|
// This is the fleet's ONE cluster list. A row is a real registration — a name, the
|
||||||
|
// provider it came from, its apiserver endpoint and the node and accelerator counts
|
||||||
|
// measured when it was attached — and there is no such thing as an unregistered
|
||||||
|
// cluster this API can see. The credential is never part of a row: it is sealed in
|
||||||
|
// KMS and resolved only at the moment an op uses it.
|
||||||
|
//
|
||||||
|
// The list is scoped by the caller's org and by nothing the caller can type, so a
|
||||||
|
// tenant sees its own clusters and no others.
|
||||||
|
//
|
||||||
|
// Response: {"clusters":[{"name":"hanzo-k8s","org":"hanzo","provider":"doks","endpoint":"https://k8s.example.com","nodes":6,"nvidiaGpu":8,"amdGpu":0,"registered":"2026-07-30T00:00:00Z","default":true}]}
|
||||||
|
func (o Ops) listClusters(ctx context.Context, _ *plane.NoArgs) (*plane.Clusters, error) {
|
||||||
|
list, err := o.Reg.List(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := make([]plane.Registered, 0, len(list))
|
||||||
|
for _, c := range list {
|
||||||
|
out = append(out, c.View())
|
||||||
|
}
|
||||||
|
return &plane.Clusters{Clusters: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// getCluster returns one registered cluster.
|
||||||
|
//
|
||||||
|
// A name the calling org has not registered answers 404 — including a name another
|
||||||
|
// org HAS registered, which is indistinguishable from one that does not exist. That
|
||||||
|
// is deliberate: a distinguishable refusal would let a tenant enumerate other
|
||||||
|
// tenants' cluster names one guess at a time.
|
||||||
|
//
|
||||||
|
// Response: {"name":"lux-k8s","org":"lux","provider":"byo","endpoint":"https://k8s.lux.example","nodes":12,"nvidiaGpu":32,"amdGpu":0,"registered":"2026-07-30T00:00:00Z","default":false}
|
||||||
|
func (o Ops) getCluster(ctx context.Context, in *plane.ClusterRef) (*plane.Registered, error) {
|
||||||
|
if err := need("cluster", in.Cluster); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
c, err := o.Reg.Get(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
v := c.View()
|
||||||
|
return &v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// registerCluster attaches a Kubernetes cluster to the calling org and answers 201.
|
||||||
|
//
|
||||||
|
// This is the cluster-registry primitive and the most important operation in this
|
||||||
|
// API: everything else takes a cluster name, and this is where a name comes to
|
||||||
|
// exist. It is also the whole of BYO-Kubernetes — an org running its own cluster,
|
||||||
|
// lux running lux-k8s, a customer bringing their own — because a registered cluster
|
||||||
|
// is a registered cluster whatever provisioned it.
|
||||||
|
//
|
||||||
|
// The kubeconfig is refused if it carries an exec or auth-provider credential
|
||||||
|
// plugin (either would run a local binary with this process's environment), and
|
||||||
|
// refused if its apiserver is not a routable https host. It is then USED, once, to
|
||||||
|
// prove the cluster answers and to measure its nodes and accelerators; a cluster
|
||||||
|
// that cannot be reached is not registered, because the alternative is a row that
|
||||||
|
// fails at first use when nobody is holding the credential any more. Finally it is
|
||||||
|
// sealed in KMS under a ref derived from the owning org and the name, and dropped.
|
||||||
|
// It is never written to disk, never logged, and no operation returns it.
|
||||||
|
//
|
||||||
|
// Registration is idempotent on name within the org, so re-registering rotates the
|
||||||
|
// credential and refreshes the inventory without a second verb.
|
||||||
|
//
|
||||||
|
// Set `namespaces` when the credential is a scoped ServiceAccount on a shared
|
||||||
|
// cluster: the registration is then bounded to those namespaces and every later op
|
||||||
|
// is refused outside them, before the apiserver is dialed. Omit it when the org owns
|
||||||
|
// the whole cluster.
|
||||||
|
//
|
||||||
|
// Example: {"name":"lux-k8s","kubeconfig":"apiVersion: v1\nkind: Config\n...","provider":"byo","default":true}
|
||||||
|
// Response: {"name":"lux-k8s","org":"lux","provider":"byo","endpoint":"https://k8s.lux.example","nodes":12,"nvidiaGpu":32,"amdGpu":0,"registered":"2026-07-30T00:00:00Z","default":true}
|
||||||
|
func (o Ops) registerCluster(ctx context.Context, in *plane.RegisterIn) (*plane.Registered, error) {
|
||||||
|
if err := need("name", in.Name); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := need("kubeconfig", in.Kubeconfig); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
rec, err := o.Reg.Register(ctx, in)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
v := rec.View()
|
||||||
|
return &v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// deregisterCluster detaches a cluster from the calling org and destroys its sealed
|
||||||
|
// credential.
|
||||||
|
//
|
||||||
|
// Nothing in the cluster is touched: this removes Hanzo's ability to reach it, not
|
||||||
|
// the cluster itself. Deleting a name that is not registered reports removed=false
|
||||||
|
// rather than failing, so a repeated deregistration is safe.
|
||||||
|
//
|
||||||
|
// Response: {"removed":true}
|
||||||
|
func (o Ops) deregisterCluster(ctx context.Context, in *plane.ClusterRef) (*plane.Deregistered, error) {
|
||||||
|
if err := need("cluster", in.Cluster); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
removed, err := o.Reg.Deregister(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
return &plane.Deregistered{Removed: removed}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// clusterStatus reports whether a registered cluster answers right now.
|
||||||
|
//
|
||||||
|
// The probe is the apiserver's version endpoint, because that is the one read every
|
||||||
|
// credential can make regardless of what RBAC it was granted: a registration bounded
|
||||||
|
// to one namespace would fail a namespace list and look dead while being perfectly
|
||||||
|
// healthy. So this answers "is the credential live and the apiserver reachable",
|
||||||
|
// which is the question a board asks, and the ops that need a permission ask for
|
||||||
|
// that permission themselves.
|
||||||
|
//
|
||||||
|
// A cluster that does not answer is connected=false with the reason verbatim — never
|
||||||
|
// an error, because "this cluster is down" is a successful answer to this question.
|
||||||
|
//
|
||||||
|
// Response: {"connected":true,"version":"v1.33.1"}
|
||||||
|
func (o Ops) clusterStatus(ctx context.Context, in *plane.ClusterRef) (*plane.Reachable, error) {
|
||||||
|
if err := need("cluster", in.Cluster); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
// A cluster that is not registered, or that a tenant may not name, is a
|
||||||
|
// refusal about the REQUEST and must not be dressed up as a down cluster.
|
||||||
|
var he *zip.HTTPError
|
||||||
|
if errors.As(err, &he) && he.Status < 500 {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &plane.Reachable{Reason: err.Error()}, nil
|
||||||
|
}
|
||||||
|
v, verr := b.Typed.Discovery().ServerVersion()
|
||||||
|
if verr != nil {
|
||||||
|
return &plane.Reachable{Reason: verr.Error()}, nil
|
||||||
|
}
|
||||||
|
return &plane.Reachable{Connected: true, Version: v.GitVersion}, nil
|
||||||
|
}
|
||||||
+191
@@ -0,0 +1,191 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"strconv"
|
||||||
|
|
||||||
|
batchv1 "k8s.io/api/batch/v1"
|
||||||
|
corev1 "k8s.io/api/core/v1"
|
||||||
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// createJob runs one batch Job and answers 201.
|
||||||
|
//
|
||||||
|
// The input is a JOB, not a PodSpec: an image, a command, environment, a service
|
||||||
|
// account, labels, and claims to mount. That is the whole of what the fleet's job
|
||||||
|
// callers need — an image build, a migration, a scheduled task — and it is deliberately
|
||||||
|
// not the Kubernetes PodSpec, because accepting a PodSpec would mean accepting
|
||||||
|
// hostPath mounts, host networking, privileged security contexts and secret volume
|
||||||
|
// references, i.e. everything a bounded surface exists to exclude.
|
||||||
|
//
|
||||||
|
// `env` is PLAIN environment. Secret material must not travel here: seal it in KMS and
|
||||||
|
// give the job a ServiceAccount that can reach what it needs. `ttlSeconds` is worth
|
||||||
|
// setting on anything periodic — a namespace that keeps every finished Job becomes a
|
||||||
|
// list nobody can read.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","namespace":"hanzo","name":"build-1a2b","image":"gcr.io/kaniko-project/executor:latest","args":["--context=git://…","--destination=registry.hanzo.ai/acme/api:1.4.2"],"ttlSeconds":3600}
|
||||||
|
// Response: {"namespace":"hanzo","name":"build-1a2b","active":1,"succeeded":0,"failed":0,"phase":"running"}
|
||||||
|
func (o Ops) createJob(ctx context.Context, in *plane.JobIn) (*plane.Job, error) {
|
||||||
|
if err := need("name", in.Name); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := need("image", in.Image); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
labels := map[string]string{"managed-by": managedBy}
|
||||||
|
for k, v := range in.Labels {
|
||||||
|
labels[k] = v
|
||||||
|
}
|
||||||
|
env := make([]corev1.EnvVar, 0, len(in.Env))
|
||||||
|
for k, v := range in.Env {
|
||||||
|
env = append(env, corev1.EnvVar{Name: k, Value: v})
|
||||||
|
}
|
||||||
|
container := corev1.Container{
|
||||||
|
Name: "main", Image: in.Image, Command: in.Command, Args: in.Args, Env: env,
|
||||||
|
}
|
||||||
|
var volumes []corev1.Volume
|
||||||
|
for i, m := range in.Mounts {
|
||||||
|
if m.Claim == "" || m.Path == "" {
|
||||||
|
return nil, zip.ErrBadRequest("every mount needs both a claim and a path")
|
||||||
|
}
|
||||||
|
name := volumeName(i)
|
||||||
|
volumes = append(volumes, corev1.Volume{
|
||||||
|
Name: name,
|
||||||
|
VolumeSource: corev1.VolumeSource{
|
||||||
|
PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: m.Claim},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
container.VolumeMounts = append(container.VolumeMounts, corev1.VolumeMount{Name: name, MountPath: m.Path})
|
||||||
|
}
|
||||||
|
job := &batchv1.Job{
|
||||||
|
ObjectMeta: metav1.ObjectMeta{Name: in.Name, Namespace: ns, Labels: labels},
|
||||||
|
Spec: batchv1.JobSpec{
|
||||||
|
Template: corev1.PodTemplateSpec{
|
||||||
|
ObjectMeta: metav1.ObjectMeta{Labels: labels},
|
||||||
|
Spec: corev1.PodSpec{
|
||||||
|
RestartPolicy: corev1.RestartPolicyNever,
|
||||||
|
ServiceAccountName: in.ServiceAccount,
|
||||||
|
Containers: []corev1.Container{container},
|
||||||
|
Volumes: volumes,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if in.BackoffLimit > 0 {
|
||||||
|
limit := in.BackoffLimit
|
||||||
|
job.Spec.BackoffLimit = &limit
|
||||||
|
}
|
||||||
|
if in.TTLSeconds > 0 {
|
||||||
|
ttl := in.TTLSeconds
|
||||||
|
job.Spec.TTLSecondsAfterFinished = &ttl
|
||||||
|
}
|
||||||
|
created, err := b.Typed.BatchV1().Jobs(ns).Create(ctx, job, metav1.CreateOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
v := jobView(*created)
|
||||||
|
return &v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// listJobs returns batch Jobs with a rolled-up phase.
|
||||||
|
//
|
||||||
|
// `phase` is one word — running, succeeded or failed — computed here rather than left
|
||||||
|
// to every caller to derive from three counters. Three consumers deriving it three ways
|
||||||
|
// is three chances to read a failed job as merely unfinished.
|
||||||
|
//
|
||||||
|
// Response: {"jobs":[{"namespace":"hanzo","name":"build-1a2b","active":0,"succeeded":1,"failed":0,"phase":"succeeded","startedAt":1785110400,"endedAt":1785110460}]}
|
||||||
|
func (o Ops) listJobs(ctx context.Context, in *plane.Selector) (*plane.Jobs, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := []plane.Job{}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Typed.BatchV1().Jobs(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
out = append(out, jobView(list.Items[i]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &plane.Jobs{Jobs: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// getJob returns one batch Job.
|
||||||
|
//
|
||||||
|
// Response: {"namespace":"hanzo","name":"build-1a2b","active":0,"succeeded":1,"failed":0,"phase":"succeeded","startedAt":1785110400,"endedAt":1785110460}
|
||||||
|
func (o Ops) getJob(ctx context.Context, in *plane.NamedIn) (*plane.Job, error) {
|
||||||
|
if err := need("name", in.Name); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
j, err := b.Typed.BatchV1().Jobs(ns).Get(ctx, in.Name, metav1.GetOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
v := jobView(*j)
|
||||||
|
return &v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func jobView(j batchv1.Job) plane.Job {
|
||||||
|
v := plane.Job{
|
||||||
|
Namespace: j.Namespace, Name: j.Name,
|
||||||
|
Active: j.Status.Active, Succeeded: j.Status.Succeeded, Failed: j.Status.Failed,
|
||||||
|
Phase: "running",
|
||||||
|
}
|
||||||
|
// A completed condition is authoritative over the counters, which is why it is read
|
||||||
|
// first: a Job whose pod succeeded and was then garbage-collected has zero of
|
||||||
|
// everything, and reading the counters alone would report it as still running.
|
||||||
|
for _, c := range j.Status.Conditions {
|
||||||
|
if c.Status != corev1.ConditionTrue {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
switch c.Type {
|
||||||
|
case batchv1.JobComplete:
|
||||||
|
v.Phase = "succeeded"
|
||||||
|
case batchv1.JobFailed:
|
||||||
|
v.Phase = "failed"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v.Phase == "running" && j.Status.Failed > 0 && j.Status.Active == 0 && j.Status.Succeeded == 0 {
|
||||||
|
v.Phase = "failed"
|
||||||
|
}
|
||||||
|
if j.Status.StartTime != nil {
|
||||||
|
v.StartedAt = j.Status.StartTime.Unix()
|
||||||
|
}
|
||||||
|
if j.Status.CompletionTime != nil {
|
||||||
|
v.EndedAt = j.Status.CompletionTime.Unix()
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
// volumeName names a job's Nth mount. Derived from the INDEX rather than from the
|
||||||
|
// claim, so two claims whose names collide after DNS normalisation still get distinct
|
||||||
|
// volumes, and stable so a re-created job mounts the same way.
|
||||||
|
func volumeName(i int) string {
|
||||||
|
return "mount-" + strconv.Itoa(i)
|
||||||
|
}
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||||
|
"k8s.io/apimachinery/pkg/api/meta"
|
||||||
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||||
|
"k8s.io/client-go/discovery"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// podMetrics returns live CPU and memory use for pods, from metrics.k8s.io.
|
||||||
|
//
|
||||||
|
// `available` is the field that carries the honesty. metrics-server is absent from most
|
||||||
|
// of our clusters, and a cluster without it answers this call with an error — which,
|
||||||
|
// rendered as zeroes, tells an operator that a busy namespace is idle. So a missing
|
||||||
|
// metrics API is reported as available=false with no rows, and a caller must render
|
||||||
|
// "unmeasured" rather than "0m".
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","namespace":"acme-prod"}
|
||||||
|
// Response: {"pods":[{"namespace":"acme-prod","name":"api-7d9f-2xk","cpuMilli":250,"memoryBytes":536870912}],"available":true}
|
||||||
|
func (o Ops) podMetrics(ctx context.Context, in *plane.Selector) (*plane.PodMetrics, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := &plane.PodMetrics{Pods: []plane.PodMetric{}}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Metrics.MetricsV1beta1().PodMetricses(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
// One namespace failing on a cluster that HAS metrics is a real error and
|
||||||
|
// must surface; a cluster with no metrics API at all fails every namespace
|
||||||
|
// the same way, and the caller needs that distinction. The apiserver
|
||||||
|
// answers a missing aggregated API with NotFound or ServiceUnavailable,
|
||||||
|
// so those two are the "no metrics here" answer and nothing else is.
|
||||||
|
if noMetrics(err) {
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
m := list.Items[i]
|
||||||
|
row := plane.PodMetric{Namespace: m.Namespace, Name: m.Name}
|
||||||
|
for _, c := range m.Containers {
|
||||||
|
row.CPUMilli += c.Usage.Cpu().MilliValue()
|
||||||
|
row.MemoryBytes += c.Usage.Memory().Value()
|
||||||
|
}
|
||||||
|
out.Pods = append(out.Pods, row)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out.Available = true
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// noMetrics reports whether an error means the cluster serves no metrics API, as
|
||||||
|
// distinct from a metrics API that refused this read.
|
||||||
|
func noMetrics(err error) bool {
|
||||||
|
return apierrors.IsNotFound(err) || apierrors.IsServiceUnavailable(err) ||
|
||||||
|
meta.IsNoMatchError(err) || discovery.IsGroupDiscoveryFailedError(err)
|
||||||
|
}
|
||||||
@@ -0,0 +1,155 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
corev1 "k8s.io/api/core/v1"
|
||||||
|
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||||
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// managedBy labels everything this app creates, so a cluster can be read for what
|
||||||
|
// Hanzo put on it without inferring ownership from a naming convention.
|
||||||
|
const managedBy = "hanzo-cloud"
|
||||||
|
|
||||||
|
// listNamespaces returns the namespaces the calling org's registration may address.
|
||||||
|
//
|
||||||
|
// For a registration that owns the whole cluster this is every namespace on it. For
|
||||||
|
// one bounded to a namespace set it is that set, read one namespace at a time — so a
|
||||||
|
// scoped tenant learns nothing about its neighbours, not even how many there are.
|
||||||
|
//
|
||||||
|
// Response: {"namespaces":[{"name":"acme-prod","phase":"Active","labels":{"managed-by":"hanzo-cloud"}}]}
|
||||||
|
func (o Ops) listNamespaces(ctx context.Context, in *plane.ClusterRef) (*plane.Namespaces, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if bound := b.Cluster.Namespaces; len(bound) > 0 {
|
||||||
|
out := make([]plane.Namespace, 0, len(bound))
|
||||||
|
for _, name := range bound {
|
||||||
|
ns, err := b.Typed.CoreV1().Namespaces().Get(ctx, name, metav1.GetOptions{})
|
||||||
|
if apierrors.IsNotFound(err) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out = append(out, namespaceView(*ns))
|
||||||
|
}
|
||||||
|
return &plane.Namespaces{Namespaces: out}, nil
|
||||||
|
}
|
||||||
|
list, err := b.Typed.CoreV1().Namespaces().List(ctx, metav1.ListOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := make([]plane.Namespace, 0, len(list.Items))
|
||||||
|
for i := range list.Items {
|
||||||
|
out = append(out, namespaceView(list.Items[i]))
|
||||||
|
}
|
||||||
|
return &plane.Namespaces{Namespaces: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// getNamespace returns one namespace.
|
||||||
|
//
|
||||||
|
// Response: {"name":"acme-prod","phase":"Active","labels":{"managed-by":"hanzo-cloud"}}
|
||||||
|
func (o Ops) getNamespace(ctx context.Context, in *plane.NamespaceRef) (*plane.Namespace, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
got, err := b.Typed.CoreV1().Namespaces().Get(ctx, ns, metav1.GetOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
v := namespaceView(*got)
|
||||||
|
return &v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ensureNamespace creates a namespace if it is absent, and reports which it did.
|
||||||
|
//
|
||||||
|
// Four separate implementations of get-then-create-if-missing collapse into this one
|
||||||
|
// name. That matters beyond tidiness: each of those four wrote the ladder itself, and
|
||||||
|
// a ladder is a race — "get said absent" and "create" are two calls, and two callers
|
||||||
|
// starting a tenant at once both take the create branch. Here an AlreadyExists from
|
||||||
|
// the apiserver is the successful "it was already there" answer, so the race has one
|
||||||
|
// outcome instead of one winner and one error.
|
||||||
|
//
|
||||||
|
// `created` is reported because creating a tenant's namespace and finding it are
|
||||||
|
// different events, and a caller that cannot tell them apart cannot log the first.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","namespace":"acme-prod","labels":{"tier":"pro"}}
|
||||||
|
// Response: {"name":"acme-prod","created":true}
|
||||||
|
func (o Ops) ensureNamespace(ctx context.Context, in *plane.EnsureNamespaceIn) (*plane.Ensured, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
if _, err := b.Typed.CoreV1().Namespaces().Get(ctx, ns, metav1.GetOptions{}); err == nil {
|
||||||
|
return &plane.Ensured{Name: ns}, nil
|
||||||
|
} else if !apierrors.IsNotFound(err) {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
labels := map[string]string{"managed-by": managedBy}
|
||||||
|
for k, v := range in.Labels {
|
||||||
|
labels[k] = v
|
||||||
|
}
|
||||||
|
_, err = b.Typed.CoreV1().Namespaces().Create(ctx, &corev1.Namespace{
|
||||||
|
ObjectMeta: metav1.ObjectMeta{Name: ns, Labels: labels},
|
||||||
|
}, metav1.CreateOptions{})
|
||||||
|
if apierrors.IsAlreadyExists(err) {
|
||||||
|
return &plane.Ensured{Name: ns}, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
return &plane.Ensured{Name: ns, Created: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// deleteNamespace deletes a namespace and everything in it.
|
||||||
|
//
|
||||||
|
// This is the highest blast radius in the whole API: Kubernetes deletes a namespace
|
||||||
|
// by garbage-collecting every object inside it, including PersistentVolumeClaims,
|
||||||
|
// which for a Delete-reclaim StorageClass destroys the tenant's data. There is no
|
||||||
|
// undo and no confirmation this API can offer that would make one.
|
||||||
|
//
|
||||||
|
// It is therefore its own named operation, gated separately from every other write,
|
||||||
|
// and it is refused outright for a registration bounded to a namespace set: a scoped
|
||||||
|
// credential exists precisely because the cluster is shared, and "delete a namespace
|
||||||
|
// on a shared cluster" is not an authority a tenant holds.
|
||||||
|
//
|
||||||
|
// Response: {"removed":true}
|
||||||
|
func (o Ops) deleteNamespace(ctx context.Context, in *plane.NamespaceRef) (*plane.Deleted, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := b.WholeCluster(); err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
err = b.Typed.CoreV1().Namespaces().Delete(ctx, ns, metav1.DeleteOptions{})
|
||||||
|
if apierrors.IsNotFound(err) {
|
||||||
|
return &plane.Deleted{}, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
return &plane.Deleted{Removed: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func namespaceView(n corev1.Namespace) plane.Namespace {
|
||||||
|
return plane.Namespace{Name: n.Name, Phase: string(n.Status.Phase), Labels: n.Labels}
|
||||||
|
}
|
||||||
+271
@@ -0,0 +1,271 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
corev1 "k8s.io/api/core/v1"
|
||||||
|
policyv1 "k8s.io/api/policy/v1"
|
||||||
|
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||||
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||||
|
"k8s.io/apimachinery/pkg/types"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
"github.com/hanzoai/k8s/registry"
|
||||||
|
)
|
||||||
|
|
||||||
|
// kubeletFanout bounds the per-cluster kubelet sweep. Each read is one small
|
||||||
|
// document through the apiserver's node proxy, so this bounds pressure on the
|
||||||
|
// apiserver rather than local work.
|
||||||
|
const kubeletFanout = 16
|
||||||
|
|
||||||
|
// listNodes returns a cluster's worker nodes: whether each is up, whether it will
|
||||||
|
// take new work, and what accelerators it offers.
|
||||||
|
//
|
||||||
|
// This is the one node inventory in the fleet. The GPU counts come from each node's
|
||||||
|
// allocatable `nvidia.com/gpu` and `amd.com/gpu`, which is what the scheduler will
|
||||||
|
// actually honour — not a label somebody typed.
|
||||||
|
//
|
||||||
|
// Nodes are cluster-scoped, so a registration bounded to a namespace set cannot read
|
||||||
|
// them: answering would hand one tenant on a shared cluster the whole cluster's
|
||||||
|
// capacity.
|
||||||
|
//
|
||||||
|
// Response: {"nodes":[{"name":"pool-gpu-a1","ready":true,"schedulable":true,"region":"nyc3","instanceKind":"gpu-h100x8-640gb","cpu":"64","memory":"755Gi","nvidiaGpu":8,"amdGpu":0,"internalIp":"10.0.0.4"}]}
|
||||||
|
func (o Ops) listNodes(ctx context.Context, in *plane.ClusterRef) (*plane.Nodes, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := b.WholeCluster(); err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
list, err := b.Typed.CoreV1().Nodes().List(ctx, metav1.ListOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := make([]plane.Node, 0, len(list.Items))
|
||||||
|
for i := range list.Items {
|
||||||
|
out = append(out, nodeView(list.Items[i]))
|
||||||
|
}
|
||||||
|
return &plane.Nodes{Nodes: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// cordonNode takes a node out of service, or puts it back, and optionally drains it.
|
||||||
|
//
|
||||||
|
// This is ONE named operation over what is really three calls — patch
|
||||||
|
// `spec.unschedulable`, list the node's pods, evict each one — because the three are
|
||||||
|
// only ever wanted together and a generic "patch a node" op would let a caller write
|
||||||
|
// any field of any node's spec. There is deliberately no such op.
|
||||||
|
//
|
||||||
|
// The drain uses the EVICTION api, not delete: an eviction respects
|
||||||
|
// PodDisruptionBudgets, so a drain that would break a quorum is REFUSED by the
|
||||||
|
// apiserver and reported verbatim, with the node left cordoned so an operator can
|
||||||
|
// scale and retry. DaemonSet and static (mirror) pods are skipped — the node
|
||||||
|
// recreates them immediately, so evicting them is a loop — and pods already
|
||||||
|
// terminal are skipped too.
|
||||||
|
//
|
||||||
|
// `drain` is ignored when `schedulable` is true, because draining a node you just
|
||||||
|
// returned to service is incoherent rather than merely useless.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","node":"pool-gpu-a1","schedulable":false,"drain":true}
|
||||||
|
// Response: {"evicted":11}
|
||||||
|
func (o Ops) cordonNode(ctx context.Context, in *plane.CordonIn) (*plane.Cordoned, error) {
|
||||||
|
if err := need("node", in.Node); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := b.WholeCluster(); err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
patch := fmt.Sprintf(`{"spec":{"unschedulable":%t}}`, !in.Schedulable)
|
||||||
|
if _, err := b.Typed.CoreV1().Nodes().Patch(
|
||||||
|
ctx, in.Node, types.MergePatchType, []byte(patch), metav1.PatchOptions{}); err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
if in.Schedulable || !in.Drain {
|
||||||
|
return &plane.Cordoned{}, nil
|
||||||
|
}
|
||||||
|
pods, err := b.Typed.CoreV1().Pods(metav1.NamespaceAll).List(ctx, metav1.ListOptions{
|
||||||
|
FieldSelector: "spec.nodeName=" + in.Node,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
evicted := 0
|
||||||
|
for i := range pods.Items {
|
||||||
|
p := pods.Items[i]
|
||||||
|
if skipEviction(p) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
ev := &policyv1.Eviction{ObjectMeta: metav1.ObjectMeta{Namespace: p.Namespace, Name: p.Name}}
|
||||||
|
if err := b.Typed.CoreV1().Pods(p.Namespace).EvictV1(ctx, ev); err != nil {
|
||||||
|
if apierrors.IsNotFound(err) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
// Report the count achieved so far alongside the refusal: an operator
|
||||||
|
// needs to know how far the drain got, not just that it stopped.
|
||||||
|
return &plane.Cordoned{Evicted: evicted}, fail(fmt.Errorf("evict %s/%s: %w", p.Namespace, p.Name, err))
|
||||||
|
}
|
||||||
|
evicted++
|
||||||
|
}
|
||||||
|
return &plane.Cordoned{Evicted: evicted}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// nodeVolumeStats reports how full every mounted volume on a cluster is.
|
||||||
|
//
|
||||||
|
// It is the ONLY source of volume fill we have. Cloud providers do not expose how
|
||||||
|
// full a disk is, and metrics-server is absent from most of our clusters — but every
|
||||||
|
// kubelet serves `stats/summary` unconditionally, so this works everywhere. The read
|
||||||
|
// goes through the apiserver's node proxy, which is why this is a SEPARATE named op:
|
||||||
|
// `nodes/proxy get` is effectively arbitrary kubelet access, and naming it here is
|
||||||
|
// what lets every other operation's grant omit it.
|
||||||
|
//
|
||||||
|
// A kubelet that will not answer costs a reading and never the call: fill is
|
||||||
|
// advisory, no safety verdict depends on it, and failing the whole sweep over one
|
||||||
|
// silent node would block every volume decision on the fleet. `nodesRead` against
|
||||||
|
// `nodesTotal` is how a caller knows the difference between "empty" and
|
||||||
|
// "unmeasured" — a claim with no row is unmeasured, and treating that as empty is
|
||||||
|
// how live data gets condemned.
|
||||||
|
//
|
||||||
|
// Response: {"volumes":[{"namespace":"acme","name":"data","usedBytes":48318382080}],"nodesRead":6,"nodesTotal":6}
|
||||||
|
func (o Ops) nodeVolumeStats(ctx context.Context, in *plane.ClusterRef) (*plane.VolumeStats, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := b.WholeCluster(); err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
nodes, err := b.Typed.CoreV1().Nodes().List(ctx, metav1.ListOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
var (
|
||||||
|
mu sync.Mutex
|
||||||
|
wg sync.WaitGroup
|
||||||
|
used = map[[2]string]int64{}
|
||||||
|
read int
|
||||||
|
sem = make(chan struct{}, kubeletFanout)
|
||||||
|
)
|
||||||
|
for i := range nodes.Items {
|
||||||
|
node := nodes.Items[i].Name
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
sem <- struct{}{}
|
||||||
|
defer func() { <-sem }()
|
||||||
|
raw, err := b.Typed.CoreV1().RESTClient().Get().
|
||||||
|
Resource("nodes").Name(node).SubResource("proxy").
|
||||||
|
Suffix("stats", "summary").DoRaw(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var sum struct {
|
||||||
|
Pods []struct {
|
||||||
|
Volume []struct {
|
||||||
|
PVCRef *struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
} `json:"pvcRef"`
|
||||||
|
UsedBytes int64 `json:"usedBytes"`
|
||||||
|
} `json:"volume"`
|
||||||
|
} `json:"pods"`
|
||||||
|
}
|
||||||
|
if json.Unmarshal(raw, &sum) != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
mu.Lock()
|
||||||
|
defer mu.Unlock()
|
||||||
|
read++
|
||||||
|
for _, p := range sum.Pods {
|
||||||
|
for _, v := range p.Volume {
|
||||||
|
if v.PVCRef == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
k := [2]string{v.PVCRef.Namespace, v.PVCRef.Name}
|
||||||
|
// Two kubelets can report one claim — ReadWriteMany, or a
|
||||||
|
// rollout with both pods briefly alive. Keep the LARGEST
|
||||||
|
// reading: more used means less waste claimed, which is the
|
||||||
|
// conservative direction for every decision downstream.
|
||||||
|
if v.UsedBytes > used[k] {
|
||||||
|
used[k] = v.UsedBytes
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
out := make([]plane.VolumeUse, 0, len(used))
|
||||||
|
for k, bytes := range used {
|
||||||
|
out = append(out, plane.VolumeUse{Namespace: k[0], Name: k[1], UsedBytes: bytes})
|
||||||
|
}
|
||||||
|
// Map order is not stable and this answer is: a reproducible sweep is worth one
|
||||||
|
// sort, and a diff between two sweeps is then a real change.
|
||||||
|
sort.Slice(out, func(i, j int) bool {
|
||||||
|
if out[i].Namespace != out[j].Namespace {
|
||||||
|
return out[i].Namespace < out[j].Namespace
|
||||||
|
}
|
||||||
|
return out[i].Name < out[j].Name
|
||||||
|
})
|
||||||
|
return &plane.VolumeStats{Volumes: out, NodesRead: read, NodesTotal: len(nodes.Items)}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// nodeView projects one node onto the fields the boards read.
|
||||||
|
func nodeView(n corev1.Node) plane.Node {
|
||||||
|
v := plane.Node{
|
||||||
|
Name: n.Name,
|
||||||
|
Ready: nodeReady(n),
|
||||||
|
Schedulable: !n.Spec.Unschedulable,
|
||||||
|
Region: n.Labels[corev1.LabelTopologyRegion],
|
||||||
|
InstanceKind: n.Labels[corev1.LabelInstanceTypeStable],
|
||||||
|
}
|
||||||
|
if q, ok := n.Status.Allocatable[corev1.ResourceCPU]; ok {
|
||||||
|
v.CPU = q.String()
|
||||||
|
}
|
||||||
|
if q, ok := n.Status.Allocatable[corev1.ResourceMemory]; ok {
|
||||||
|
v.Memory = q.String()
|
||||||
|
}
|
||||||
|
if q, ok := n.Status.Allocatable["nvidia.com/gpu"]; ok {
|
||||||
|
v.NvidiaGPU = registry.QuantityInt(q.String())
|
||||||
|
}
|
||||||
|
if q, ok := n.Status.Allocatable["amd.com/gpu"]; ok {
|
||||||
|
v.AmdGPU = registry.QuantityInt(q.String())
|
||||||
|
}
|
||||||
|
for _, a := range n.Status.Addresses {
|
||||||
|
if a.Type == corev1.NodeInternalIP {
|
||||||
|
v.InternalIP = a.Address
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
// nodeReady reads the one condition that decides whether a node is up.
|
||||||
|
func nodeReady(n corev1.Node) bool {
|
||||||
|
for _, c := range n.Status.Conditions {
|
||||||
|
if c.Type == corev1.NodeReady {
|
||||||
|
return c.Status == corev1.ConditionTrue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// skipEviction reports pods a drain must leave alone: DaemonSet-owned and static
|
||||||
|
// (mirror) pods, which the node recreates immediately, and pods already terminal.
|
||||||
|
func skipEviction(p corev1.Pod) bool {
|
||||||
|
if _, mirror := p.Annotations[corev1.MirrorPodAnnotationKey]; mirror {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
for _, ref := range p.OwnerReferences {
|
||||||
|
if ref.Kind == "DaemonSet" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return p.Status.Phase == corev1.PodSucceeded || p.Status.Phase == corev1.PodFailed
|
||||||
|
}
|
||||||
+332
@@ -0,0 +1,332 @@
|
|||||||
|
// Package ops is the /v1/k8s surface: one named, typed operation per thing this
|
||||||
|
// deployment may do to a Kubernetes cluster, and nothing else.
|
||||||
|
//
|
||||||
|
// # Every op names its cluster, and Reach is the only door
|
||||||
|
//
|
||||||
|
// Not one handler here constructs a Kubernetes client. Each one calls
|
||||||
|
// registry.Registry.Reach with the cluster the caller named, and Reach is the
|
||||||
|
// module's only client constructor — so "which cluster" cannot be omitted, cannot
|
||||||
|
// be defaulted, and cannot be inherited from the pod this process happens to run
|
||||||
|
// in. hanzo-k8s, lux-k8s, zoo-k8s and a customer's own cluster are four rows in a
|
||||||
|
// registry and one code path.
|
||||||
|
//
|
||||||
|
// # There is no generic passthrough, and that is the point
|
||||||
|
//
|
||||||
|
// Every operation is NAMED and TYPED, including the custom-resource quartet, whose
|
||||||
|
// kinds come from the closed table in resources.go. An "apply this YAML" op would
|
||||||
|
// have been shorter to write and would have relocated the credential problem
|
||||||
|
// behind an RPC instead of solving it: with one, the union of privilege this app
|
||||||
|
// needs cannot be written down, so nothing can be least-privileged and no reviewer
|
||||||
|
// can bound a blast radius. With this set, the union RBAC is a finite list and it
|
||||||
|
// is in the repo's LLM.md.
|
||||||
|
//
|
||||||
|
// # Every op is registered twice, from one handler
|
||||||
|
//
|
||||||
|
// Once on the app — the world's HTTP surface, which projects OpenAPI, an MCP tool
|
||||||
|
// and a CLI command — and once on app.Peer(), the fleet's socket, where a peer
|
||||||
|
// invokes it by op token through zip.Ask. One handler, one token, two transports.
|
||||||
|
// The peer app listens only on this app's 0600 socket, so an internal op is never
|
||||||
|
// served on the edge; that separation is structural rather than a check that could
|
||||||
|
// be forgotten.
|
||||||
|
//
|
||||||
|
// The registrations are written out as direct zip.Get / zip.Post calls with literal
|
||||||
|
// paths, deliberately and not by a helper: zipdoc reads the AST for exactly that
|
||||||
|
// shape, so a registration wrapped in a loop or a generic helper is a registration
|
||||||
|
// whose doc comment reaches NEITHER the OpenAPI description NOR the MCP tool
|
||||||
|
// description. The prose below each handler is the product surface an agent reads
|
||||||
|
// when deciding whether to call the op, so the shape that carries it wins over the
|
||||||
|
// shape that saves lines.
|
||||||
|
package ops
|
||||||
|
|
||||||
|
//go:generate go run github.com/zap-proto/zip/cmd/zipdoc
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
"github.com/hanzoai/k8s/registry"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Ops is the service: the registry, and nothing else. Every fact an op needs
|
||||||
|
// beyond its argument comes through Reach, so there is no second source of
|
||||||
|
// cluster state to keep in step.
|
||||||
|
type Ops struct {
|
||||||
|
Reg *registry.Registry
|
||||||
|
}
|
||||||
|
|
||||||
|
// Mount registers every operation on both surfaces.
|
||||||
|
//
|
||||||
|
// The order is the order the API documentation reads in, not a routing order:
|
||||||
|
// zip's router decides by specificity, so mount sequence carries no meaning and a
|
||||||
|
// reader is free to find an op where they expect it.
|
||||||
|
func (o Ops) Mount(app *zip.App) {
|
||||||
|
p := app.Peer()
|
||||||
|
|
||||||
|
// ---- clusters — the registry ------------------------------------------
|
||||||
|
zip.Get(app, "/v1/k8s/clusters", o.listClusters,
|
||||||
|
zip.WithOperationID(plane.ClustersList), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/clusters/list", o.listClusters,
|
||||||
|
zip.WithOperationID(plane.ClustersList))
|
||||||
|
|
||||||
|
zip.Post(app, "/v1/k8s/clusters", o.registerCluster,
|
||||||
|
zip.WithOperationID(plane.ClustersRegister), zip.WithTags("k8s"), zip.WithStatus(201))
|
||||||
|
zip.Post(p, "/k8s/clusters/register", o.registerCluster,
|
||||||
|
zip.WithOperationID(plane.ClustersRegister))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/clusters/:cluster", o.getCluster,
|
||||||
|
zip.WithOperationID(plane.ClustersGet), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/clusters/get", o.getCluster,
|
||||||
|
zip.WithOperationID(plane.ClustersGet))
|
||||||
|
|
||||||
|
zip.Delete(app, "/v1/k8s/clusters/:cluster", o.deregisterCluster,
|
||||||
|
zip.WithOperationID(plane.ClustersDeregister), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/clusters/deregister", o.deregisterCluster,
|
||||||
|
zip.WithOperationID(plane.ClustersDeregister))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/clusters/:cluster/status", o.clusterStatus,
|
||||||
|
zip.WithOperationID(plane.ClustersStatus), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/clusters/status", o.clusterStatus,
|
||||||
|
zip.WithOperationID(plane.ClustersStatus))
|
||||||
|
|
||||||
|
// ---- nodes -------------------------------------------------------------
|
||||||
|
zip.Get(app, "/v1/k8s/nodes", o.listNodes,
|
||||||
|
zip.WithOperationID(plane.NodesList), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/nodes/list", o.listNodes,
|
||||||
|
zip.WithOperationID(plane.NodesList))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/nodes/volume-stats", o.nodeVolumeStats,
|
||||||
|
zip.WithOperationID(plane.NodesVolumeStats), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/nodes/volume-stats", o.nodeVolumeStats,
|
||||||
|
zip.WithOperationID(plane.NodesVolumeStats))
|
||||||
|
|
||||||
|
zip.Post(app, "/v1/k8s/nodes/:node/cordon", o.cordonNode,
|
||||||
|
zip.WithOperationID(plane.NodesCordon), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/nodes/cordon", o.cordonNode,
|
||||||
|
zip.WithOperationID(plane.NodesCordon))
|
||||||
|
|
||||||
|
// ---- namespaces --------------------------------------------------------
|
||||||
|
zip.Get(app, "/v1/k8s/namespaces", o.listNamespaces,
|
||||||
|
zip.WithOperationID(plane.NamespacesList), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/namespaces/list", o.listNamespaces,
|
||||||
|
zip.WithOperationID(plane.NamespacesList))
|
||||||
|
|
||||||
|
zip.Post(app, "/v1/k8s/namespaces", o.ensureNamespace,
|
||||||
|
zip.WithOperationID(plane.NamespacesEnsure), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/namespaces/ensure", o.ensureNamespace,
|
||||||
|
zip.WithOperationID(plane.NamespacesEnsure))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/namespaces/:namespace", o.getNamespace,
|
||||||
|
zip.WithOperationID(plane.NamespacesGet), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/namespaces/get", o.getNamespace,
|
||||||
|
zip.WithOperationID(plane.NamespacesGet))
|
||||||
|
|
||||||
|
zip.Delete(app, "/v1/k8s/namespaces/:namespace", o.deleteNamespace,
|
||||||
|
zip.WithOperationID(plane.NamespacesDelete), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/namespaces/delete", o.deleteNamespace,
|
||||||
|
zip.WithOperationID(plane.NamespacesDelete))
|
||||||
|
|
||||||
|
// ---- pods --------------------------------------------------------------
|
||||||
|
zip.Get(app, "/v1/k8s/pods", o.listPods,
|
||||||
|
zip.WithOperationID(plane.PodsList), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/pods/list", o.listPods,
|
||||||
|
zip.WithOperationID(plane.PodsList))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/pods/:namespace/:pod/logs", o.podLogs,
|
||||||
|
zip.WithOperationID(plane.PodsLogs), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/pods/logs", o.podLogs,
|
||||||
|
zip.WithOperationID(plane.PodsLogs))
|
||||||
|
|
||||||
|
// ---- volumes -----------------------------------------------------------
|
||||||
|
zip.Get(app, "/v1/k8s/volumes/claims", o.listClaims,
|
||||||
|
zip.WithOperationID(plane.VolumesListClaims), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/volumes/list-claims", o.listClaims,
|
||||||
|
zip.WithOperationID(plane.VolumesListClaims))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/volumes/persistent", o.listPersistentVolumes,
|
||||||
|
zip.WithOperationID(plane.VolumesListPersistent), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/volumes/list-persistent", o.listPersistentVolumes,
|
||||||
|
zip.WithOperationID(plane.VolumesListPersistent))
|
||||||
|
|
||||||
|
zip.Post(app, "/v1/k8s/volumes/claims", o.ensureClaim,
|
||||||
|
zip.WithOperationID(plane.VolumesEnsureClaim), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/volumes/ensure-claim", o.ensureClaim,
|
||||||
|
zip.WithOperationID(plane.VolumesEnsureClaim))
|
||||||
|
|
||||||
|
zip.Post(app, "/v1/k8s/volumes/claims/:namespace/:name/expand", o.expandClaim,
|
||||||
|
zip.WithOperationID(plane.VolumesExpandClaim), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/volumes/expand-claim", o.expandClaim,
|
||||||
|
zip.WithOperationID(plane.VolumesExpandClaim))
|
||||||
|
|
||||||
|
// ---- workloads ---------------------------------------------------------
|
||||||
|
zip.Get(app, "/v1/k8s/workloads/deployments", o.listDeployments,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListDeployments), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/workloads/list-deployments", o.listDeployments,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListDeployments))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/workloads/deployments/:namespace/:name", o.getDeployment,
|
||||||
|
zip.WithOperationID(plane.WorkloadsGetDeployment), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/workloads/get-deployment", o.getDeployment,
|
||||||
|
zip.WithOperationID(plane.WorkloadsGetDeployment))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/workloads/replicasets", o.listReplicaSets,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListReplicaSets), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/workloads/list-replicasets", o.listReplicaSets,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListReplicaSets))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/workloads/statefulsets", o.listStatefulSets,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListStatefulSets), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/workloads/list-statefulsets", o.listStatefulSets,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListStatefulSets))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/workloads/services", o.listServices,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListServices), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/workloads/list-services", o.listServices,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListServices))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/workloads/ingresses", o.listIngresses,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListIngresses), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/workloads/list-ingresses", o.listIngresses,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListIngresses))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/workloads/events", o.listEvents,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListEvents), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/workloads/list-events", o.listEvents,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListEvents))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/workloads/configmaps", o.listConfigMaps,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListConfigMaps), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/workloads/list-configmaps", o.listConfigMaps,
|
||||||
|
zip.WithOperationID(plane.WorkloadsListConfigMaps))
|
||||||
|
|
||||||
|
// ---- jobs --------------------------------------------------------------
|
||||||
|
zip.Post(app, "/v1/k8s/jobs", o.createJob,
|
||||||
|
zip.WithOperationID(plane.JobsCreate), zip.WithTags("k8s"), zip.WithStatus(201))
|
||||||
|
zip.Post(p, "/k8s/jobs/create", o.createJob,
|
||||||
|
zip.WithOperationID(plane.JobsCreate))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/jobs", o.listJobs,
|
||||||
|
zip.WithOperationID(plane.JobsList), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/jobs/list", o.listJobs,
|
||||||
|
zip.WithOperationID(plane.JobsList))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/jobs/:namespace/:name", o.getJob,
|
||||||
|
zip.WithOperationID(plane.JobsGet), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/jobs/get", o.getJob,
|
||||||
|
zip.WithOperationID(plane.JobsGet))
|
||||||
|
|
||||||
|
// ---- resources — the closed kind set -----------------------------------
|
||||||
|
zip.Get(app, "/v1/k8s/resources", o.listResources,
|
||||||
|
zip.WithOperationID(plane.ResourcesList), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/resources/list", o.listResources,
|
||||||
|
zip.WithOperationID(plane.ResourcesList))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/resources/watch", o.watchResources,
|
||||||
|
zip.WithOperationID(plane.ResourcesWatch), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/resources/watch", o.watchResources,
|
||||||
|
zip.WithOperationID(plane.ResourcesWatch))
|
||||||
|
|
||||||
|
zip.Put(app, "/v1/k8s/resources", o.applyResource,
|
||||||
|
zip.WithOperationID(plane.ResourcesApply), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/resources/apply", o.applyResource,
|
||||||
|
zip.WithOperationID(plane.ResourcesApply))
|
||||||
|
|
||||||
|
zip.Get(app, "/v1/k8s/resources/:kind/:namespace/:name", o.getResource,
|
||||||
|
zip.WithOperationID(plane.ResourcesGet), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/resources/get", o.getResource,
|
||||||
|
zip.WithOperationID(plane.ResourcesGet))
|
||||||
|
|
||||||
|
zip.Delete(app, "/v1/k8s/resources/:kind/:namespace/:name", o.deleteResource,
|
||||||
|
zip.WithOperationID(plane.ResourcesDelete), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/resources/delete", o.deleteResource,
|
||||||
|
zip.WithOperationID(plane.ResourcesDelete))
|
||||||
|
|
||||||
|
// ---- access ------------------------------------------------------------
|
||||||
|
zip.Post(app, "/v1/k8s/access/can", o.can,
|
||||||
|
zip.WithOperationID(plane.AccessCan), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/access/can", o.can,
|
||||||
|
zip.WithOperationID(plane.AccessCan))
|
||||||
|
|
||||||
|
// ---- metrics -----------------------------------------------------------
|
||||||
|
zip.Get(app, "/v1/k8s/metrics/pods", o.podMetrics,
|
||||||
|
zip.WithOperationID(plane.MetricsPods), zip.WithTags("k8s"))
|
||||||
|
zip.Post(p, "/k8s/metrics/pods", o.podMetrics,
|
||||||
|
zip.WithOperationID(plane.MetricsPods))
|
||||||
|
}
|
||||||
|
|
||||||
|
// reach resolves a cluster for the caller and maps a refusal onto its HTTP
|
||||||
|
// status. It is the one line every handler starts with, and the one place a
|
||||||
|
// registry refusal becomes a wire answer.
|
||||||
|
func (o Ops) reach(ctx context.Context, cluster string) (*registry.Bound, error) {
|
||||||
|
b, err := o.Reg.Reach(ctx, cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
return b, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// fail maps an error onto the status a caller should see.
|
||||||
|
//
|
||||||
|
// The mapping is deliberate about ONE thing above all: a cluster another org
|
||||||
|
// registered answers 404, never 403. A "forbidden" would confirm the name exists
|
||||||
|
// and turn the refusal into an oracle for enumerating other tenants' clusters.
|
||||||
|
// A namespace outside a registration's bound DOES answer 403, because the caller
|
||||||
|
// already knows that namespace name — it typed it — so there is nothing to leak
|
||||||
|
// and everything to explain.
|
||||||
|
func fail(err error) error {
|
||||||
|
if err == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var noTenant registry.ErrNoTenant
|
||||||
|
if errors.As(err, &noTenant) {
|
||||||
|
return zip.ErrForbidden(err.Error())
|
||||||
|
}
|
||||||
|
var noCluster registry.ErrNoCluster
|
||||||
|
if errors.As(err, &noCluster) {
|
||||||
|
return zip.ErrNotFound(err.Error())
|
||||||
|
}
|
||||||
|
var badNS registry.ErrNamespace
|
||||||
|
if errors.As(err, &badNS) {
|
||||||
|
return zip.ErrForbidden(err.Error())
|
||||||
|
}
|
||||||
|
var badScope registry.ErrScope
|
||||||
|
if errors.As(err, &badScope) {
|
||||||
|
return zip.ErrForbidden(err.Error())
|
||||||
|
}
|
||||||
|
var badName registry.ErrName
|
||||||
|
if errors.As(err, &badName) {
|
||||||
|
return zip.ErrBadRequest(err.Error())
|
||||||
|
}
|
||||||
|
// An apiserver's own verdict is surfaced with its own status and its own
|
||||||
|
// message. A RBAC refusal read as an internal error would send an operator
|
||||||
|
// hunting a bug in this app instead of granting the verb it named.
|
||||||
|
switch {
|
||||||
|
case apierrors.IsNotFound(err):
|
||||||
|
return zip.ErrNotFound(err.Error())
|
||||||
|
case apierrors.IsForbidden(err), apierrors.IsUnauthorized(err):
|
||||||
|
return zip.ErrForbidden(err.Error())
|
||||||
|
case apierrors.IsConflict(err), apierrors.IsAlreadyExists(err):
|
||||||
|
return zip.ErrConflict(err.Error())
|
||||||
|
case apierrors.IsInvalid(err), apierrors.IsBadRequest(err):
|
||||||
|
return zip.ErrBadRequest(err.Error())
|
||||||
|
}
|
||||||
|
var he *zip.HTTPError
|
||||||
|
if errors.As(err, &he) {
|
||||||
|
return he
|
||||||
|
}
|
||||||
|
return zip.ErrInternal(err.Error())
|
||||||
|
}
|
||||||
|
|
||||||
|
// need refuses an empty required argument. zip's `validate:"required"` covers the
|
||||||
|
// HTTP body; a peer calling over ZAP hands the struct straight in, so the check
|
||||||
|
// belongs in the handler as well as in the tag.
|
||||||
|
func need(field, value string) error {
|
||||||
|
if value == "" {
|
||||||
|
return zip.ErrBadRequest(fmt.Sprintf("%q is required", field))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
+260
@@ -0,0 +1,260 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
"github.com/hanzoai/k8s/registry"
|
||||||
|
)
|
||||||
|
|
||||||
|
// This suite drives the REAL router, because the guards are only worth what the wire
|
||||||
|
// enforces. A handler that refuses correctly behind a route nothing reaches is a guard
|
||||||
|
// in no path, which is the failure mode §8.9 of the plugin contract exists to catch.
|
||||||
|
|
||||||
|
// store is an in-memory registry store, seeded by org.
|
||||||
|
type store struct{ data map[string][]byte }
|
||||||
|
|
||||||
|
func (s store) Get(_ context.Context, ref string) ([]byte, error) { return s.data[ref], nil }
|
||||||
|
func (s store) Put(_ context.Context, ref string, v []byte) error {
|
||||||
|
if v == nil {
|
||||||
|
delete(s.data, ref)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
s.data[ref] = v
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// seeded builds an app whose registry holds one cluster per org given.
|
||||||
|
func seeded(t *testing.T, rows map[string][]registry.Cluster) *zip.App {
|
||||||
|
t.Helper()
|
||||||
|
t.Setenv(registry.AllowPrivateHostsEnv, "1")
|
||||||
|
s := store{data: map[string][]byte{}}
|
||||||
|
for org, list := range rows {
|
||||||
|
for i := range list {
|
||||||
|
list[i].Org = org
|
||||||
|
if list[i].Endpoint == "" {
|
||||||
|
list[i].Endpoint = "https://" + list[i].Name + "." + org + ".invalid:6443"
|
||||||
|
}
|
||||||
|
s.data["orgs/"+org+"/k8s/clusters/"+list[i].Name+"/kubeconfig"] = []byte(`apiVersion: v1
|
||||||
|
kind: Config
|
||||||
|
clusters: [{name: c, cluster: {server: ` + list[i].Endpoint + `, insecure-skip-tls-verify: true}}]
|
||||||
|
contexts: [{name: c, context: {cluster: c, user: u}}]
|
||||||
|
current-context: c
|
||||||
|
users: [{name: u, user: {token: t}}]
|
||||||
|
`)
|
||||||
|
}
|
||||||
|
raw, err := json.Marshal(list)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s.data["orgs/"+org+"/k8s/clusters"] = raw
|
||||||
|
}
|
||||||
|
app := zip.New(zip.Config{AppName: "k8s"})
|
||||||
|
Ops{Reg: registry.New(s)}.Mount(app)
|
||||||
|
if err := app.Build(); err != nil {
|
||||||
|
t.Fatalf("Build: %v", err)
|
||||||
|
}
|
||||||
|
return app
|
||||||
|
}
|
||||||
|
|
||||||
|
// ask sends one request through the composed router with the identity headers a
|
||||||
|
// gateway would have attached, and returns the status and body.
|
||||||
|
func ask(t *testing.T, app *zip.App, method, path, org string) (int, string) {
|
||||||
|
t.Helper()
|
||||||
|
req := httptest.NewRequest(method, path, nil)
|
||||||
|
if org != "" {
|
||||||
|
req.Header.Set(zip.HeaderOrg, org)
|
||||||
|
req.Header.Set(zip.HeaderUser, "u@"+org)
|
||||||
|
}
|
||||||
|
resp, err := app.Fiber().Test(req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s %s: %v", method, path, err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
body, _ := io.ReadAll(resp.Body)
|
||||||
|
return resp.StatusCode, string(body)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNoIdentityIsRefusedOnTheWire(t *testing.T) {
|
||||||
|
app := seeded(t, map[string][]registry.Cluster{"alpha": {{Name: "prod"}}})
|
||||||
|
// An X-Org-Id with no validated user beside it is a claim the client made about
|
||||||
|
// itself. This is the request an attacker actually sends.
|
||||||
|
req := httptest.NewRequest("GET", "/v1/k8s/clusters", nil)
|
||||||
|
req.Header.Set(zip.HeaderOrg, "alpha")
|
||||||
|
resp, err := app.Fiber().Test(req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != 403 {
|
||||||
|
body, _ := io.ReadAll(resp.Body)
|
||||||
|
t.Fatalf("a forged org header was served: %d %s", resp.StatusCode, body)
|
||||||
|
}
|
||||||
|
// And with nothing at all.
|
||||||
|
if code, body := ask(t, app, "GET", "/v1/k8s/clusters", ""); code != 403 {
|
||||||
|
t.Fatalf("an unauthenticated request was served: %d %s", code, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTenantSeesOnlyItsOwnClusters(t *testing.T) {
|
||||||
|
app := seeded(t, map[string][]registry.Cluster{
|
||||||
|
"alpha": {{Name: "prod"}, {Name: "hanzo-k8s"}},
|
||||||
|
"beta": {{Name: "prod"}},
|
||||||
|
})
|
||||||
|
code, body := ask(t, app, "GET", "/v1/k8s/clusters", "beta")
|
||||||
|
if code != 200 {
|
||||||
|
t.Fatalf("beta: %d %s", code, body)
|
||||||
|
}
|
||||||
|
var out plane.Clusters
|
||||||
|
if err := json.Unmarshal([]byte(body), &out); err != nil {
|
||||||
|
t.Fatalf("%v: %s", err, body)
|
||||||
|
}
|
||||||
|
if len(out.Clusters) != 1 || out.Clusters[0].Name != "prod" || out.Clusters[0].Org != "beta" {
|
||||||
|
t.Fatalf("beta's cluster list is not beta's: %s", body)
|
||||||
|
}
|
||||||
|
if strings.Contains(body, "hanzo-k8s") || strings.Contains(body, "alpha") {
|
||||||
|
t.Fatalf("beta's list leaked alpha's fleet: %s", body)
|
||||||
|
}
|
||||||
|
// The kubeconfig must not be in the answer under any spelling.
|
||||||
|
for _, leak := range []string{"kubeconfig", "apiVersion: v1", "token", "insecure"} {
|
||||||
|
if strings.Contains(body, leak) {
|
||||||
|
t.Fatalf("a cluster row carried %q: %s", leak, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestForeignClusterIs404OnTheWire(t *testing.T) {
|
||||||
|
app := seeded(t, map[string][]registry.Cluster{"alpha": {{Name: "prod"}}})
|
||||||
|
for _, path := range []string{
|
||||||
|
"/v1/k8s/clusters/prod",
|
||||||
|
"/v1/k8s/clusters/prod/status",
|
||||||
|
"/v1/k8s/nodes?cluster=prod",
|
||||||
|
"/v1/k8s/pods?cluster=prod",
|
||||||
|
"/v1/k8s/namespaces?cluster=prod",
|
||||||
|
"/v1/k8s/volumes/claims?cluster=prod",
|
||||||
|
"/v1/k8s/workloads/deployments?cluster=prod",
|
||||||
|
"/v1/k8s/jobs?cluster=prod",
|
||||||
|
"/v1/k8s/resources?cluster=prod&kind=App",
|
||||||
|
"/v1/k8s/metrics/pods?cluster=prod",
|
||||||
|
} {
|
||||||
|
code, body := ask(t, app, "GET", path, "beta")
|
||||||
|
if code != 404 {
|
||||||
|
t.Fatalf("%s answered %d for a foreign cluster (a 403 would confirm the name exists): %s", path, code, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestForeignNamespaceIs403OnTheWire(t *testing.T) {
|
||||||
|
app := seeded(t, map[string][]registry.Cluster{
|
||||||
|
"beta": {{Name: "shared", Namespaces: []string{"beta-prod"}}},
|
||||||
|
})
|
||||||
|
// beta knows alpha's namespace name. Knowing it must buy nothing.
|
||||||
|
code, body := ask(t, app, "GET", "/v1/k8s/namespaces/alpha-prod?cluster=shared", "beta")
|
||||||
|
if code != 403 {
|
||||||
|
t.Fatalf("beta read a namespace outside its bound: %d %s", code, body)
|
||||||
|
}
|
||||||
|
if !strings.Contains(body, "outside what this org's registration") {
|
||||||
|
t.Fatalf("the refusal does not explain itself: %s", body)
|
||||||
|
}
|
||||||
|
// And a cluster-scoped read on a bounded registration is refused, so a scoped
|
||||||
|
// tenant cannot see the shared cluster's nodes or PersistentVolumes.
|
||||||
|
for _, path := range []string{
|
||||||
|
"/v1/k8s/nodes?cluster=shared",
|
||||||
|
"/v1/k8s/volumes/persistent?cluster=shared",
|
||||||
|
"/v1/k8s/nodes/volume-stats?cluster=shared",
|
||||||
|
} {
|
||||||
|
if code, body := ask(t, app, "GET", path, "beta"); code != 403 {
|
||||||
|
t.Fatalf("%s answered %d for a namespace-bounded registration: %s", path, code, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUnregisteredKindIsRefused(t *testing.T) {
|
||||||
|
app := seeded(t, map[string][]registry.Cluster{"alpha": {{Name: "prod"}}})
|
||||||
|
// The kinds a passthrough would have accepted. Each must be refused by NAME, before
|
||||||
|
// any cluster is dialed, because the kind table is the bound on this app's authority.
|
||||||
|
for _, kind := range []string{"Secret", "ClusterRole", "Pod", "Deployment", "Node", "CustomResourceDefinition", ""} {
|
||||||
|
code, body := ask(t, app, "GET", "/v1/k8s/resources?cluster=prod&kind="+kind, "alpha")
|
||||||
|
if code != 400 {
|
||||||
|
t.Fatalf("kind %q answered %d — the closed kind set is not being applied: %s", kind, code, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// And the refusal tells the caller what IS addressable, so a legitimate consumer
|
||||||
|
// is not left guessing.
|
||||||
|
_, body := ask(t, app, "GET", "/v1/k8s/resources?cluster=prod&kind=Secret", "alpha")
|
||||||
|
for _, kind := range Kinds() {
|
||||||
|
if !strings.Contains(body, kind) {
|
||||||
|
t.Fatalf("the refusal omits the registered kind %q: %s", kind, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestKindSetIsClosedAndNamed(t *testing.T) {
|
||||||
|
// The set is asserted here so growing it is a visible, reviewed diff rather than a
|
||||||
|
// side effect of adding a line to a map. Every entry is one more thing the union
|
||||||
|
// ClusterRole in LLM.md has to grant.
|
||||||
|
want := []string{
|
||||||
|
"App", "AppProject", "Application", "Datastore", "InferenceService",
|
||||||
|
"IngressRoute", "KMSSecret", "LuxNetwork", "Middleware", "TrainJob",
|
||||||
|
}
|
||||||
|
got := Kinds()
|
||||||
|
if strings.Join(got, ",") != strings.Join(want, ",") {
|
||||||
|
t.Fatalf("the closed kind set changed\n got: %v\nwant: %v", got, want)
|
||||||
|
}
|
||||||
|
// A core-group kind must never appear here: a Secret or a Pod reachable through
|
||||||
|
// resources.apply would be the passthrough this table exists to prevent.
|
||||||
|
for _, kind := range got {
|
||||||
|
if g := kinds[kind]; g.Group == "" {
|
||||||
|
t.Fatalf("kind %q is in the CORE group — resources.* addresses custom resources only", kind)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNoSecretOpExists(t *testing.T) {
|
||||||
|
// The strongest statement this suite can make about secrets is that there is no
|
||||||
|
// route to one. Material reaches a namespace as a KMSSecret the operator resolves;
|
||||||
|
// nothing here reads a Secret back, so nothing here can exfiltrate one.
|
||||||
|
app := seeded(t, map[string][]registry.Cluster{"alpha": {{Name: "prod"}}})
|
||||||
|
for _, r := range app.Declaration().Routes {
|
||||||
|
if strings.Contains(strings.ToLower(r.Pattern), "secret") {
|
||||||
|
t.Fatalf("a route addresses secrets: %s %s", r.Method, r.Pattern)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, op := range app.Declaration().Ops {
|
||||||
|
if strings.Contains(op, "secret") {
|
||||||
|
t.Fatalf("an op addresses secrets: %s", op)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFailMapsRefusalsToStatuses(t *testing.T) {
|
||||||
|
// The mapping is the wire contract for a refusal, so it is asserted directly as
|
||||||
|
// well as through the routes above.
|
||||||
|
for _, tc := range []struct {
|
||||||
|
err error
|
||||||
|
want int
|
||||||
|
}{
|
||||||
|
{registry.ErrNoTenant{}, 403},
|
||||||
|
{registry.ErrNoCluster{Name: "prod"}, 404},
|
||||||
|
{registry.ErrNamespace{Cluster: "c", Namespace: "n"}, 403},
|
||||||
|
{registry.ErrScope{Cluster: "c"}, 403},
|
||||||
|
{registry.ErrName{Name: "../x"}, 400},
|
||||||
|
{errors.New("boom"), 500},
|
||||||
|
} {
|
||||||
|
var he *zip.HTTPError
|
||||||
|
if !errors.As(fail(tc.err), &he) {
|
||||||
|
t.Fatalf("%T did not map to an HTTP error", tc.err)
|
||||||
|
}
|
||||||
|
if he.Status != tc.want {
|
||||||
|
t.Fatalf("%T mapped to %d, want %d", tc.err, he.Status, tc.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+128
@@ -0,0 +1,128 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
|
||||||
|
corev1 "k8s.io/api/core/v1"
|
||||||
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// maxLogBytes caps one log read. A container's log is unbounded by nature, and an
|
||||||
|
// answer that has to fit in one ZAP message needs a bound that is this app's and not
|
||||||
|
// the caller's.
|
||||||
|
const maxLogBytes = 1 << 20
|
||||||
|
|
||||||
|
// listPods returns pods, narrowed by namespace, labels or node.
|
||||||
|
//
|
||||||
|
// One op covers every way the fleet asks for pods: the whole-cluster sweep an
|
||||||
|
// infrastructure board takes, the label-selected set that answers "is this app's
|
||||||
|
// workload up", and the single-node set a drain needs. They were three call sites
|
||||||
|
// with three shapes; the difference between them is which argument is set.
|
||||||
|
//
|
||||||
|
// Container ENV is deliberately absent from the projection. A pod spec carries the
|
||||||
|
// secret material injected into it, so a read surface that returned the spec would be
|
||||||
|
// an exfiltration door wearing the name of a health check.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","namespace":"acme-prod","labelSelector":"app=api"}
|
||||||
|
// Response: {"pods":[{"namespace":"acme-prod","name":"api-7d9f-2xk","phase":"Running","node":"pool-a1","controller":"ReplicaSet/api-7d9f","images":["registry.hanzo.ai/acme/api:1.4.2"],"restarts":0}]}
|
||||||
|
func (o Ops) listPods(ctx context.Context, in *plane.PodsIn) (*plane.Pods, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
opts := metav1.ListOptions{LabelSelector: in.LabelSelector}
|
||||||
|
if in.NodeName != "" {
|
||||||
|
opts.FieldSelector = "spec.nodeName=" + in.NodeName
|
||||||
|
}
|
||||||
|
out := []plane.Pod{}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Typed.CoreV1().Pods(ns).List(ctx, opts)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
out = append(out, podView(list.Items[i]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &plane.Pods{Pods: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// podLogs returns what one container printed.
|
||||||
|
//
|
||||||
|
// `tailLines` is the bound a caller should set; without one the read is capped at one
|
||||||
|
// mebibyte and truncated from the FRONT, so the newest output — the part that
|
||||||
|
// explains a crash — is what survives.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","namespace":"acme-prod","pod":"api-7d9f-2xk","tailLines":200}
|
||||||
|
// Response: {"log":"2026-07-30T00:00:01Z listening on :8000\n"}
|
||||||
|
func (o Ops) podLogs(ctx context.Context, in *plane.LogsIn) (*plane.Logs, error) {
|
||||||
|
if err := need("pod", in.Pod); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
opts := &corev1.PodLogOptions{Container: in.Container}
|
||||||
|
if in.TailLines > 0 {
|
||||||
|
tail := in.TailLines
|
||||||
|
opts.TailLines = &tail
|
||||||
|
}
|
||||||
|
stream, err := b.Typed.CoreV1().Pods(ns).GetLogs(in.Pod, opts).Stream(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
defer stream.Close()
|
||||||
|
raw, err := io.ReadAll(io.LimitReader(stream, maxLogBytes))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
return &plane.Logs{Log: string(raw)}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// podView projects one pod onto placement, health, what it mounts and what it runs.
|
||||||
|
func podView(p corev1.Pod) plane.Pod {
|
||||||
|
v := plane.Pod{
|
||||||
|
Namespace: p.Namespace, Name: p.Name,
|
||||||
|
Phase: string(p.Status.Phase), Reason: p.Status.Reason, Node: p.Spec.NodeName,
|
||||||
|
}
|
||||||
|
for _, vol := range p.Spec.Volumes {
|
||||||
|
if vol.PersistentVolumeClaim != nil {
|
||||||
|
v.Claims = append(v.Claims, vol.PersistentVolumeClaim.ClaimName)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The controlling owner names the workload that has to be edited to change this
|
||||||
|
// pod, and its KIND decides how — a StatefulSet's volumeClaimTemplates are
|
||||||
|
// immutable, so the workload must be recreated around a change rather than patched.
|
||||||
|
for _, ref := range p.OwnerReferences {
|
||||||
|
if ref.Controller != nil && *ref.Controller {
|
||||||
|
v.Controller = ref.Kind + "/" + ref.Name
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, c := range p.Spec.InitContainers {
|
||||||
|
v.Images = append(v.Images, c.Image)
|
||||||
|
}
|
||||||
|
for _, c := range p.Spec.Containers {
|
||||||
|
v.Images = append(v.Images, c.Image)
|
||||||
|
}
|
||||||
|
for _, cs := range p.Status.ContainerStatuses {
|
||||||
|
v.Restarts += cs.RestartCount
|
||||||
|
// A waiting container's reason (CrashLoopBackOff, ImagePullBackOff) is the
|
||||||
|
// real health signal; pod.status.reason stays empty for exactly those.
|
||||||
|
if v.Reason == "" && cs.State.Waiting != nil && cs.State.Waiting.Reason != "" {
|
||||||
|
v.Reason = cs.State.Waiting.Reason
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
@@ -0,0 +1,364 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||||
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||||
|
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
|
||||||
|
"k8s.io/apimachinery/pkg/runtime/schema"
|
||||||
|
"k8s.io/apimachinery/pkg/watch"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// kinds is the CLOSED set of custom resources this app addresses.
|
||||||
|
//
|
||||||
|
// This table is the whole difference between a bounded surface and a passthrough.
|
||||||
|
// The function it replaces decoded arbitrary YAML, resolved its kind through a
|
||||||
|
// discovery RESTMapper and create-or-updated ANY kind the cluster served — so one
|
||||||
|
// edited database row reached any object on any cluster, and no ClusterRole could
|
||||||
|
// be written that was smaller than "everything". Here the kind is a key in a map
|
||||||
|
// that only a code change can grow, which is what makes the union of privilege in
|
||||||
|
// LLM.md a finite list a reviewer can read.
|
||||||
|
//
|
||||||
|
// A group is never taken from a caller. `LuxNetwork` is the clearest case: cloud's
|
||||||
|
// validator path made the group configurable and then had to add a runtime guard
|
||||||
|
// refusing the two legacy groups that reconcile against a live hand-managed
|
||||||
|
// StatefulSet. Pinning the group here means those groups cannot be named at all,
|
||||||
|
// so the guard is a property of the table rather than a check that could be
|
||||||
|
// deleted.
|
||||||
|
//
|
||||||
|
// Every kind here is namespaced. That is not an assumption: it is why there is no
|
||||||
|
// cluster-scoped branch below, and adding a cluster-scoped kind requires adding
|
||||||
|
// one — deliberately, because a cluster-scoped write is a different blast radius.
|
||||||
|
var kinds = map[string]schema.GroupVersionResource{
|
||||||
|
// The operator's App CR — the ONE workload kind the fleet runs on.
|
||||||
|
"App": {Group: "hanzo.ai", Version: "v1", Resource: "apps"},
|
||||||
|
// Hanzo CD's own record of a tracked git source: the revision it last applied,
|
||||||
|
// its sync verdict, its deploy history. A different fact from an App.
|
||||||
|
"Application": {Group: "apps.hanzo.ai", Version: "v1alpha1", Resource: "applications"},
|
||||||
|
// The GitOps plane's project grouping.
|
||||||
|
"AppProject": {Group: "argoproj.io", Version: "v1alpha1", Resource: "appprojects"},
|
||||||
|
// A dedicated datastore the provisioning path materialises.
|
||||||
|
"Datastore": {Group: "hanzo.ai", Version: "v1", Resource: "datastores"},
|
||||||
|
// Ingress routing and its middleware chain.
|
||||||
|
"IngressRoute": {Group: "hanzo.ai", Version: "v1alpha1", Resource: "ingressroutes"},
|
||||||
|
"Middleware": {Group: "hanzo.ai", Version: "v1alpha1", Resource: "middlewares"},
|
||||||
|
// The validator node CR, under the DEDICATED group. Never "lux.network" and
|
||||||
|
// never "lux.cloud" — both reconcile against the live hand-managed StatefulSet.
|
||||||
|
"LuxNetwork": {Group: "node.lux.cloud", Version: "v1", Resource: "luxnetworks"},
|
||||||
|
// Training and serving.
|
||||||
|
"TrainJob": {Group: "trainer.kubeflow.org", Version: "v1alpha1", Resource: "trainjobs"},
|
||||||
|
"InferenceService": {Group: "serving.kserve.io", Version: "v1beta1", Resource: "inferenceservices"},
|
||||||
|
// The KMS→Secret bridge. This is how material reaches a namespace: a KMSSecret
|
||||||
|
// names a KMS ref and the kms-operator resolves it into a mounted Secret inside
|
||||||
|
// the cluster. This app therefore never reads or writes a core Secret — there
|
||||||
|
// is no op that does, because an RPC that returns a tenant's Secret is a
|
||||||
|
// credential-exfiltration endpoint however carefully it is gated.
|
||||||
|
"KMSSecret": {Group: "secrets.lux.network", Version: "v1alpha1", Resource: "kmssecrets"},
|
||||||
|
}
|
||||||
|
|
||||||
|
// maxWatch bounds one watch call. A caller cannot pin a connection open past this,
|
||||||
|
// and the bound is this app's rather than the caller's for exactly that reason.
|
||||||
|
const maxWatch = 55 * time.Second
|
||||||
|
|
||||||
|
// Kinds returns the registered kind names, sorted. Exported so a gate can assert the
|
||||||
|
// set is what the documentation says it is.
|
||||||
|
func Kinds() []string {
|
||||||
|
out := make([]string, 0, len(kinds))
|
||||||
|
for k := range kinds {
|
||||||
|
out = append(out, k)
|
||||||
|
}
|
||||||
|
sort.Strings(out)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// gvr resolves a kind name against the closed table.
|
||||||
|
func gvr(kind string) (schema.GroupVersionResource, error) {
|
||||||
|
g, ok := kinds[strings.TrimSpace(kind)]
|
||||||
|
if !ok {
|
||||||
|
return schema.GroupVersionResource{}, zip.ErrBadRequest(fmt.Sprintf(
|
||||||
|
"kind %q is not one this API addresses; the registered kinds are %s",
|
||||||
|
kind, strings.Join(Kinds(), ", ")))
|
||||||
|
}
|
||||||
|
return g, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// listResources returns custom resources of one registered kind.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","kind":"App","namespace":"acme-prod"}
|
||||||
|
// Response: {"resources":[{"kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2"},"status":{"phase":"Running"},"version":"918273"}],"version":"918273"}
|
||||||
|
func (o Ops) listResources(ctx context.Context, in *plane.ResourceIn) (*plane.Resources, error) {
|
||||||
|
g, err := gvr(in.Kind)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := &plane.Resources{Resources: []plane.Resource{}}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Dyn.Resource(g).Namespace(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
out.Resources = append(out.Resources, resourceView(in.Kind, list.Items[i]))
|
||||||
|
}
|
||||||
|
// The list's own resourceVersion is the cursor a watch resumes from. With a
|
||||||
|
// multi-namespace scope the last one wins, which is why a watch asks for a
|
||||||
|
// single namespace rather than inheriting a scope's cursor.
|
||||||
|
out.Version = list.GetResourceVersion()
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// getResource returns one custom resource.
|
||||||
|
//
|
||||||
|
// Response: {"kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2"},"status":{"phase":"Running"},"version":"918273"}
|
||||||
|
func (o Ops) getResource(ctx context.Context, in *plane.ResourceRef) (*plane.Resource, error) {
|
||||||
|
g, err := gvr(in.Kind)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := need("name", in.Name); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
got, err := b.Dyn.Resource(g).Namespace(ns).Get(ctx, in.Name, metav1.GetOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
v := resourceView(in.Kind, *got)
|
||||||
|
return &v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// watchResources returns what changed to one kind since a cursor.
|
||||||
|
//
|
||||||
|
// It is a BOUNDED read rather than a stream, and that is a design decision worth
|
||||||
|
// stating: a stream cannot cross the call plane as a value, and rendering one —
|
||||||
|
// Server-Sent Events, a websocket — is the EDGE's concern. A consumer builds its own
|
||||||
|
// stream by calling this in a loop with the cursor it got back, so the streaming
|
||||||
|
// shape exists once, in the consumer that renders it, instead of once per transport.
|
||||||
|
//
|
||||||
|
// `expired` is the field that matters most. When the cursor is older than the
|
||||||
|
// apiserver's history the watch cannot be resumed, and an empty page would look
|
||||||
|
// exactly like "nothing changed" — which is how a watch-backed board silently stops
|
||||||
|
// updating for hours. On `expired` a caller must LIST again and take the new cursor.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","kind":"App","namespace":"acme-prod","version":"918273","waitSeconds":30}
|
||||||
|
// Response: {"changes":[{"type":"MODIFIED","resource":{"kind":"App","namespace":"acme-prod","name":"api","status":{"phase":"Running"},"version":"918280"}}],"version":"918280"}
|
||||||
|
func (o Ops) watchResources(ctx context.Context, in *plane.WatchIn) (*plane.Changes, error) {
|
||||||
|
g, err := gvr(in.Kind)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
if len(scope) != 1 {
|
||||||
|
return nil, zip.ErrBadRequest("watch takes one namespace: a cursor belongs to one watch, so a multi-namespace watch would return a cursor that resumes none of them")
|
||||||
|
}
|
||||||
|
wait := maxWatch
|
||||||
|
if in.WaitSeconds > 0 && time.Duration(in.WaitSeconds)*time.Second < maxWatch {
|
||||||
|
wait = time.Duration(in.WaitSeconds) * time.Second
|
||||||
|
}
|
||||||
|
secs := int64(wait / time.Second)
|
||||||
|
w, err := b.Dyn.Resource(g).Namespace(scope[0]).Watch(ctx, metav1.ListOptions{
|
||||||
|
ResourceVersion: in.Version,
|
||||||
|
TimeoutSeconds: &secs,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
if apierrors.IsResourceExpired(err) || apierrors.IsGone(err) {
|
||||||
|
return &plane.Changes{Changes: []plane.Change{}, Expired: true}, nil
|
||||||
|
}
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
defer w.Stop()
|
||||||
|
out := &plane.Changes{Changes: []plane.Change{}, Version: in.Version}
|
||||||
|
deadline := time.NewTimer(wait)
|
||||||
|
defer deadline.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return out, nil
|
||||||
|
case <-deadline.C:
|
||||||
|
return out, nil
|
||||||
|
case ev, ok := <-w.ResultChan():
|
||||||
|
if !ok {
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
if ev.Type == watch.Error {
|
||||||
|
// A watch error arrives as an event, not as a call failure. A too-old
|
||||||
|
// cursor is the one a caller must act on differently, so it is a bit
|
||||||
|
// rather than a message they would have to match on.
|
||||||
|
if s, isStatus := ev.Object.(*metav1.Status); isStatus && (s.Reason == metav1.StatusReasonExpired || s.Reason == metav1.StatusReasonGone) {
|
||||||
|
out.Expired = true
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
u, isU := ev.Object.(*unstructured.Unstructured)
|
||||||
|
if !isU {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
r := resourceView(in.Kind, *u)
|
||||||
|
out.Changes = append(out.Changes, plane.Change{Type: string(ev.Type), Resource: r})
|
||||||
|
out.Version = r.Version
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// applyResource creates one custom resource of a registered kind, or patches the one
|
||||||
|
// already there.
|
||||||
|
//
|
||||||
|
// Five separate get-then-create-else-patch ladders in the fleet collapse into this one
|
||||||
|
// name. Each of those wrote the ladder itself and each was a race; here the apiserver
|
||||||
|
// resolves the race, because a create that loses falls through to the patch.
|
||||||
|
//
|
||||||
|
// The `spec` is the caller's domain payload and is applied whole — its SHAPE belongs to
|
||||||
|
// whoever authored the CRD, not to this app, which would otherwise have to be edited
|
||||||
|
// every time any kind grew a field. What this app owns is the KIND and the VERB, and
|
||||||
|
// both are named and typed. That is the line: `kind` is a key in a closed table, so
|
||||||
|
// this cannot become "apply any object the cluster has".
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2","replicas":3}}
|
||||||
|
// Response: {"kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2","replicas":3},"version":"918281"}
|
||||||
|
func (o Ops) applyResource(ctx context.Context, in *plane.ApplyIn) (*plane.Resource, error) {
|
||||||
|
g, err := gvr(in.Kind)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := need("name", in.Name); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if in.Spec == nil {
|
||||||
|
return nil, zip.ErrBadRequest("'spec' is required")
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
labels := map[string]any{"managed-by": managedBy}
|
||||||
|
for k, v := range in.Labels {
|
||||||
|
labels[k] = v
|
||||||
|
}
|
||||||
|
desired := &unstructured.Unstructured{Object: map[string]any{
|
||||||
|
"apiVersion": apiVersion(g),
|
||||||
|
"kind": in.Kind,
|
||||||
|
"metadata": map[string]any{
|
||||||
|
"name": in.Name, "namespace": ns, "labels": labels,
|
||||||
|
},
|
||||||
|
"spec": in.Spec,
|
||||||
|
}}
|
||||||
|
res := b.Dyn.Resource(g).Namespace(ns)
|
||||||
|
existing, err := res.Get(ctx, in.Name, metav1.GetOptions{})
|
||||||
|
switch {
|
||||||
|
case apierrors.IsNotFound(err):
|
||||||
|
created, cerr := res.Create(ctx, desired, metav1.CreateOptions{})
|
||||||
|
if apierrors.IsAlreadyExists(cerr) {
|
||||||
|
// Lost the create race. The other writer's object is now the one to
|
||||||
|
// update, so read it and fall through rather than reporting a conflict
|
||||||
|
// the caller can do nothing useful with.
|
||||||
|
existing, err = res.Get(ctx, in.Name, metav1.GetOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
} else if cerr != nil {
|
||||||
|
return nil, fail(cerr)
|
||||||
|
} else {
|
||||||
|
v := resourceView(in.Kind, *created)
|
||||||
|
return &v, nil
|
||||||
|
}
|
||||||
|
case err != nil:
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
desired.SetResourceVersion(existing.GetResourceVersion())
|
||||||
|
updated, err := res.Update(ctx, desired, metav1.UpdateOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
v := resourceView(in.Kind, *updated)
|
||||||
|
return &v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// deleteResource deletes one custom resource of a registered kind.
|
||||||
|
//
|
||||||
|
// Deleting an object is not deleting its data: a workload's App CR is desired state and
|
||||||
|
// can be recreated freely, while the PersistentVolumeClaim it mounted holds the only
|
||||||
|
// copy of the tenant's data and is a different lifetime. Nothing here deletes a claim.
|
||||||
|
//
|
||||||
|
// Response: {"removed":true}
|
||||||
|
func (o Ops) deleteResource(ctx context.Context, in *plane.ResourceRef) (*plane.Deleted, error) {
|
||||||
|
g, err := gvr(in.Kind)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := need("name", in.Name); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
err = b.Dyn.Resource(g).Namespace(ns).Delete(ctx, in.Name, metav1.DeleteOptions{})
|
||||||
|
if apierrors.IsNotFound(err) {
|
||||||
|
return &plane.Deleted{}, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
return &plane.Deleted{Removed: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func apiVersion(g schema.GroupVersionResource) string {
|
||||||
|
if g.Group == "" {
|
||||||
|
return g.Version
|
||||||
|
}
|
||||||
|
return g.Group + "/" + g.Version
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceView(kind string, u unstructured.Unstructured) plane.Resource {
|
||||||
|
r := plane.Resource{
|
||||||
|
Kind: kind, Namespace: u.GetNamespace(), Name: u.GetName(),
|
||||||
|
Labels: u.GetLabels(), Version: u.GetResourceVersion(),
|
||||||
|
}
|
||||||
|
if spec, found, _ := unstructured.NestedMap(u.Object, "spec"); found {
|
||||||
|
r.Spec = spec
|
||||||
|
}
|
||||||
|
if status, found, _ := unstructured.NestedMap(u.Object, "status"); found {
|
||||||
|
r.Status = status
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}
|
||||||
+214
@@ -0,0 +1,214 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
corev1 "k8s.io/api/core/v1"
|
||||||
|
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||||
|
"k8s.io/apimachinery/pkg/api/resource"
|
||||||
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||||
|
"k8s.io/apimachinery/pkg/types"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// listClaims returns PersistentVolumeClaims.
|
||||||
|
//
|
||||||
|
// Response: {"claims":[{"namespace":"acme-prod","name":"data","phase":"Bound","volume":"pvc-9f2","requestGi":100,"storageClass":"do-block-storage"}]}
|
||||||
|
func (o Ops) listClaims(ctx context.Context, in *plane.Selector) (*plane.Claims, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := []plane.Claim{}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Typed.CoreV1().PersistentVolumeClaims(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
out = append(out, claimView(list.Items[i]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &plane.Claims{Claims: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// listPersistentVolumes returns the cluster's PersistentVolumes.
|
||||||
|
//
|
||||||
|
// A PV is where the link to the real cloud device lives: `handle` is the provider's
|
||||||
|
// own volume id, read from the CSI source or from the legacy flexVolume shape, and it
|
||||||
|
// is what lets a reconciliation say whether a disk somebody is paying for is attached
|
||||||
|
// to anything. The claim reference travels with it because a PV with no claim and a PV
|
||||||
|
// whose claim is gone are different situations.
|
||||||
|
//
|
||||||
|
// PVs are cluster-scoped, so a namespace-bounded registration cannot read them.
|
||||||
|
//
|
||||||
|
// Response: {"volumes":[{"name":"pvc-9f2","phase":"Bound","handle":"a1b2c3","claimNamespace":"acme-prod","claimName":"data","capacityGi":100}]}
|
||||||
|
func (o Ops) listPersistentVolumes(ctx context.Context, in *plane.ClusterRef) (*plane.Volumes, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := b.WholeCluster(); err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
list, err := b.Typed.CoreV1().PersistentVolumes().List(ctx, metav1.ListOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := make([]plane.Volume, 0, len(list.Items))
|
||||||
|
for i := range list.Items {
|
||||||
|
out = append(out, volumeView(list.Items[i]))
|
||||||
|
}
|
||||||
|
return &plane.Volumes{Volumes: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ensureClaim creates a PersistentVolumeClaim if it is absent.
|
||||||
|
//
|
||||||
|
// Idempotent on name in the namespace, for the reason ensureNamespace is: the
|
||||||
|
// get-then-create ladder is a race when each caller writes it.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","namespace":"acme-prod","name":"data","gi":100,"storageClass":"do-block-storage"}
|
||||||
|
// Response: {"name":"data","created":true}
|
||||||
|
func (o Ops) ensureClaim(ctx context.Context, in *plane.EnsureClaimIn) (*plane.Ensured, error) {
|
||||||
|
if err := need("name", in.Name); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if in.Gi <= 0 {
|
||||||
|
return nil, zip.ErrBadRequest("'gi' must be at least 1")
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
if _, err := b.Typed.CoreV1().PersistentVolumeClaims(ns).Get(ctx, in.Name, metav1.GetOptions{}); err == nil {
|
||||||
|
return &plane.Ensured{Name: in.Name}, nil
|
||||||
|
} else if !apierrors.IsNotFound(err) {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
mode := corev1.ReadWriteOnce
|
||||||
|
if strings.EqualFold(in.AccessMode, string(corev1.ReadWriteMany)) {
|
||||||
|
mode = corev1.ReadWriteMany
|
||||||
|
}
|
||||||
|
pvc := &corev1.PersistentVolumeClaim{
|
||||||
|
ObjectMeta: metav1.ObjectMeta{
|
||||||
|
Name: in.Name, Namespace: ns,
|
||||||
|
Labels: map[string]string{"managed-by": managedBy},
|
||||||
|
},
|
||||||
|
Spec: corev1.PersistentVolumeClaimSpec{
|
||||||
|
AccessModes: []corev1.PersistentVolumeAccessMode{mode},
|
||||||
|
Resources: corev1.VolumeResourceRequirements{
|
||||||
|
Requests: corev1.ResourceList{corev1.ResourceStorage: gi(in.Gi)},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if in.StorageClass != "" {
|
||||||
|
sc := in.StorageClass
|
||||||
|
pvc.Spec.StorageClassName = &sc
|
||||||
|
}
|
||||||
|
_, err = b.Typed.CoreV1().PersistentVolumeClaims(ns).Create(ctx, pvc, metav1.CreateOptions{})
|
||||||
|
if apierrors.IsAlreadyExists(err) {
|
||||||
|
return &plane.Ensured{Name: in.Name}, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
return &plane.Ensured{Name: in.Name, Created: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// expandClaim grows a PersistentVolumeClaim to a new size.
|
||||||
|
//
|
||||||
|
// This is the ONE correct way to grow a volume Kubernetes manages, and the reason it
|
||||||
|
// is a named op rather than a patch: the resize controller acts on the CLAIM, growing
|
||||||
|
// the backing cloud device and then the filesystem on it, so claim, PersistentVolume,
|
||||||
|
// device and filesystem all end up agreeing. Growing the device through the cloud
|
||||||
|
// provider's own API instead leaves the claim and the PV declaring the old capacity
|
||||||
|
// and the filesystem never grown at all — three sources of truth, two of them wrong.
|
||||||
|
//
|
||||||
|
// The StorageClass must set `allowVolumeExpansion`; when it does not, the apiserver
|
||||||
|
// refuses and its message is surfaced verbatim rather than worked around. A shrink is
|
||||||
|
// refused by the apiserver too, and there is no op that deletes a claim: the claim
|
||||||
|
// holds the only copy of the tenant's data, so this app can make one bigger and
|
||||||
|
// nothing else.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","namespace":"acme-prod","name":"data","gi":200}
|
||||||
|
// Response: {"gi":200}
|
||||||
|
func (o Ops) expandClaim(ctx context.Context, in *plane.ExpandClaimIn) (*plane.Expanded, error) {
|
||||||
|
if err := need("name", in.Name); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if in.Gi <= 0 {
|
||||||
|
return nil, zip.ErrBadRequest("'gi' must be at least 1")
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
patch := fmt.Sprintf(`{"spec":{"resources":{"requests":{"storage":"%dGi"}}}}`, in.Gi)
|
||||||
|
if _, err := b.Typed.CoreV1().PersistentVolumeClaims(ns).Patch(
|
||||||
|
ctx, in.Name, types.MergePatchType, []byte(patch), metav1.PatchOptions{}); err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
return &plane.Expanded{Gi: in.Gi}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func claimView(p corev1.PersistentVolumeClaim) plane.Claim {
|
||||||
|
v := plane.Claim{
|
||||||
|
Namespace: p.Namespace, Name: p.Name,
|
||||||
|
Phase: string(p.Status.Phase), Volume: p.Spec.VolumeName,
|
||||||
|
}
|
||||||
|
if q, ok := p.Spec.Resources.Requests[corev1.ResourceStorage]; ok {
|
||||||
|
v.RequestGi = int(q.Value() >> 30)
|
||||||
|
}
|
||||||
|
if p.Spec.StorageClassName != nil {
|
||||||
|
v.StorageClass = *p.Spec.StorageClassName
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
func volumeView(pv corev1.PersistentVolume) plane.Volume {
|
||||||
|
v := plane.Volume{Name: pv.Name, Phase: string(pv.Status.Phase), Handle: volumeHandle(pv)}
|
||||||
|
if pv.Spec.ClaimRef != nil {
|
||||||
|
v.ClaimNS, v.ClaimName = pv.Spec.ClaimRef.Namespace, pv.Spec.ClaimRef.Name
|
||||||
|
}
|
||||||
|
if q, ok := pv.Spec.Capacity[corev1.ResourceStorage]; ok {
|
||||||
|
v.CapacityGi = int(q.Value() >> 30)
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
// volumeHandle extracts the backing device id a PV claims. Deliberately NOT filtered
|
||||||
|
// by CSI driver name: matching broadly means MORE volumes are treated as in use, which
|
||||||
|
// is the safe direction for anything that would otherwise delete one. The legacy
|
||||||
|
// flexVolume shape is read too, so a pre-CSI PV still protects its device.
|
||||||
|
func volumeHandle(pv corev1.PersistentVolume) string {
|
||||||
|
if pv.Spec.CSI != nil && strings.TrimSpace(pv.Spec.CSI.VolumeHandle) != "" {
|
||||||
|
return pv.Spec.CSI.VolumeHandle
|
||||||
|
}
|
||||||
|
if pv.Spec.FlexVolume != nil {
|
||||||
|
if v := strings.TrimSpace(pv.Spec.FlexVolume.Options["volumeID"]); v != "" {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func gi(n int) resource.Quantity {
|
||||||
|
return resource.MustParse(fmt.Sprintf("%dGi", n))
|
||||||
|
}
|
||||||
@@ -0,0 +1,368 @@
|
|||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
appsv1 "k8s.io/api/apps/v1"
|
||||||
|
corev1 "k8s.io/api/core/v1"
|
||||||
|
networkingv1 "k8s.io/api/networking/v1"
|
||||||
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// lbIDAnnotation is the annotation a cloud-controller stamps on a Service once it has
|
||||||
|
// provisioned a load balancer for it — the strongest link between the two, and what
|
||||||
|
// lets a reconciliation tell an orphaned balancer from a live one.
|
||||||
|
const lbIDAnnotation = "kubernetes.digitalocean.com/load-balancer-id"
|
||||||
|
|
||||||
|
// listDeployments returns Deployments with their live replica counts and the images
|
||||||
|
// they are ACTUALLY running.
|
||||||
|
//
|
||||||
|
// The running image is why this op exists rather than a read of the App CR: a CR's
|
||||||
|
// status does not surface the tag the pods are on, so the live Deployment is the only
|
||||||
|
// place "declared v1.4.2, running v1.4.1" can be seen at all — which is the whole of
|
||||||
|
// drift detection.
|
||||||
|
//
|
||||||
|
// Example: {"cluster":"hanzo-k8s","namespace":"acme-prod"}
|
||||||
|
// Response: {"workloads":[{"namespace":"acme-prod","name":"api","kind":"Deployment","replicas":3,"ready":3,"updated":3,"images":["registry.hanzo.ai/acme/api:1.4.2"]}]}
|
||||||
|
func (o Ops) listDeployments(ctx context.Context, in *plane.Selector) (*plane.Workloads, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := []plane.Workload{}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Typed.AppsV1().Deployments(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
out = append(out, deploymentView(list.Items[i]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &plane.Workloads{Workloads: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// getDeployment returns one Deployment.
|
||||||
|
//
|
||||||
|
// Response: {"namespace":"acme-prod","name":"api","kind":"Deployment","replicas":3,"ready":3,"updated":3,"images":["registry.hanzo.ai/acme/api:1.4.2"]}
|
||||||
|
func (o Ops) getDeployment(ctx context.Context, in *plane.NamedIn) (*plane.Workload, error) {
|
||||||
|
if err := need("name", in.Name); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ns, err := b.One(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
d, err := b.Typed.AppsV1().Deployments(ns).Get(ctx, in.Name, metav1.GetOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
v := deploymentView(*d)
|
||||||
|
return &v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// listReplicaSets returns ReplicaSets.
|
||||||
|
//
|
||||||
|
// A deploy tree needs them because they are the layer between a Deployment and its
|
||||||
|
// pods: during a rollout two exist, and which pods belong to which is the difference
|
||||||
|
// between "the new version is up" and "the old one still is".
|
||||||
|
//
|
||||||
|
// Response: {"workloads":[{"namespace":"acme-prod","name":"api-7d9f","kind":"ReplicaSet","replicas":3,"ready":3,"owner":"Deployment/api"}]}
|
||||||
|
func (o Ops) listReplicaSets(ctx context.Context, in *plane.Selector) (*plane.Workloads, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := []plane.Workload{}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Typed.AppsV1().ReplicaSets(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
r := list.Items[i]
|
||||||
|
w := plane.Workload{
|
||||||
|
Namespace: r.Namespace, Name: r.Name, Kind: "ReplicaSet",
|
||||||
|
Ready: r.Status.ReadyReplicas, Owner: controllerOf(r.OwnerReferences),
|
||||||
|
}
|
||||||
|
if r.Spec.Replicas != nil {
|
||||||
|
w.Replicas = *r.Spec.Replicas
|
||||||
|
}
|
||||||
|
w.Images = images(r.Spec.Template.Spec)
|
||||||
|
out = append(out, w)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &plane.Workloads{Workloads: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// listStatefulSets returns StatefulSets.
|
||||||
|
//
|
||||||
|
// They are read separately from Deployments because they are a different lifetime: a
|
||||||
|
// StatefulSet's volumeClaimTemplates are immutable, so a storage change means
|
||||||
|
// recreating the workload around the claims rather than patching it — and a board that
|
||||||
|
// folded the two kinds together would offer an operation that cannot work.
|
||||||
|
//
|
||||||
|
// Response: {"workloads":[{"namespace":"acme-prod","name":"pg","kind":"StatefulSet","replicas":3,"ready":3,"images":["postgres:17"]}]}
|
||||||
|
func (o Ops) listStatefulSets(ctx context.Context, in *plane.Selector) (*plane.Workloads, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := []plane.Workload{}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Typed.AppsV1().StatefulSets(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
s := list.Items[i]
|
||||||
|
w := plane.Workload{
|
||||||
|
Namespace: s.Namespace, Name: s.Name, Kind: "StatefulSet",
|
||||||
|
Ready: s.Status.ReadyReplicas, Updated: s.Status.UpdatedReplicas,
|
||||||
|
Owner: controllerOf(s.OwnerReferences), Images: images(s.Spec.Template.Spec),
|
||||||
|
}
|
||||||
|
if s.Spec.Replicas != nil {
|
||||||
|
w.Replicas = *s.Spec.Replicas
|
||||||
|
}
|
||||||
|
out = append(out, w)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &plane.Workloads{Workloads: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// listServices returns Services with every identity by which one can be matched to the
|
||||||
|
// load balancer in front of it.
|
||||||
|
//
|
||||||
|
// Both the cloud-controller's own annotation and the assigned addresses travel, and
|
||||||
|
// either match is enough. That breadth is the safe direction: matching MORE Services to
|
||||||
|
// a balancer means fewer balancers look orphaned, and the consequence of a wrong match
|
||||||
|
// in the other direction is deleting a live one.
|
||||||
|
//
|
||||||
|
// Response: {"services":[{"namespace":"acme-prod","name":"api","type":"LoadBalancer","clusterIp":"10.3.0.7","ips":["203.0.113.10"],"lbId":"a1b2","ports":[443]}]}
|
||||||
|
func (o Ops) listServices(ctx context.Context, in *plane.Selector) (*plane.Services, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := []plane.Service{}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Typed.CoreV1().Services(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
out = append(out, serviceView(list.Items[i]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &plane.Services{Services: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// listIngresses returns Ingresses with the hosts they answer and which of those are
|
||||||
|
// covered by TLS.
|
||||||
|
//
|
||||||
|
// The two host lists are separate because a host in `hosts` but not in `tlsHosts` is a
|
||||||
|
// public URL served over plain HTTP, which is a finding rather than a detail.
|
||||||
|
//
|
||||||
|
// Response: {"ingresses":[{"namespace":"acme-prod","name":"api","hosts":["api.acme.com"],"tlsHosts":["api.acme.com"],"addresses":["203.0.113.10"]}]}
|
||||||
|
func (o Ops) listIngresses(ctx context.Context, in *plane.Selector) (*plane.Ingresses, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := []plane.Ingress{}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Typed.NetworkingV1().Ingresses(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
out = append(out, ingressView(list.Items[i]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &plane.Ingresses{Ingresses: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// listEvents returns cluster events, oldest first.
|
||||||
|
//
|
||||||
|
// This is the only place a failure with no pod behind it can be explained — an
|
||||||
|
// unschedulable workload, a quota refusal, a volume that never bound — because those
|
||||||
|
// produce an event and nothing else. Events are ordered by last-seen so the tail is
|
||||||
|
// the current situation.
|
||||||
|
//
|
||||||
|
// Response: {"events":[{"namespace":"acme-prod","name":"api.17f","type":"Warning","reason":"FailedScheduling","message":"0/6 nodes are available: insufficient nvidia.com/gpu","object":"Pod/api-7d9f-2xk","count":4,"lastSeen":1785110400}]}
|
||||||
|
func (o Ops) listEvents(ctx context.Context, in *plane.Selector) (*plane.Events, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := []plane.Event{}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Typed.CoreV1().Events(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
out = append(out, eventView(list.Items[i]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.SliceStable(out, func(i, j int) bool { return out[i].LastSeen < out[j].LastSeen })
|
||||||
|
return &plane.Events{Events: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// listConfigMaps returns ConfigMaps, with their data.
|
||||||
|
//
|
||||||
|
// The data travels because a ConfigMap is by definition not secret material. Secrets
|
||||||
|
// are a different kind and this API does not read one back, at all: an RPC that returns
|
||||||
|
// a tenant's Secret is a credential-exfiltration endpoint however carefully it is
|
||||||
|
// gated, and material that needs protecting belongs in KMS.
|
||||||
|
//
|
||||||
|
// Response: {"configMaps":[{"namespace":"hanzo","name":"cron-schedules","data":{"nightly":"0 3 * * *"}}]}
|
||||||
|
func (o Ops) listConfigMaps(ctx context.Context, in *plane.Selector) (*plane.ConfigMaps, error) {
|
||||||
|
b, err := o.reach(ctx, in.Cluster)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
scope, err := b.Scope(in.Namespace)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
out := []plane.ConfigMap{}
|
||||||
|
for _, ns := range scope {
|
||||||
|
list, err := b.Typed.CoreV1().ConfigMaps(ns).List(ctx, metav1.ListOptions{LabelSelector: in.LabelSelector})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fail(err)
|
||||||
|
}
|
||||||
|
for i := range list.Items {
|
||||||
|
c := list.Items[i]
|
||||||
|
out = append(out, plane.ConfigMap{Namespace: c.Namespace, Name: c.Name, Data: c.Data})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &plane.ConfigMaps{ConfigMaps: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func deploymentView(d appsv1.Deployment) plane.Workload {
|
||||||
|
w := plane.Workload{
|
||||||
|
Namespace: d.Namespace, Name: d.Name, Kind: "Deployment",
|
||||||
|
Ready: d.Status.ReadyReplicas, Updated: d.Status.UpdatedReplicas,
|
||||||
|
Owner: controllerOf(d.OwnerReferences), Images: images(d.Spec.Template.Spec),
|
||||||
|
}
|
||||||
|
if d.Spec.Replicas != nil {
|
||||||
|
w.Replicas = *d.Spec.Replicas
|
||||||
|
}
|
||||||
|
return w
|
||||||
|
}
|
||||||
|
|
||||||
|
func serviceView(s corev1.Service) plane.Service {
|
||||||
|
v := plane.Service{
|
||||||
|
Namespace: s.Namespace, Name: s.Name, Type: string(s.Spec.Type),
|
||||||
|
ClusterIP: s.Spec.ClusterIP,
|
||||||
|
LBID: strings.TrimSpace(s.Annotations[lbIDAnnotation]),
|
||||||
|
}
|
||||||
|
if ip := strings.TrimSpace(s.Spec.LoadBalancerIP); ip != "" {
|
||||||
|
v.IPs = append(v.IPs, ip)
|
||||||
|
}
|
||||||
|
for _, in := range s.Status.LoadBalancer.Ingress {
|
||||||
|
if ip := strings.TrimSpace(in.IP); ip != "" {
|
||||||
|
v.IPs = append(v.IPs, ip)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, p := range s.Spec.Ports {
|
||||||
|
v.Ports = append(v.Ports, p.Port)
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
func ingressView(i networkingv1.Ingress) plane.Ingress {
|
||||||
|
v := plane.Ingress{Namespace: i.Namespace, Name: i.Name}
|
||||||
|
for _, r := range i.Spec.Rules {
|
||||||
|
if r.Host != "" {
|
||||||
|
v.Hosts = append(v.Hosts, r.Host)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, t := range i.Spec.TLS {
|
||||||
|
v.TLSHosts = append(v.TLSHosts, t.Hosts...)
|
||||||
|
}
|
||||||
|
for _, lb := range i.Status.LoadBalancer.Ingress {
|
||||||
|
switch {
|
||||||
|
case lb.IP != "":
|
||||||
|
v.Addresses = append(v.Addresses, lb.IP)
|
||||||
|
case lb.Hostname != "":
|
||||||
|
v.Addresses = append(v.Addresses, lb.Hostname)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
func eventView(e corev1.Event) plane.Event {
|
||||||
|
v := plane.Event{
|
||||||
|
Namespace: e.Namespace, Name: e.Name, Type: e.Type,
|
||||||
|
Reason: e.Reason, Message: e.Message, Count: e.Count,
|
||||||
|
}
|
||||||
|
if e.InvolvedObject.Kind != "" {
|
||||||
|
v.Object = e.InvolvedObject.Kind + "/" + e.InvolvedObject.Name
|
||||||
|
}
|
||||||
|
// LastTimestamp is the field a legacy event carries; a modern one leaves it zero
|
||||||
|
// and sets EventTime. Read both, or every event on a current cluster sorts to the
|
||||||
|
// beginning of time.
|
||||||
|
switch {
|
||||||
|
case !e.LastTimestamp.IsZero():
|
||||||
|
v.LastSeen = e.LastTimestamp.Unix()
|
||||||
|
case !e.EventTime.IsZero():
|
||||||
|
v.LastSeen = e.EventTime.Time.Unix()
|
||||||
|
case !e.FirstTimestamp.IsZero():
|
||||||
|
v.LastSeen = e.FirstTimestamp.Unix()
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
func images(spec corev1.PodSpec) []string {
|
||||||
|
out := make([]string, 0, len(spec.InitContainers)+len(spec.Containers))
|
||||||
|
for _, c := range spec.InitContainers {
|
||||||
|
out = append(out, c.Image)
|
||||||
|
}
|
||||||
|
for _, c := range spec.Containers {
|
||||||
|
out = append(out, c.Image)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func controllerOf(refs []metav1.OwnerReference) string {
|
||||||
|
for _, r := range refs {
|
||||||
|
if r.Controller != nil && *r.Controller {
|
||||||
|
return r.Kind + "/" + r.Name
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
@@ -0,0 +1,752 @@
|
|||||||
|
// Code generated by zipdoc; DO NOT EDIT.
|
||||||
|
|
||||||
|
package ops
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
)
|
||||||
|
|
||||||
|
func init() {
|
||||||
|
zip.Describe("DELETE /v1/k8s/clusters/:cluster", zip.Doc{
|
||||||
|
Description: "Detaches a cluster from the calling org and destroys its sealed\ncredential.\n\nNothing in the cluster is touched: this removes Hanzo's ability to reach it, not\nthe cluster itself. Deleting a name that is not registered reports removed=false\nrather than failing, so a repeated deregistration is safe.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"removed":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("DELETE /v1/k8s/namespaces/:namespace", zip.Doc{
|
||||||
|
Description: "Deletes a namespace and everything in it.\n\nThis is the highest blast radius in the whole API: Kubernetes deletes a namespace\nby garbage-collecting every object inside it, including PersistentVolumeClaims,\nwhich for a Delete-reclaim StorageClass destroys the tenant's data. There is no\nundo and no confirmation this API can offer that would make one.\n\nIt is therefore its own named operation, gated separately from every other write,\nand it is refused outright for a registration bounded to a namespace set: a scoped\ncredential exists precisely because the cluster is shared, and \"delete a namespace\non a shared cluster\" is not an authority a tenant holds.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"NamespaceRef.cluster": "Cluster is the registered cluster the namespace lives on.",
|
||||||
|
"NamespaceRef.namespace": "Namespace is the namespace's name.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"removed":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("DELETE /v1/k8s/resources/:kind/:namespace/:name", zip.Doc{
|
||||||
|
Description: "Deletes one custom resource of a registered kind.\n\nDeleting an object is not deleting its data: a workload's App CR is desired state and\ncan be recreated freely, while the PersistentVolumeClaim it mounted holds the only\ncopy of the tenant's data and is a different lifetime. Nothing here deletes a claim.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ResourceRef.cluster": "Cluster is the registered cluster the object lives on.",
|
||||||
|
"ResourceRef.kind": "Kind is the registered kind.",
|
||||||
|
"ResourceRef.name": "Name is the object's name.",
|
||||||
|
"ResourceRef.namespace": "Namespace is the object's namespace; empty for a cluster-scoped kind.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"removed":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/clusters", zip.Doc{
|
||||||
|
Description: "Returns every Kubernetes cluster the calling org may reach.\n\nThis is the fleet's ONE cluster list. A row is a real registration — a name, the\nprovider it came from, its apiserver endpoint and the node and accelerator counts\nmeasured when it was attached — and there is no such thing as an unregistered\ncluster this API can see. The credential is never part of a row: it is sealed in\nKMS and resolved only at the moment an op uses it.\n\nThe list is scoped by the caller's org and by nothing the caller can type, so a\ntenant sees its own clusters and no others.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Registered.namespaces": "Namespaces bounds what this registration may address. Empty means the\nwhole cluster, which is what an org registering its OWN cluster gets.",
|
||||||
|
"Registered.org": "Org is which org OWNS this cluster. On a reply that is a property of the\nrecord, not a claim by the caller.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"clusters":[{"name":"hanzo-k8s","org":"hanzo","provider":"doks","endpoint":"https://k8s.example.com","nodes":6,"nvidiaGpu":8,"amdGpu":0,"registered":"2026-07-30T00:00:00Z","default":true}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/clusters/:cluster", zip.Doc{
|
||||||
|
Description: "Returns one registered cluster.\n\nA name the calling org has not registered answers 404 — including a name another\norg HAS registered, which is indistinguishable from one that does not exist. That\nis deliberate: a distinguishable refusal would let a tenant enumerate other\ntenants' cluster names one guess at a time.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
"Registered.namespaces": "Namespaces bounds what this registration may address. Empty means the\nwhole cluster, which is what an org registering its OWN cluster gets.",
|
||||||
|
"Registered.org": "Org is which org OWNS this cluster. On a reply that is a property of the\nrecord, not a claim by the caller.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"name":"lux-k8s","org":"lux","provider":"byo","endpoint":"https://k8s.lux.example","nodes":12,"nvidiaGpu":32,"amdGpu":0,"registered":"2026-07-30T00:00:00Z","default":false}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/clusters/:cluster/status", zip.Doc{
|
||||||
|
Description: "Reports whether a registered cluster answers right now.\n\nThe probe is the apiserver's version endpoint, because that is the one read every\ncredential can make regardless of what RBAC it was granted: a registration bounded\nto one namespace would fail a namespace list and look dead while being perfectly\nhealthy. So this answers \"is the credential live and the apiserver reachable\",\nwhich is the question a board asks, and the ops that need a permission ask for\nthat permission themselves.\n\nA cluster that does not answer is connected=false with the reason verbatim — never\nan error, because \"this cluster is down\" is a successful answer to this question.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
"Reachable.connected": "Connected is whether the apiserver answered.",
|
||||||
|
"Reachable.reason": "Reason is why it did not, verbatim. Absent on success.",
|
||||||
|
"Reachable.version": "Version is the server version string, when it answered.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"connected":true,"version":"v1.33.1"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/jobs", zip.Doc{
|
||||||
|
Description: "Returns batch Jobs with a rolled-up phase.\n\n`phase` is one word — running, succeeded or failed — computed here rather than left\nto every caller to derive from three counters. Three consumers deriving it three ways\nis three chances to read a failed job as merely unfinished.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Job.phase": "Phase is \"running\", \"succeeded\" or \"failed\" — the one word a caller wants,\nrolled here rather than re-derived from three counters by every consumer.",
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"jobs":[{"namespace":"hanzo","name":"build-1a2b","active":0,"succeeded":1,"failed":0,"phase":"succeeded","startedAt":1785110400,"endedAt":1785110460}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/jobs/:namespace/:name", zip.Doc{
|
||||||
|
Description: "Returns one batch Job.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Job.phase": "Phase is \"running\", \"succeeded\" or \"failed\" — the one word a caller wants,\nrolled here rather than re-derived from three counters by every consumer.",
|
||||||
|
"NamedIn.cluster": "Cluster is the registered cluster the object lives on.",
|
||||||
|
"NamedIn.name": "Name is the object's name.",
|
||||||
|
"NamedIn.namespace": "Namespace is the object's namespace.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"namespace":"hanzo","name":"build-1a2b","active":0,"succeeded":1,"failed":0,"phase":"succeeded","startedAt":1785110400,"endedAt":1785110460}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/metrics/pods", zip.Doc{
|
||||||
|
Description: "Returns live CPU and memory use for pods, from metrics.k8s.io.\n\n`available` is the field that carries the honesty. metrics-server is absent from most\nof our clusters, and a cluster without it answers this call with an error — which,\nrendered as zeroes, tells an operator that a busy namespace is idle. So a missing\nmetrics API is reported as available=false with no rows, and a caller must render\n\"unmeasured\" rather than \"0m\".",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"PodMetric.cpuMilli": "CPUMilli is CPU use in millicores; MemoryBytes is working-set memory.",
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod"}`),
|
||||||
|
Response: json.RawMessage(`{"pods":[{"namespace":"acme-prod","name":"api-7d9f-2xk","cpuMilli":250,"memoryBytes":536870912}],"available":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/namespaces", zip.Doc{
|
||||||
|
Description: "Returns the namespaces the calling org's registration may address.\n\nFor a registration that owns the whole cluster this is every namespace on it. For\none bounded to a namespace set it is that set, read one namespace at a time — so a\nscoped tenant learns nothing about its neighbours, not even how many there are.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"namespaces":[{"name":"acme-prod","phase":"Active","labels":{"managed-by":"hanzo-cloud"}}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/namespaces/:namespace", zip.Doc{
|
||||||
|
Description: "Returns one namespace.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"NamespaceRef.cluster": "Cluster is the registered cluster the namespace lives on.",
|
||||||
|
"NamespaceRef.namespace": "Namespace is the namespace's name.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"name":"acme-prod","phase":"Active","labels":{"managed-by":"hanzo-cloud"}}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/nodes", zip.Doc{
|
||||||
|
Description: "Returns a cluster's worker nodes: whether each is up, whether it will\ntake new work, and what accelerators it offers.\n\nThis is the one node inventory in the fleet. The GPU counts come from each node's\nallocatable `nvidia.com/gpu` and `amd.com/gpu`, which is what the scheduler will\nactually honour — not a label somebody typed.\n\nNodes are cluster-scoped, so a registration bounded to a namespace set cannot read\nthem: answering would hand one tenant on a shared cluster the whole cluster's\ncapacity.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"nodes":[{"name":"pool-gpu-a1","ready":true,"schedulable":true,"region":"nyc3","instanceKind":"gpu-h100x8-640gb","cpu":"64","memory":"755Gi","nvidiaGpu":8,"amdGpu":0,"internalIp":"10.0.0.4"}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/nodes/volume-stats", zip.Doc{
|
||||||
|
Description: "Reports how full every mounted volume on a cluster is.\n\nIt is the ONLY source of volume fill we have. Cloud providers do not expose how\nfull a disk is, and metrics-server is absent from most of our clusters — but every\nkubelet serves `stats/summary` unconditionally, so this works everywhere. The read\ngoes through the apiserver's node proxy, which is why this is a SEPARATE named op:\n`nodes/proxy get` is effectively arbitrary kubelet access, and naming it here is\nwhat lets every other operation's grant omit it.\n\nA kubelet that will not answer costs a reading and never the call: fill is\nadvisory, no safety verdict depends on it, and failing the whole sweep over one\nsilent node would block every volume decision on the fleet. `nodesRead` against\n`nodesTotal` is how a caller knows the difference between \"empty\" and\n\"unmeasured\" — a claim with no row is unmeasured, and treating that as empty is\nhow live data gets condemned.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
"VolumeStats.nodesRead": "NodesRead is how many kubelets answered; NodesTotal how many were asked.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"volumes":[{"namespace":"acme","name":"data","usedBytes":48318382080}],"nodesRead":6,"nodesTotal":6}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/pods", zip.Doc{
|
||||||
|
Description: "Returns pods, narrowed by namespace, labels or node.\n\nOne op covers every way the fleet asks for pods: the whole-cluster sweep an\ninfrastructure board takes, the label-selected set that answers \"is this app's\nworkload up\", and the single-node set a drain needs. They were three call sites\nwith three shapes; the difference between them is which argument is set.\n\nContainer ENV is deliberately absent from the projection. A pod spec carries the\nsecret material injected into it, so a read surface that returned the spec would be\nan exfiltration door wearing the name of a health check.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"PodsIn.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"PodsIn.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"PodsIn.namespace": "Namespace narrows to one namespace. Empty means every namespace this\nregistration may address, which for a bounded registration is its own set\nand never the whole cluster.",
|
||||||
|
"PodsIn.nodeName": "NodeName narrows to the pods on one node — the field-selector variant the\ndrain path needs, named rather than left to a raw selector string.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod","labelSelector":"app=api"}`),
|
||||||
|
Response: json.RawMessage(`{"pods":[{"namespace":"acme-prod","name":"api-7d9f-2xk","phase":"Running","node":"pool-a1","controller":"ReplicaSet/api-7d9f","images":["registry.hanzo.ai/acme/api:1.4.2"],"restarts":0}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/pods/:namespace/:pod/logs", zip.Doc{
|
||||||
|
Description: "Returns what one container printed.\n\n`tailLines` is the bound a caller should set; without one the read is capped at one\nmebibyte and truncated from the FRONT, so the newest output — the part that\nexplains a crash — is what survives.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"LogsIn.cluster": "Cluster is the registered cluster the pod runs on.",
|
||||||
|
"LogsIn.container": "Container names the container; empty takes the pod's only one.",
|
||||||
|
"LogsIn.namespace": "Namespace is the pod's namespace.",
|
||||||
|
"LogsIn.pod": "Pod is the pod's name.",
|
||||||
|
"LogsIn.tailLines": "TailLines caps how many lines come back, newest last. Zero takes the\nserver's default. A log read is unbounded by nature, so this is the bound.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod","pod":"api-7d9f-2xk","tailLines":200}`),
|
||||||
|
Response: json.RawMessage(`{"log":"2026-07-30T00:00:01Z listening on :8000\n"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/resources", zip.Doc{
|
||||||
|
Description: "Returns custom resources of one registered kind.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Resource.version": "Version is the object's resourceVersion — the cursor a Watch resumes from.",
|
||||||
|
"ResourceIn.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"ResourceIn.kind": "Kind is the registered kind: \"App\", \"Application\", \"Datastore\",\n\"Validator\", \"TrainJob\", \"InferenceService\", \"IngressRoute\", \"Middleware\"\nor \"AppProject\".",
|
||||||
|
"ResourceIn.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"ResourceIn.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address. Ignored for a cluster-scoped kind.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","kind":"App","namespace":"acme-prod"}`),
|
||||||
|
Response: json.RawMessage(`{"resources":[{"kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2"},"status":{"phase":"Running"},"version":"918273"}],"version":"918273"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/resources/:kind/:namespace/:name", zip.Doc{
|
||||||
|
Description: "Returns one custom resource.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Resource.version": "Version is the object's resourceVersion — the cursor a Watch resumes from.",
|
||||||
|
"ResourceRef.cluster": "Cluster is the registered cluster the object lives on.",
|
||||||
|
"ResourceRef.kind": "Kind is the registered kind.",
|
||||||
|
"ResourceRef.name": "Name is the object's name.",
|
||||||
|
"ResourceRef.namespace": "Namespace is the object's namespace; empty for a cluster-scoped kind.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2"},"status":{"phase":"Running"},"version":"918273"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/resources/watch", zip.Doc{
|
||||||
|
Description: "Returns what changed to one kind since a cursor.\n\nIt is a BOUNDED read rather than a stream, and that is a design decision worth\nstating: a stream cannot cross the call plane as a value, and rendering one —\nServer-Sent Events, a websocket — is the EDGE's concern. A consumer builds its own\nstream by calling this in a loop with the cursor it got back, so the streaming\nshape exists once, in the consumer that renders it, instead of once per transport.\n\n`expired` is the field that matters most. When the cursor is older than the\napiserver's history the watch cannot be resumed, and an empty page would look\nexactly like \"nothing changed\" — which is how a watch-backed board silently stops\nupdating for hours. On `expired` a caller must LIST again and take the new cursor.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Change.type": "Type is \"ADDED\", \"MODIFIED\" or \"DELETED\".",
|
||||||
|
"Resource.version": "Version is the object's resourceVersion — the cursor a Watch resumes from.",
|
||||||
|
"WatchIn.cluster": "Cluster is the registered cluster to watch.",
|
||||||
|
"WatchIn.kind": "Kind is the registered kind.",
|
||||||
|
"WatchIn.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
"WatchIn.version": "Version is the cursor to resume from — the Version a previous list or\nwatch returned. Empty starts from the current state.",
|
||||||
|
"WatchIn.waitSeconds": "WaitSeconds is how long to wait for a change before answering empty.\nClamped to a bound this app owns, so a caller cannot pin a watch open.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","kind":"App","namespace":"acme-prod","version":"918273","waitSeconds":30}`),
|
||||||
|
Response: json.RawMessage(`{"changes":[{"type":"MODIFIED","resource":{"kind":"App","namespace":"acme-prod","name":"api","status":{"phase":"Running"},"version":"918280"}}],"version":"918280"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/volumes/claims", zip.Doc{
|
||||||
|
Description: "Returns PersistentVolumeClaims.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Claim.requestGi": "RequestGi is the size the claim asks for, in GiB.",
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"claims":[{"namespace":"acme-prod","name":"data","phase":"Bound","volume":"pvc-9f2","requestGi":100,"storageClass":"do-block-storage"}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/volumes/persistent", zip.Doc{
|
||||||
|
Description: "Returns the cluster's PersistentVolumes.\n\nA PV is where the link to the real cloud device lives: `handle` is the provider's\nown volume id, read from the CSI source or from the legacy flexVolume shape, and it\nis what lets a reconciliation say whether a disk somebody is paying for is attached\nto anything. The claim reference travels with it because a PV with no claim and a PV\nwhose claim is gone are different situations.\n\nPVs are cluster-scoped, so a namespace-bounded registration cannot read them.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
"Volume.handle": "Handle is the backing device's provider id, when the PV carries one.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"volumes":[{"name":"pvc-9f2","phase":"Bound","handle":"a1b2c3","claimNamespace":"acme-prod","claimName":"data","capacityGi":100}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/workloads/configmaps", zip.Doc{
|
||||||
|
Description: "Returns ConfigMaps, with their data.\n\nThe data travels because a ConfigMap is by definition not secret material. Secrets\nare a different kind and this API does not read one back, at all: an RPC that returns\na tenant's Secret is a credential-exfiltration endpoint however carefully it is\ngated, and material that needs protecting belongs in KMS.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"configMaps":[{"namespace":"hanzo","name":"cron-schedules","data":{"nightly":"0 3 * * *"}}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/workloads/deployments", zip.Doc{
|
||||||
|
Description: "Returns Deployments with their live replica counts and the images\nthey are ACTUALLY running.\n\nThe running image is why this op exists rather than a read of the App CR: a CR's\nstatus does not surface the tag the pods are on, so the live Deployment is the only\nplace \"declared v1.4.2, running v1.4.1\" can be seen at all — which is the whole of\ndrift detection.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod"}`),
|
||||||
|
Response: json.RawMessage(`{"workloads":[{"namespace":"acme-prod","name":"api","kind":"Deployment","replicas":3,"ready":3,"updated":3,"images":["registry.hanzo.ai/acme/api:1.4.2"]}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/workloads/deployments/:namespace/:name", zip.Doc{
|
||||||
|
Description: "Returns one Deployment.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"NamedIn.cluster": "Cluster is the registered cluster the object lives on.",
|
||||||
|
"NamedIn.name": "Name is the object's name.",
|
||||||
|
"NamedIn.namespace": "Namespace is the object's namespace.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"namespace":"acme-prod","name":"api","kind":"Deployment","replicas":3,"ready":3,"updated":3,"images":["registry.hanzo.ai/acme/api:1.4.2"]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/workloads/events", zip.Doc{
|
||||||
|
Description: "Returns cluster events, oldest first.\n\nThis is the only place a failure with no pod behind it can be explained — an\nunschedulable workload, a quota refusal, a volume that never bound — because those\nproduce an event and nothing else. Events are ordered by last-seen so the tail is\nthe current situation.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"events":[{"namespace":"acme-prod","name":"api.17f","type":"Warning","reason":"FailedScheduling","message":"0/6 nodes are available: insufficient nvidia.com/gpu","object":"Pod/api-7d9f-2xk","count":4,"lastSeen":1785110400}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/workloads/ingresses", zip.Doc{
|
||||||
|
Description: "Returns Ingresses with the hosts they answer and which of those are\ncovered by TLS.\n\nThe two host lists are separate because a host in `hosts` but not in `tlsHosts` is a\npublic URL served over plain HTTP, which is a finding rather than a detail.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"ingresses":[{"namespace":"acme-prod","name":"api","hosts":["api.acme.com"],"tlsHosts":["api.acme.com"],"addresses":["203.0.113.10"]}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/workloads/replicasets", zip.Doc{
|
||||||
|
Description: "Returns ReplicaSets.\n\nA deploy tree needs them because they are the layer between a Deployment and its\npods: during a rollout two exist, and which pods belong to which is the difference\nbetween \"the new version is up\" and \"the old one still is\".",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"workloads":[{"namespace":"acme-prod","name":"api-7d9f","kind":"ReplicaSet","replicas":3,"ready":3,"owner":"Deployment/api"}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/workloads/services", zip.Doc{
|
||||||
|
Description: "Returns Services with every identity by which one can be matched to the\nload balancer in front of it.\n\nBoth the cloud-controller's own annotation and the assigned addresses travel, and\neither match is enough. That breadth is the safe direction: matching MORE Services to\na balancer means fewer balancers look orphaned, and the consequence of a wrong match\nin the other direction is deleting a live one.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
"Service.lbId": "LBID is the provider's load-balancer id, from the cloud-controller's own\nannotation — the strongest link between a Service and a real balancer.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"services":[{"namespace":"acme-prod","name":"api","type":"LoadBalancer","clusterIp":"10.3.0.7","ips":["203.0.113.10"],"lbId":"a1b2","ports":[443]}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("GET /v1/k8s/workloads/statefulsets", zip.Doc{
|
||||||
|
Description: "Returns StatefulSets.\n\nThey are read separately from Deployments because they are a different lifetime: a\nStatefulSet's volumeClaimTemplates are immutable, so a storage change means\nrecreating the workload around the claims rather than patching it — and a board that\nfolded the two kinds together would offer an operation that cannot work.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"workloads":[{"namespace":"acme-prod","name":"pg","kind":"StatefulSet","replicas":3,"ready":3,"images":["postgres:17"]}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/access/can", zip.Doc{
|
||||||
|
Description: "Asks a cluster whether this app's own credential may perform one act.\n\nIt is a SelfSubjectAccessReview, which means the answer comes from the cluster's\nRBAC rather than from a grant this app asserts about itself. That distinction is the\nreason the op exists: the union ClusterRole this deployment needs is granted OUT of\ntree, so a drifted or half-applied grant is a real and undetectable condition — and\nasking beats asserting, because a refusal a caller can read is a feature it can\ndegrade gracefully on instead of a 403 arriving in the middle of a mutation.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"CanIn.cluster": "Cluster is the registered cluster to ask.",
|
||||||
|
"CanIn.group": "Group is the API group; empty for the core group.",
|
||||||
|
"CanIn.namespace": "Namespace scopes the question; empty asks cluster-wide.",
|
||||||
|
"CanIn.resource": "Resource is the plural resource name (\"pods\", \"resourcequotas\").",
|
||||||
|
"CanIn.verb": "Verb is the Kubernetes verb: \"get\", \"list\", \"create\", \"patch\", \"delete\", …",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","verb":"get","resource":"resourcequotas","namespace":"acme-prod"}`),
|
||||||
|
Response: json.RawMessage(`{"allowed":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/clusters/deregister", zip.Doc{
|
||||||
|
Description: "Detaches a cluster from the calling org and destroys its sealed\ncredential.\n\nNothing in the cluster is touched: this removes Hanzo's ability to reach it, not\nthe cluster itself. Deleting a name that is not registered reports removed=false\nrather than failing, so a repeated deregistration is safe.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"removed":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/clusters/get", zip.Doc{
|
||||||
|
Description: "Returns one registered cluster.\n\nA name the calling org has not registered answers 404 — including a name another\norg HAS registered, which is indistinguishable from one that does not exist. That\nis deliberate: a distinguishable refusal would let a tenant enumerate other\ntenants' cluster names one guess at a time.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
"Registered.namespaces": "Namespaces bounds what this registration may address. Empty means the\nwhole cluster, which is what an org registering its OWN cluster gets.",
|
||||||
|
"Registered.org": "Org is which org OWNS this cluster. On a reply that is a property of the\nrecord, not a claim by the caller.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"name":"lux-k8s","org":"lux","provider":"byo","endpoint":"https://k8s.lux.example","nodes":12,"nvidiaGpu":32,"amdGpu":0,"registered":"2026-07-30T00:00:00Z","default":false}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/clusters/list", zip.Doc{
|
||||||
|
Description: "Returns every Kubernetes cluster the calling org may reach.\n\nThis is the fleet's ONE cluster list. A row is a real registration — a name, the\nprovider it came from, its apiserver endpoint and the node and accelerator counts\nmeasured when it was attached — and there is no such thing as an unregistered\ncluster this API can see. The credential is never part of a row: it is sealed in\nKMS and resolved only at the moment an op uses it.\n\nThe list is scoped by the caller's org and by nothing the caller can type, so a\ntenant sees its own clusters and no others.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Registered.namespaces": "Namespaces bounds what this registration may address. Empty means the\nwhole cluster, which is what an org registering its OWN cluster gets.",
|
||||||
|
"Registered.org": "Org is which org OWNS this cluster. On a reply that is a property of the\nrecord, not a claim by the caller.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"clusters":[{"name":"hanzo-k8s","org":"hanzo","provider":"doks","endpoint":"https://k8s.example.com","nodes":6,"nvidiaGpu":8,"amdGpu":0,"registered":"2026-07-30T00:00:00Z","default":true}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/clusters/register", zip.Doc{
|
||||||
|
Description: "Attaches a Kubernetes cluster to the calling org and answers 201.\n\nThis is the cluster-registry primitive and the most important operation in this\nAPI: everything else takes a cluster name, and this is where a name comes to\nexist. It is also the whole of BYO-Kubernetes — an org running its own cluster,\nlux running lux-k8s, a customer bringing their own — because a registered cluster\nis a registered cluster whatever provisioned it.\n\nThe kubeconfig is refused if it carries an exec or auth-provider credential\nplugin (either would run a local binary with this process's environment), and\nrefused if its apiserver is not a routable https host. It is then USED, once, to\nprove the cluster answers and to measure its nodes and accelerators; a cluster\nthat cannot be reached is not registered, because the alternative is a row that\nfails at first use when nobody is holding the credential any more. Finally it is\nsealed in KMS under a ref derived from the owning org and the name, and dropped.\nIt is never written to disk, never logged, and no operation returns it.\n\nRegistration is idempotent on name within the org, so re-registering rotates the\ncredential and refreshes the inventory without a second verb.\n\nSet `namespaces` when the credential is a scoped ServiceAccount on a shared\ncluster: the registration is then bounded to those namespaces and every later op\nis refused outside them, before the apiserver is dialed. Omit it when the org owns\nthe whole cluster.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"RegisterIn.default": "Default marks the cluster an org's workloads target when none is named by\na human — the ONE place a default is legitimate, because it is the org's\nown recorded choice rather than a process's ambient neighbour.",
|
||||||
|
"RegisterIn.kubeconfig": "Kubeconfig is the credential, in kubeconfig form. It is validated, used\nonce to prove the cluster is reachable, sealed in KMS, and then dropped:\nit is never written to disk, never logged and never returned.",
|
||||||
|
"RegisterIn.name": "Name is what every later op will call this cluster. Unique within the org.",
|
||||||
|
"RegisterIn.namespaces": "Namespaces bounds this registration to a namespace set. Omit it when the\norg owns the whole cluster; state it when the credential is a scoped\nServiceAccount on a shared cluster.",
|
||||||
|
"RegisterIn.provider": "Provider labels where the cluster came from (\"doks\", \"byo\", \"k3s\", …).",
|
||||||
|
"Registered.namespaces": "Namespaces bounds what this registration may address. Empty means the\nwhole cluster, which is what an org registering its OWN cluster gets.",
|
||||||
|
"Registered.org": "Org is which org OWNS this cluster. On a reply that is a property of the\nrecord, not a claim by the caller.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"name":"lux-k8s","kubeconfig":"apiVersion: v1\nkind: Config\n...","provider":"byo","default":true}`),
|
||||||
|
Response: json.RawMessage(`{"name":"lux-k8s","org":"lux","provider":"byo","endpoint":"https://k8s.lux.example","nodes":12,"nvidiaGpu":32,"amdGpu":0,"registered":"2026-07-30T00:00:00Z","default":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/clusters/status", zip.Doc{
|
||||||
|
Description: "Reports whether a registered cluster answers right now.\n\nThe probe is the apiserver's version endpoint, because that is the one read every\ncredential can make regardless of what RBAC it was granted: a registration bounded\nto one namespace would fail a namespace list and look dead while being perfectly\nhealthy. So this answers \"is the credential live and the apiserver reachable\",\nwhich is the question a board asks, and the ops that need a permission ask for\nthat permission themselves.\n\nA cluster that does not answer is connected=false with the reason verbatim — never\nan error, because \"this cluster is down\" is a successful answer to this question.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
"Reachable.connected": "Connected is whether the apiserver answered.",
|
||||||
|
"Reachable.reason": "Reason is why it did not, verbatim. Absent on success.",
|
||||||
|
"Reachable.version": "Version is the server version string, when it answered.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"connected":true,"version":"v1.33.1"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/jobs/create", zip.Doc{
|
||||||
|
Description: "Runs one batch Job and answers 201.\n\nThe input is a JOB, not a PodSpec: an image, a command, environment, a service\naccount, labels, and claims to mount. That is the whole of what the fleet's job\ncallers need — an image build, a migration, a scheduled task — and it is deliberately\nnot the Kubernetes PodSpec, because accepting a PodSpec would mean accepting\nhostPath mounts, host networking, privileged security contexts and secret volume\nreferences, i.e. everything a bounded surface exists to exclude.\n\n`env` is PLAIN environment. Secret material must not travel here: seal it in KMS and\ngive the job a ServiceAccount that can reach what it needs. `ttlSeconds` is worth\nsetting on anything periodic — a namespace that keeps every finished Job becomes a\nlist nobody can read.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Job.phase": "Phase is \"running\", \"succeeded\" or \"failed\" — the one word a caller wants,\nrolled here rather than re-derived from three counters by every consumer.",
|
||||||
|
"JobIn.args": "Args are the arguments passed to the entrypoint.",
|
||||||
|
"JobIn.backoffLimit": "BackoffLimit caps retries; zero means the Kubernetes default.",
|
||||||
|
"JobIn.cluster": "Cluster is the registered cluster to run the job on.",
|
||||||
|
"JobIn.command": "Command replaces the image's entrypoint. Empty keeps it.",
|
||||||
|
"JobIn.env": "Env is plain environment. Secret material does NOT belong here: seal it in\nKMS and project it with a scoped ServiceAccount instead.",
|
||||||
|
"JobIn.image": "Image is the container image to run.",
|
||||||
|
"JobIn.labels": "Labels are stamped on the Job and its pods.",
|
||||||
|
"JobIn.mounts": "Mounts attaches existing claims by name at a path.",
|
||||||
|
"JobIn.name": "Name is the job's name.",
|
||||||
|
"JobIn.namespace": "Namespace is where the job runs.",
|
||||||
|
"JobIn.serviceAccount": "ServiceAccount runs the pod as a named ServiceAccount.",
|
||||||
|
"JobIn.ttlSeconds": "TTLSeconds deletes the finished Job after this long. Zero leaves it, which\non a busy namespace is how a Job list becomes unreadable.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"hanzo","name":"build-1a2b","image":"gcr.io/kaniko-project/executor:latest","args":["--context=git://…","--destination=registry.hanzo.ai/acme/api:1.4.2"],"ttlSeconds":3600}`),
|
||||||
|
Response: json.RawMessage(`{"namespace":"hanzo","name":"build-1a2b","active":1,"succeeded":0,"failed":0,"phase":"running"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/jobs/get", zip.Doc{
|
||||||
|
Description: "Returns one batch Job.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Job.phase": "Phase is \"running\", \"succeeded\" or \"failed\" — the one word a caller wants,\nrolled here rather than re-derived from three counters by every consumer.",
|
||||||
|
"NamedIn.cluster": "Cluster is the registered cluster the object lives on.",
|
||||||
|
"NamedIn.name": "Name is the object's name.",
|
||||||
|
"NamedIn.namespace": "Namespace is the object's namespace.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"namespace":"hanzo","name":"build-1a2b","active":0,"succeeded":1,"failed":0,"phase":"succeeded","startedAt":1785110400,"endedAt":1785110460}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/jobs/list", zip.Doc{
|
||||||
|
Description: "Returns batch Jobs with a rolled-up phase.\n\n`phase` is one word — running, succeeded or failed — computed here rather than left\nto every caller to derive from three counters. Three consumers deriving it three ways\nis three chances to read a failed job as merely unfinished.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Job.phase": "Phase is \"running\", \"succeeded\" or \"failed\" — the one word a caller wants,\nrolled here rather than re-derived from three counters by every consumer.",
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"jobs":[{"namespace":"hanzo","name":"build-1a2b","active":0,"succeeded":1,"failed":0,"phase":"succeeded","startedAt":1785110400,"endedAt":1785110460}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/metrics/pods", zip.Doc{
|
||||||
|
Description: "Returns live CPU and memory use for pods, from metrics.k8s.io.\n\n`available` is the field that carries the honesty. metrics-server is absent from most\nof our clusters, and a cluster without it answers this call with an error — which,\nrendered as zeroes, tells an operator that a busy namespace is idle. So a missing\nmetrics API is reported as available=false with no rows, and a caller must render\n\"unmeasured\" rather than \"0m\".",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"PodMetric.cpuMilli": "CPUMilli is CPU use in millicores; MemoryBytes is working-set memory.",
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod"}`),
|
||||||
|
Response: json.RawMessage(`{"pods":[{"namespace":"acme-prod","name":"api-7d9f-2xk","cpuMilli":250,"memoryBytes":536870912}],"available":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/namespaces/delete", zip.Doc{
|
||||||
|
Description: "Deletes a namespace and everything in it.\n\nThis is the highest blast radius in the whole API: Kubernetes deletes a namespace\nby garbage-collecting every object inside it, including PersistentVolumeClaims,\nwhich for a Delete-reclaim StorageClass destroys the tenant's data. There is no\nundo and no confirmation this API can offer that would make one.\n\nIt is therefore its own named operation, gated separately from every other write,\nand it is refused outright for a registration bounded to a namespace set: a scoped\ncredential exists precisely because the cluster is shared, and \"delete a namespace\non a shared cluster\" is not an authority a tenant holds.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"NamespaceRef.cluster": "Cluster is the registered cluster the namespace lives on.",
|
||||||
|
"NamespaceRef.namespace": "Namespace is the namespace's name.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"removed":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/namespaces/ensure", zip.Doc{
|
||||||
|
Description: "Creates a namespace if it is absent, and reports which it did.\n\nFour separate implementations of get-then-create-if-missing collapse into this one\nname. That matters beyond tidiness: each of those four wrote the ladder itself, and\na ladder is a race — \"get said absent\" and \"create\" are two calls, and two callers\nstarting a tenant at once both take the create branch. Here an AlreadyExists from\nthe apiserver is the successful \"it was already there\" answer, so the race has one\noutcome instead of one winner and one error.\n\n`created` is reported because creating a tenant's namespace and finding it are\ndifferent events, and a caller that cannot tell them apart cannot log the first.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"EnsureNamespaceIn.cluster": "Cluster is the registered cluster to create the namespace on.",
|
||||||
|
"EnsureNamespaceIn.labels": "Labels are merged onto the namespace when it is created.",
|
||||||
|
"EnsureNamespaceIn.namespace": "Namespace is the namespace's name.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod","labels":{"tier":"pro"}}`),
|
||||||
|
Response: json.RawMessage(`{"name":"acme-prod","created":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/namespaces/get", zip.Doc{
|
||||||
|
Description: "Returns one namespace.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"NamespaceRef.cluster": "Cluster is the registered cluster the namespace lives on.",
|
||||||
|
"NamespaceRef.namespace": "Namespace is the namespace's name.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"name":"acme-prod","phase":"Active","labels":{"managed-by":"hanzo-cloud"}}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/namespaces/list", zip.Doc{
|
||||||
|
Description: "Returns the namespaces the calling org's registration may address.\n\nFor a registration that owns the whole cluster this is every namespace on it. For\none bounded to a namespace set it is that set, read one namespace at a time — so a\nscoped tenant learns nothing about its neighbours, not even how many there are.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"namespaces":[{"name":"acme-prod","phase":"Active","labels":{"managed-by":"hanzo-cloud"}}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/nodes/cordon", zip.Doc{
|
||||||
|
Description: "Takes a node out of service, or puts it back, and optionally drains it.\n\nThis is ONE named operation over what is really three calls — patch\n`spec.unschedulable`, list the node's pods, evict each one — because the three are\nonly ever wanted together and a generic \"patch a node\" op would let a caller write\nany field of any node's spec. There is deliberately no such op.\n\nThe drain uses the EVICTION api, not delete: an eviction respects\nPodDisruptionBudgets, so a drain that would break a quorum is REFUSED by the\napiserver and reported verbatim, with the node left cordoned so an operator can\nscale and retry. DaemonSet and static (mirror) pods are skipped — the node\nrecreates them immediately, so evicting them is a loop — and pods already\nterminal are skipped too.\n\n`drain` is ignored when `schedulable` is true, because draining a node you just\nreturned to service is incoherent rather than merely useless.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"CordonIn.cluster": "Cluster is the registered cluster the node belongs to.",
|
||||||
|
"CordonIn.drain": "Drain evicts the node's pods after cordoning. Ignored when Schedulable is\ntrue, because draining a node you just returned to service is incoherent.",
|
||||||
|
"CordonIn.node": "Node is the node's name.",
|
||||||
|
"CordonIn.schedulable": "Schedulable is the state to leave the node in: false cordons it, true\nuncordons it.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","node":"pool-gpu-a1","schedulable":false,"drain":true}`),
|
||||||
|
Response: json.RawMessage(`{"evicted":11}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/nodes/list", zip.Doc{
|
||||||
|
Description: "Returns a cluster's worker nodes: whether each is up, whether it will\ntake new work, and what accelerators it offers.\n\nThis is the one node inventory in the fleet. The GPU counts come from each node's\nallocatable `nvidia.com/gpu` and `amd.com/gpu`, which is what the scheduler will\nactually honour — not a label somebody typed.\n\nNodes are cluster-scoped, so a registration bounded to a namespace set cannot read\nthem: answering would hand one tenant on a shared cluster the whole cluster's\ncapacity.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"nodes":[{"name":"pool-gpu-a1","ready":true,"schedulable":true,"region":"nyc3","instanceKind":"gpu-h100x8-640gb","cpu":"64","memory":"755Gi","nvidiaGpu":8,"amdGpu":0,"internalIp":"10.0.0.4"}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/nodes/volume-stats", zip.Doc{
|
||||||
|
Description: "Reports how full every mounted volume on a cluster is.\n\nIt is the ONLY source of volume fill we have. Cloud providers do not expose how\nfull a disk is, and metrics-server is absent from most of our clusters — but every\nkubelet serves `stats/summary` unconditionally, so this works everywhere. The read\ngoes through the apiserver's node proxy, which is why this is a SEPARATE named op:\n`nodes/proxy get` is effectively arbitrary kubelet access, and naming it here is\nwhat lets every other operation's grant omit it.\n\nA kubelet that will not answer costs a reading and never the call: fill is\nadvisory, no safety verdict depends on it, and failing the whole sweep over one\nsilent node would block every volume decision on the fleet. `nodesRead` against\n`nodesTotal` is how a caller knows the difference between \"empty\" and\n\"unmeasured\" — a claim with no row is unmeasured, and treating that as empty is\nhow live data gets condemned.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
"VolumeStats.nodesRead": "NodesRead is how many kubelets answered; NodesTotal how many were asked.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"volumes":[{"namespace":"acme","name":"data","usedBytes":48318382080}],"nodesRead":6,"nodesTotal":6}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/pods/list", zip.Doc{
|
||||||
|
Description: "Returns pods, narrowed by namespace, labels or node.\n\nOne op covers every way the fleet asks for pods: the whole-cluster sweep an\ninfrastructure board takes, the label-selected set that answers \"is this app's\nworkload up\", and the single-node set a drain needs. They were three call sites\nwith three shapes; the difference between them is which argument is set.\n\nContainer ENV is deliberately absent from the projection. A pod spec carries the\nsecret material injected into it, so a read surface that returned the spec would be\nan exfiltration door wearing the name of a health check.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"PodsIn.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"PodsIn.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"PodsIn.namespace": "Namespace narrows to one namespace. Empty means every namespace this\nregistration may address, which for a bounded registration is its own set\nand never the whole cluster.",
|
||||||
|
"PodsIn.nodeName": "NodeName narrows to the pods on one node — the field-selector variant the\ndrain path needs, named rather than left to a raw selector string.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod","labelSelector":"app=api"}`),
|
||||||
|
Response: json.RawMessage(`{"pods":[{"namespace":"acme-prod","name":"api-7d9f-2xk","phase":"Running","node":"pool-a1","controller":"ReplicaSet/api-7d9f","images":["registry.hanzo.ai/acme/api:1.4.2"],"restarts":0}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/pods/logs", zip.Doc{
|
||||||
|
Description: "Returns what one container printed.\n\n`tailLines` is the bound a caller should set; without one the read is capped at one\nmebibyte and truncated from the FRONT, so the newest output — the part that\nexplains a crash — is what survives.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"LogsIn.cluster": "Cluster is the registered cluster the pod runs on.",
|
||||||
|
"LogsIn.container": "Container names the container; empty takes the pod's only one.",
|
||||||
|
"LogsIn.namespace": "Namespace is the pod's namespace.",
|
||||||
|
"LogsIn.pod": "Pod is the pod's name.",
|
||||||
|
"LogsIn.tailLines": "TailLines caps how many lines come back, newest last. Zero takes the\nserver's default. A log read is unbounded by nature, so this is the bound.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod","pod":"api-7d9f-2xk","tailLines":200}`),
|
||||||
|
Response: json.RawMessage(`{"log":"2026-07-30T00:00:01Z listening on :8000\n"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/resources/apply", zip.Doc{
|
||||||
|
Description: "Creates one custom resource of a registered kind, or patches the one\nalready there.\n\nFive separate get-then-create-else-patch ladders in the fleet collapse into this one\nname. Each of those wrote the ladder itself and each was a race; here the apiserver\nresolves the race, because a create that loses falls through to the patch.\n\nThe `spec` is the caller's domain payload and is applied whole — its SHAPE belongs to\nwhoever authored the CRD, not to this app, which would otherwise have to be edited\nevery time any kind grew a field. What this app owns is the KIND and the VERB, and\nboth are named and typed. That is the line: `kind` is a key in a closed table, so\nthis cannot become \"apply any object the cluster has\".",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ApplyIn.cluster": "Cluster is the registered cluster to apply to.",
|
||||||
|
"ApplyIn.kind": "Kind is the registered kind.",
|
||||||
|
"ApplyIn.labels": "Labels are merged onto the object.",
|
||||||
|
"ApplyIn.name": "Name is the object's name.",
|
||||||
|
"ApplyIn.namespace": "Namespace is where the object lives; empty for a cluster-scoped kind.",
|
||||||
|
"ApplyIn.spec": "Spec is the desired spec, applied whole.",
|
||||||
|
"Resource.version": "Version is the object's resourceVersion — the cursor a Watch resumes from.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2","replicas":3}}`),
|
||||||
|
Response: json.RawMessage(`{"kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2","replicas":3},"version":"918281"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/resources/delete", zip.Doc{
|
||||||
|
Description: "Deletes one custom resource of a registered kind.\n\nDeleting an object is not deleting its data: a workload's App CR is desired state and\ncan be recreated freely, while the PersistentVolumeClaim it mounted holds the only\ncopy of the tenant's data and is a different lifetime. Nothing here deletes a claim.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ResourceRef.cluster": "Cluster is the registered cluster the object lives on.",
|
||||||
|
"ResourceRef.kind": "Kind is the registered kind.",
|
||||||
|
"ResourceRef.name": "Name is the object's name.",
|
||||||
|
"ResourceRef.namespace": "Namespace is the object's namespace; empty for a cluster-scoped kind.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"removed":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/resources/get", zip.Doc{
|
||||||
|
Description: "Returns one custom resource.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Resource.version": "Version is the object's resourceVersion — the cursor a Watch resumes from.",
|
||||||
|
"ResourceRef.cluster": "Cluster is the registered cluster the object lives on.",
|
||||||
|
"ResourceRef.kind": "Kind is the registered kind.",
|
||||||
|
"ResourceRef.name": "Name is the object's name.",
|
||||||
|
"ResourceRef.namespace": "Namespace is the object's namespace; empty for a cluster-scoped kind.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2"},"status":{"phase":"Running"},"version":"918273"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/resources/list", zip.Doc{
|
||||||
|
Description: "Returns custom resources of one registered kind.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Resource.version": "Version is the object's resourceVersion — the cursor a Watch resumes from.",
|
||||||
|
"ResourceIn.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"ResourceIn.kind": "Kind is the registered kind: \"App\", \"Application\", \"Datastore\",\n\"Validator\", \"TrainJob\", \"InferenceService\", \"IngressRoute\", \"Middleware\"\nor \"AppProject\".",
|
||||||
|
"ResourceIn.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"ResourceIn.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address. Ignored for a cluster-scoped kind.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","kind":"App","namespace":"acme-prod"}`),
|
||||||
|
Response: json.RawMessage(`{"resources":[{"kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2"},"status":{"phase":"Running"},"version":"918273"}],"version":"918273"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/resources/watch", zip.Doc{
|
||||||
|
Description: "Returns what changed to one kind since a cursor.\n\nIt is a BOUNDED read rather than a stream, and that is a design decision worth\nstating: a stream cannot cross the call plane as a value, and rendering one —\nServer-Sent Events, a websocket — is the EDGE's concern. A consumer builds its own\nstream by calling this in a loop with the cursor it got back, so the streaming\nshape exists once, in the consumer that renders it, instead of once per transport.\n\n`expired` is the field that matters most. When the cursor is older than the\napiserver's history the watch cannot be resumed, and an empty page would look\nexactly like \"nothing changed\" — which is how a watch-backed board silently stops\nupdating for hours. On `expired` a caller must LIST again and take the new cursor.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Change.type": "Type is \"ADDED\", \"MODIFIED\" or \"DELETED\".",
|
||||||
|
"Resource.version": "Version is the object's resourceVersion — the cursor a Watch resumes from.",
|
||||||
|
"WatchIn.cluster": "Cluster is the registered cluster to watch.",
|
||||||
|
"WatchIn.kind": "Kind is the registered kind.",
|
||||||
|
"WatchIn.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
"WatchIn.version": "Version is the cursor to resume from — the Version a previous list or\nwatch returned. Empty starts from the current state.",
|
||||||
|
"WatchIn.waitSeconds": "WaitSeconds is how long to wait for a change before answering empty.\nClamped to a bound this app owns, so a caller cannot pin a watch open.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","kind":"App","namespace":"acme-prod","version":"918273","waitSeconds":30}`),
|
||||||
|
Response: json.RawMessage(`{"changes":[{"type":"MODIFIED","resource":{"kind":"App","namespace":"acme-prod","name":"api","status":{"phase":"Running"},"version":"918280"}}],"version":"918280"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/volumes/ensure-claim", zip.Doc{
|
||||||
|
Description: "Creates a PersistentVolumeClaim if it is absent.\n\nIdempotent on name in the namespace, for the reason ensureNamespace is: the\nget-then-create ladder is a race when each caller writes it.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"EnsureClaimIn.accessMode": "AccessMode is \"ReadWriteOnce\" (default) or \"ReadWriteMany\".",
|
||||||
|
"EnsureClaimIn.cluster": "Cluster is the registered cluster to create the claim on.",
|
||||||
|
"EnsureClaimIn.gi": "Gi is the storage to request, in GiB.",
|
||||||
|
"EnsureClaimIn.name": "Name is the claim's name.",
|
||||||
|
"EnsureClaimIn.namespace": "Namespace is where the claim lives.",
|
||||||
|
"EnsureClaimIn.storageClass": "StorageClass names the class; empty takes the cluster's default.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod","name":"data","gi":100,"storageClass":"do-block-storage"}`),
|
||||||
|
Response: json.RawMessage(`{"name":"data","created":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/volumes/expand-claim", zip.Doc{
|
||||||
|
Description: "Grows a PersistentVolumeClaim to a new size.\n\nThis is the ONE correct way to grow a volume Kubernetes manages, and the reason it\nis a named op rather than a patch: the resize controller acts on the CLAIM, growing\nthe backing cloud device and then the filesystem on it, so claim, PersistentVolume,\ndevice and filesystem all end up agreeing. Growing the device through the cloud\nprovider's own API instead leaves the claim and the PV declaring the old capacity\nand the filesystem never grown at all — three sources of truth, two of them wrong.\n\nThe StorageClass must set `allowVolumeExpansion`; when it does not, the apiserver\nrefuses and its message is surfaced verbatim rather than worked around. A shrink is\nrefused by the apiserver too, and there is no op that deletes a claim: the claim\nholds the only copy of the tenant's data, so this app can make one bigger and\nnothing else.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ExpandClaimIn.cluster": "Cluster is the registered cluster the claim lives on.",
|
||||||
|
"ExpandClaimIn.gi": "Gi is the new size in GiB. It must exceed the current request; the\napiserver refuses a shrink and its message is surfaced verbatim.",
|
||||||
|
"ExpandClaimIn.name": "Name is the claim's name.",
|
||||||
|
"ExpandClaimIn.namespace": "Namespace is the claim's namespace.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod","name":"data","gi":200}`),
|
||||||
|
Response: json.RawMessage(`{"gi":200}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/volumes/list-claims", zip.Doc{
|
||||||
|
Description: "Returns PersistentVolumeClaims.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Claim.requestGi": "RequestGi is the size the claim asks for, in GiB.",
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"claims":[{"namespace":"acme-prod","name":"data","phase":"Bound","volume":"pvc-9f2","requestGi":100,"storageClass":"do-block-storage"}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/volumes/list-persistent", zip.Doc{
|
||||||
|
Description: "Returns the cluster's PersistentVolumes.\n\nA PV is where the link to the real cloud device lives: `handle` is the provider's\nown volume id, read from the CSI source or from the legacy flexVolume shape, and it\nis what lets a reconciliation say whether a disk somebody is paying for is attached\nto anything. The claim reference travels with it because a PV with no claim and a PV\nwhose claim is gone are different situations.\n\nPVs are cluster-scoped, so a namespace-bounded registration cannot read them.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ClusterRef.cluster": "Cluster is the registered cluster's name — \"hanzo-k8s\", \"lux-k8s\",\n\"zoo-k8s\", \"bootnode-k8s\", or any name an org registered its own cluster\nunder. It is resolved against the CALLING ORG's registry, so two orgs may\nuse the same name for two different clusters and neither can reach the\nother's.",
|
||||||
|
"Volume.handle": "Handle is the backing device's provider id, when the PV carries one.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"volumes":[{"name":"pvc-9f2","phase":"Bound","handle":"a1b2c3","claimNamespace":"acme-prod","claimName":"data","capacityGi":100}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/workloads/get-deployment", zip.Doc{
|
||||||
|
Description: "Returns one Deployment.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"NamedIn.cluster": "Cluster is the registered cluster the object lives on.",
|
||||||
|
"NamedIn.name": "Name is the object's name.",
|
||||||
|
"NamedIn.namespace": "Namespace is the object's namespace.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"namespace":"acme-prod","name":"api","kind":"Deployment","replicas":3,"ready":3,"updated":3,"images":["registry.hanzo.ai/acme/api:1.4.2"]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/workloads/list-configmaps", zip.Doc{
|
||||||
|
Description: "Returns ConfigMaps, with their data.\n\nThe data travels because a ConfigMap is by definition not secret material. Secrets\nare a different kind and this API does not read one back, at all: an RPC that returns\na tenant's Secret is a credential-exfiltration endpoint however carefully it is\ngated, and material that needs protecting belongs in KMS.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"configMaps":[{"namespace":"hanzo","name":"cron-schedules","data":{"nightly":"0 3 * * *"}}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/workloads/list-deployments", zip.Doc{
|
||||||
|
Description: "Returns Deployments with their live replica counts and the images\nthey are ACTUALLY running.\n\nThe running image is why this op exists rather than a read of the App CR: a CR's\nstatus does not surface the tag the pods are on, so the live Deployment is the only\nplace \"declared v1.4.2, running v1.4.1\" can be seen at all — which is the whole of\ndrift detection.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod"}`),
|
||||||
|
Response: json.RawMessage(`{"workloads":[{"namespace":"acme-prod","name":"api","kind":"Deployment","replicas":3,"ready":3,"updated":3,"images":["registry.hanzo.ai/acme/api:1.4.2"]}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/workloads/list-events", zip.Doc{
|
||||||
|
Description: "Returns cluster events, oldest first.\n\nThis is the only place a failure with no pod behind it can be explained — an\nunschedulable workload, a quota refusal, a volume that never bound — because those\nproduce an event and nothing else. Events are ordered by last-seen so the tail is\nthe current situation.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"events":[{"namespace":"acme-prod","name":"api.17f","type":"Warning","reason":"FailedScheduling","message":"0/6 nodes are available: insufficient nvidia.com/gpu","object":"Pod/api-7d9f-2xk","count":4,"lastSeen":1785110400}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/workloads/list-ingresses", zip.Doc{
|
||||||
|
Description: "Returns Ingresses with the hosts they answer and which of those are\ncovered by TLS.\n\nThe two host lists are separate because a host in `hosts` but not in `tlsHosts` is a\npublic URL served over plain HTTP, which is a finding rather than a detail.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"ingresses":[{"namespace":"acme-prod","name":"api","hosts":["api.acme.com"],"tlsHosts":["api.acme.com"],"addresses":["203.0.113.10"]}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/workloads/list-replicasets", zip.Doc{
|
||||||
|
Description: "Returns ReplicaSets.\n\nA deploy tree needs them because they are the layer between a Deployment and its\npods: during a rollout two exist, and which pods belong to which is the difference\nbetween \"the new version is up\" and \"the old one still is\".",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"workloads":[{"namespace":"acme-prod","name":"api-7d9f","kind":"ReplicaSet","replicas":3,"ready":3,"owner":"Deployment/api"}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/workloads/list-services", zip.Doc{
|
||||||
|
Description: "Returns Services with every identity by which one can be matched to the\nload balancer in front of it.\n\nBoth the cloud-controller's own annotation and the assigned addresses travel, and\neither match is enough. That breadth is the safe direction: matching MORE Services to\na balancer means fewer balancers look orphaned, and the consequence of a wrong match\nin the other direction is deleting a live one.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
"Service.lbId": "LBID is the provider's load-balancer id, from the cloud-controller's own\nannotation — the strongest link between a Service and a real balancer.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"services":[{"namespace":"acme-prod","name":"api","type":"LoadBalancer","clusterIp":"10.3.0.7","ips":["203.0.113.10"],"lbId":"a1b2","ports":[443]}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /k8s/workloads/list-statefulsets", zip.Doc{
|
||||||
|
Description: "Returns StatefulSets.\n\nThey are read separately from Deployments because they are a different lifetime: a\nStatefulSet's volumeClaimTemplates are immutable, so a storage change means\nrecreating the workload around the claims rather than patching it — and a board that\nfolded the two kinds together would offer an operation that cannot work.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Selector.cluster": "Cluster is the registered cluster to read.",
|
||||||
|
"Selector.labelSelector": "LabelSelector is a Kubernetes label selector, verbatim.",
|
||||||
|
"Selector.namespace": "Namespace narrows to one namespace; empty means every namespace this\nregistration may address.",
|
||||||
|
},
|
||||||
|
Response: json.RawMessage(`{"workloads":[{"namespace":"acme-prod","name":"pg","kind":"StatefulSet","replicas":3,"ready":3,"images":["postgres:17"]}]}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /v1/k8s/access/can", zip.Doc{
|
||||||
|
Description: "Asks a cluster whether this app's own credential may perform one act.\n\nIt is a SelfSubjectAccessReview, which means the answer comes from the cluster's\nRBAC rather than from a grant this app asserts about itself. That distinction is the\nreason the op exists: the union ClusterRole this deployment needs is granted OUT of\ntree, so a drifted or half-applied grant is a real and undetectable condition — and\nasking beats asserting, because a refusal a caller can read is a feature it can\ndegrade gracefully on instead of a 403 arriving in the middle of a mutation.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"CanIn.cluster": "Cluster is the registered cluster to ask.",
|
||||||
|
"CanIn.group": "Group is the API group; empty for the core group.",
|
||||||
|
"CanIn.namespace": "Namespace scopes the question; empty asks cluster-wide.",
|
||||||
|
"CanIn.resource": "Resource is the plural resource name (\"pods\", \"resourcequotas\").",
|
||||||
|
"CanIn.verb": "Verb is the Kubernetes verb: \"get\", \"list\", \"create\", \"patch\", \"delete\", …",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","verb":"get","resource":"resourcequotas","namespace":"acme-prod"}`),
|
||||||
|
Response: json.RawMessage(`{"allowed":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /v1/k8s/clusters", zip.Doc{
|
||||||
|
Description: "Attaches a Kubernetes cluster to the calling org and answers 201.\n\nThis is the cluster-registry primitive and the most important operation in this\nAPI: everything else takes a cluster name, and this is where a name comes to\nexist. It is also the whole of BYO-Kubernetes — an org running its own cluster,\nlux running lux-k8s, a customer bringing their own — because a registered cluster\nis a registered cluster whatever provisioned it.\n\nThe kubeconfig is refused if it carries an exec or auth-provider credential\nplugin (either would run a local binary with this process's environment), and\nrefused if its apiserver is not a routable https host. It is then USED, once, to\nprove the cluster answers and to measure its nodes and accelerators; a cluster\nthat cannot be reached is not registered, because the alternative is a row that\nfails at first use when nobody is holding the credential any more. Finally it is\nsealed in KMS under a ref derived from the owning org and the name, and dropped.\nIt is never written to disk, never logged, and no operation returns it.\n\nRegistration is idempotent on name within the org, so re-registering rotates the\ncredential and refreshes the inventory without a second verb.\n\nSet `namespaces` when the credential is a scoped ServiceAccount on a shared\ncluster: the registration is then bounded to those namespaces and every later op\nis refused outside them, before the apiserver is dialed. Omit it when the org owns\nthe whole cluster.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"RegisterIn.default": "Default marks the cluster an org's workloads target when none is named by\na human — the ONE place a default is legitimate, because it is the org's\nown recorded choice rather than a process's ambient neighbour.",
|
||||||
|
"RegisterIn.kubeconfig": "Kubeconfig is the credential, in kubeconfig form. It is validated, used\nonce to prove the cluster is reachable, sealed in KMS, and then dropped:\nit is never written to disk, never logged and never returned.",
|
||||||
|
"RegisterIn.name": "Name is what every later op will call this cluster. Unique within the org.",
|
||||||
|
"RegisterIn.namespaces": "Namespaces bounds this registration to a namespace set. Omit it when the\norg owns the whole cluster; state it when the credential is a scoped\nServiceAccount on a shared cluster.",
|
||||||
|
"RegisterIn.provider": "Provider labels where the cluster came from (\"doks\", \"byo\", \"k3s\", …).",
|
||||||
|
"Registered.namespaces": "Namespaces bounds what this registration may address. Empty means the\nwhole cluster, which is what an org registering its OWN cluster gets.",
|
||||||
|
"Registered.org": "Org is which org OWNS this cluster. On a reply that is a property of the\nrecord, not a claim by the caller.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"name":"lux-k8s","kubeconfig":"apiVersion: v1\nkind: Config\n...","provider":"byo","default":true}`),
|
||||||
|
Response: json.RawMessage(`{"name":"lux-k8s","org":"lux","provider":"byo","endpoint":"https://k8s.lux.example","nodes":12,"nvidiaGpu":32,"amdGpu":0,"registered":"2026-07-30T00:00:00Z","default":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /v1/k8s/jobs", zip.Doc{
|
||||||
|
Description: "Runs one batch Job and answers 201.\n\nThe input is a JOB, not a PodSpec: an image, a command, environment, a service\naccount, labels, and claims to mount. That is the whole of what the fleet's job\ncallers need — an image build, a migration, a scheduled task — and it is deliberately\nnot the Kubernetes PodSpec, because accepting a PodSpec would mean accepting\nhostPath mounts, host networking, privileged security contexts and secret volume\nreferences, i.e. everything a bounded surface exists to exclude.\n\n`env` is PLAIN environment. Secret material must not travel here: seal it in KMS and\ngive the job a ServiceAccount that can reach what it needs. `ttlSeconds` is worth\nsetting on anything periodic — a namespace that keeps every finished Job becomes a\nlist nobody can read.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"Job.phase": "Phase is \"running\", \"succeeded\" or \"failed\" — the one word a caller wants,\nrolled here rather than re-derived from three counters by every consumer.",
|
||||||
|
"JobIn.args": "Args are the arguments passed to the entrypoint.",
|
||||||
|
"JobIn.backoffLimit": "BackoffLimit caps retries; zero means the Kubernetes default.",
|
||||||
|
"JobIn.cluster": "Cluster is the registered cluster to run the job on.",
|
||||||
|
"JobIn.command": "Command replaces the image's entrypoint. Empty keeps it.",
|
||||||
|
"JobIn.env": "Env is plain environment. Secret material does NOT belong here: seal it in\nKMS and project it with a scoped ServiceAccount instead.",
|
||||||
|
"JobIn.image": "Image is the container image to run.",
|
||||||
|
"JobIn.labels": "Labels are stamped on the Job and its pods.",
|
||||||
|
"JobIn.mounts": "Mounts attaches existing claims by name at a path.",
|
||||||
|
"JobIn.name": "Name is the job's name.",
|
||||||
|
"JobIn.namespace": "Namespace is where the job runs.",
|
||||||
|
"JobIn.serviceAccount": "ServiceAccount runs the pod as a named ServiceAccount.",
|
||||||
|
"JobIn.ttlSeconds": "TTLSeconds deletes the finished Job after this long. Zero leaves it, which\non a busy namespace is how a Job list becomes unreadable.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"hanzo","name":"build-1a2b","image":"gcr.io/kaniko-project/executor:latest","args":["--context=git://…","--destination=registry.hanzo.ai/acme/api:1.4.2"],"ttlSeconds":3600}`),
|
||||||
|
Response: json.RawMessage(`{"namespace":"hanzo","name":"build-1a2b","active":1,"succeeded":0,"failed":0,"phase":"running"}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /v1/k8s/namespaces", zip.Doc{
|
||||||
|
Description: "Creates a namespace if it is absent, and reports which it did.\n\nFour separate implementations of get-then-create-if-missing collapse into this one\nname. That matters beyond tidiness: each of those four wrote the ladder itself, and\na ladder is a race — \"get said absent\" and \"create\" are two calls, and two callers\nstarting a tenant at once both take the create branch. Here an AlreadyExists from\nthe apiserver is the successful \"it was already there\" answer, so the race has one\noutcome instead of one winner and one error.\n\n`created` is reported because creating a tenant's namespace and finding it are\ndifferent events, and a caller that cannot tell them apart cannot log the first.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"EnsureNamespaceIn.cluster": "Cluster is the registered cluster to create the namespace on.",
|
||||||
|
"EnsureNamespaceIn.labels": "Labels are merged onto the namespace when it is created.",
|
||||||
|
"EnsureNamespaceIn.namespace": "Namespace is the namespace's name.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod","labels":{"tier":"pro"}}`),
|
||||||
|
Response: json.RawMessage(`{"name":"acme-prod","created":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /v1/k8s/nodes/:node/cordon", zip.Doc{
|
||||||
|
Description: "Takes a node out of service, or puts it back, and optionally drains it.\n\nThis is ONE named operation over what is really three calls — patch\n`spec.unschedulable`, list the node's pods, evict each one — because the three are\nonly ever wanted together and a generic \"patch a node\" op would let a caller write\nany field of any node's spec. There is deliberately no such op.\n\nThe drain uses the EVICTION api, not delete: an eviction respects\nPodDisruptionBudgets, so a drain that would break a quorum is REFUSED by the\napiserver and reported verbatim, with the node left cordoned so an operator can\nscale and retry. DaemonSet and static (mirror) pods are skipped — the node\nrecreates them immediately, so evicting them is a loop — and pods already\nterminal are skipped too.\n\n`drain` is ignored when `schedulable` is true, because draining a node you just\nreturned to service is incoherent rather than merely useless.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"CordonIn.cluster": "Cluster is the registered cluster the node belongs to.",
|
||||||
|
"CordonIn.drain": "Drain evicts the node's pods after cordoning. Ignored when Schedulable is\ntrue, because draining a node you just returned to service is incoherent.",
|
||||||
|
"CordonIn.node": "Node is the node's name.",
|
||||||
|
"CordonIn.schedulable": "Schedulable is the state to leave the node in: false cordons it, true\nuncordons it.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","node":"pool-gpu-a1","schedulable":false,"drain":true}`),
|
||||||
|
Response: json.RawMessage(`{"evicted":11}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /v1/k8s/volumes/claims", zip.Doc{
|
||||||
|
Description: "Creates a PersistentVolumeClaim if it is absent.\n\nIdempotent on name in the namespace, for the reason ensureNamespace is: the\nget-then-create ladder is a race when each caller writes it.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"EnsureClaimIn.accessMode": "AccessMode is \"ReadWriteOnce\" (default) or \"ReadWriteMany\".",
|
||||||
|
"EnsureClaimIn.cluster": "Cluster is the registered cluster to create the claim on.",
|
||||||
|
"EnsureClaimIn.gi": "Gi is the storage to request, in GiB.",
|
||||||
|
"EnsureClaimIn.name": "Name is the claim's name.",
|
||||||
|
"EnsureClaimIn.namespace": "Namespace is where the claim lives.",
|
||||||
|
"EnsureClaimIn.storageClass": "StorageClass names the class; empty takes the cluster's default.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod","name":"data","gi":100,"storageClass":"do-block-storage"}`),
|
||||||
|
Response: json.RawMessage(`{"name":"data","created":true}`),
|
||||||
|
})
|
||||||
|
zip.Describe("POST /v1/k8s/volumes/claims/:namespace/:name/expand", zip.Doc{
|
||||||
|
Description: "Grows a PersistentVolumeClaim to a new size.\n\nThis is the ONE correct way to grow a volume Kubernetes manages, and the reason it\nis a named op rather than a patch: the resize controller acts on the CLAIM, growing\nthe backing cloud device and then the filesystem on it, so claim, PersistentVolume,\ndevice and filesystem all end up agreeing. Growing the device through the cloud\nprovider's own API instead leaves the claim and the PV declaring the old capacity\nand the filesystem never grown at all — three sources of truth, two of them wrong.\n\nThe StorageClass must set `allowVolumeExpansion`; when it does not, the apiserver\nrefuses and its message is surfaced verbatim rather than worked around. A shrink is\nrefused by the apiserver too, and there is no op that deletes a claim: the claim\nholds the only copy of the tenant's data, so this app can make one bigger and\nnothing else.",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ExpandClaimIn.cluster": "Cluster is the registered cluster the claim lives on.",
|
||||||
|
"ExpandClaimIn.gi": "Gi is the new size in GiB. It must exceed the current request; the\napiserver refuses a shrink and its message is surfaced verbatim.",
|
||||||
|
"ExpandClaimIn.name": "Name is the claim's name.",
|
||||||
|
"ExpandClaimIn.namespace": "Namespace is the claim's namespace.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","namespace":"acme-prod","name":"data","gi":200}`),
|
||||||
|
Response: json.RawMessage(`{"gi":200}`),
|
||||||
|
})
|
||||||
|
zip.Describe("PUT /v1/k8s/resources", zip.Doc{
|
||||||
|
Description: "Creates one custom resource of a registered kind, or patches the one\nalready there.\n\nFive separate get-then-create-else-patch ladders in the fleet collapse into this one\nname. Each of those wrote the ladder itself and each was a race; here the apiserver\nresolves the race, because a create that loses falls through to the patch.\n\nThe `spec` is the caller's domain payload and is applied whole — its SHAPE belongs to\nwhoever authored the CRD, not to this app, which would otherwise have to be edited\nevery time any kind grew a field. What this app owns is the KIND and the VERB, and\nboth are named and typed. That is the line: `kind` is a key in a closed table, so\nthis cannot become \"apply any object the cluster has\".",
|
||||||
|
Fields: map[string]string{
|
||||||
|
"ApplyIn.cluster": "Cluster is the registered cluster to apply to.",
|
||||||
|
"ApplyIn.kind": "Kind is the registered kind.",
|
||||||
|
"ApplyIn.labels": "Labels are merged onto the object.",
|
||||||
|
"ApplyIn.name": "Name is the object's name.",
|
||||||
|
"ApplyIn.namespace": "Namespace is where the object lives; empty for a cluster-scoped kind.",
|
||||||
|
"ApplyIn.spec": "Spec is the desired spec, applied whole.",
|
||||||
|
"Resource.version": "Version is the object's resourceVersion — the cursor a Watch resumes from.",
|
||||||
|
},
|
||||||
|
Example: json.RawMessage(`{"cluster":"hanzo-k8s","kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2","replicas":3}}`),
|
||||||
|
Response: json.RawMessage(`{"kind":"App","namespace":"acme-prod","name":"api","spec":{"image":"registry.hanzo.ai/acme/api:1.4.2","replicas":3},"version":"918281"}`),
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -0,0 +1,153 @@
|
|||||||
|
package plane
|
||||||
|
|
||||||
|
import (
|
||||||
|
"go/ast"
|
||||||
|
"go/parser"
|
||||||
|
"go/token"
|
||||||
|
"io/fs"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The contract this file gates is the one the compiler cannot state: that no input type
|
||||||
|
// carries a tenant, and that no input type reaches its cluster name through an embedded
|
||||||
|
// struct. Both are read off the source, so a type added tomorrow is covered without
|
||||||
|
// anybody remembering to add it here.
|
||||||
|
|
||||||
|
// structs parses this package and returns every struct type it declares.
|
||||||
|
func structs(t *testing.T) map[string]*ast.StructType {
|
||||||
|
t.Helper()
|
||||||
|
fset := token.NewFileSet()
|
||||||
|
pkgs, err := parser.ParseDir(fset, ".", func(fi fs.FileInfo) bool {
|
||||||
|
return !strings.HasSuffix(fi.Name(), "_test.go")
|
||||||
|
}, 0)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parse: %v", err)
|
||||||
|
}
|
||||||
|
out := map[string]*ast.StructType{}
|
||||||
|
for _, p := range pkgs {
|
||||||
|
for _, f := range p.Files {
|
||||||
|
ast.Inspect(f, func(n ast.Node) bool {
|
||||||
|
ts, ok := n.(*ast.TypeSpec)
|
||||||
|
if !ok {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if st, isStruct := ts.Type.(*ast.StructType); isStruct {
|
||||||
|
out[ts.Name.Name] = st
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(out) == 0 {
|
||||||
|
t.Fatal("this gate proved nothing: no struct types were parsed")
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func fields(st *ast.StructType) []string {
|
||||||
|
var out []string
|
||||||
|
for _, f := range st.Fields.List {
|
||||||
|
for _, n := range f.Names {
|
||||||
|
out = append(out, n.Name)
|
||||||
|
}
|
||||||
|
if len(f.Names) == 0 {
|
||||||
|
// An embedded field. Recorded under its type name so the embedding gate
|
||||||
|
// below can see it.
|
||||||
|
if id, ok := f.Type.(*ast.Ident); ok {
|
||||||
|
out = append(out, "«embedded "+id.Name+"»")
|
||||||
|
} else {
|
||||||
|
out = append(out, "«embedded»")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// isInput reports whether a type name is one that travels IN. The convention IS the
|
||||||
|
// discriminator — a name ending in "In" or "Ref", plus the two general selectors — and it
|
||||||
|
// is a convention worth enforcing precisely so this gate can tell an input from a reply
|
||||||
|
// without a hand-maintained list a new type would be missing from.
|
||||||
|
//
|
||||||
|
// A reply may legitimately carry an org: Registered.Org says who OWNS a cluster, which a
|
||||||
|
// cross-org observer has to see. It may also carry an owner: Workload.Owner is a
|
||||||
|
// Kubernetes ownerReference, the same word for a different thing.
|
||||||
|
func isInput(name string) bool {
|
||||||
|
return strings.HasSuffix(name, "In") || strings.HasSuffix(name, "Ref") ||
|
||||||
|
name == "Selector" || name == "NoArgs"
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNoInputCarriesATenant(t *testing.T) {
|
||||||
|
// An org in the argument is an org the CALLER chose, and a caller that can name the
|
||||||
|
// org can read another tenant. The tenant rides the caller and the callee re-derives
|
||||||
|
// it; there is no field for it, on any op, on any transport.
|
||||||
|
banned := map[string]bool{
|
||||||
|
"org": true, "orgid": true, "organization": true,
|
||||||
|
"tenant": true, "tenantid": true,
|
||||||
|
}
|
||||||
|
checked := 0
|
||||||
|
for name, st := range structs(t) {
|
||||||
|
if !isInput(name) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
checked++
|
||||||
|
for _, f := range fields(st) {
|
||||||
|
if banned[strings.ToLower(f)] {
|
||||||
|
t.Errorf("%s.%s: an input must not name a tenant — the tenant rides the caller", name, f)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Non-vacuity: this app answers 36 ops, so a run that examined a handful of types
|
||||||
|
// found the wrong set and must fail rather than pass quietly.
|
||||||
|
if checked < 18 {
|
||||||
|
t.Fatalf("this gate examined only %d input types; the naming convention it discriminates on has drifted", checked)
|
||||||
|
}
|
||||||
|
t.Logf("examined %d input types", checked)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNoTypeEmbedsAnother(t *testing.T) {
|
||||||
|
// This is not style. zip binds a URL value onto the TOP LEVEL of an input only, so a
|
||||||
|
// `Cluster` promoted through an embedded struct never binds from `?cluster=` or from
|
||||||
|
// a `:cluster` path segment — every op would silently read the empty cluster name and
|
||||||
|
// answer 404 for a cluster that exists. The field is spelled per type for that
|
||||||
|
// reason, and embedding one back in is the regression this gate names.
|
||||||
|
for name, st := range structs(t) {
|
||||||
|
for _, f := range fields(st) {
|
||||||
|
if strings.HasPrefix(f, "«embedded") {
|
||||||
|
t.Errorf("%s embeds %s: a promoted field does not bind from a URL, so every op using it would read an empty value",
|
||||||
|
name, strings.TrimSuffix(strings.TrimPrefix(f, "«embedded "), "»"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOpTokensAreNamespacedAndUnique(t *testing.T) {
|
||||||
|
// Every op token is this app's, prefixed so a token can never collide with another
|
||||||
|
// service's on the shared call plane, and unique so two ops cannot answer one name.
|
||||||
|
tokens := []string{
|
||||||
|
ClustersList, ClustersGet, ClustersRegister, ClustersDeregister, ClustersStatus,
|
||||||
|
NodesList, NodesCordon, NodesVolumeStats,
|
||||||
|
NamespacesList, NamespacesGet, NamespacesEnsure, NamespacesDelete,
|
||||||
|
PodsList, PodsLogs,
|
||||||
|
VolumesListClaims, VolumesListPersistent, VolumesEnsureClaim, VolumesExpandClaim,
|
||||||
|
WorkloadsListDeployments, WorkloadsGetDeployment, WorkloadsListReplicaSets,
|
||||||
|
WorkloadsListStatefulSets, WorkloadsListServices, WorkloadsListIngresses,
|
||||||
|
WorkloadsListEvents, WorkloadsListConfigMaps,
|
||||||
|
JobsCreate, JobsList, JobsGet,
|
||||||
|
ResourcesList, ResourcesGet, ResourcesWatch, ResourcesApply, ResourcesDelete,
|
||||||
|
AccessCan, MetricsPods,
|
||||||
|
}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for _, tok := range tokens {
|
||||||
|
if !strings.HasPrefix(tok, "k8s_") {
|
||||||
|
t.Errorf("op token %q is not namespaced to this app", tok)
|
||||||
|
}
|
||||||
|
if seen[tok] {
|
||||||
|
t.Errorf("op token %q is declared twice", tok)
|
||||||
|
}
|
||||||
|
seen[tok] = true
|
||||||
|
}
|
||||||
|
if len(seen) != 36 {
|
||||||
|
t.Fatalf("the op set is %d tokens; the measured demand this app answers is 36 — add the token to this list with the op", len(seen))
|
||||||
|
}
|
||||||
|
}
|
||||||
+791
@@ -0,0 +1,791 @@
|
|||||||
|
// Package plane is this app's call contract: the name of every op a peer may
|
||||||
|
// invoke on it, and the input and output each one carries.
|
||||||
|
//
|
||||||
|
// It is a LEAF — stdlib only. Both halves of a call import it, so the two halves
|
||||||
|
// cannot drift, and a consumer that wants to ASK about a cluster links this
|
||||||
|
// package and nothing else: no client-go, no kubeconfig parser, no credential.
|
||||||
|
// That is the whole point of the split. `go list -deps ./plane` is the proof.
|
||||||
|
//
|
||||||
|
// # The op token is the operation's one identity
|
||||||
|
//
|
||||||
|
// Each constant below is simultaneously the OpenAPI operationId, the MCP tool
|
||||||
|
// name, the CLI command and the plane op. One string, four projections, spelled
|
||||||
|
// once here so a caller and a callee cannot disagree at run time about a name
|
||||||
|
// the compiler never compared.
|
||||||
|
//
|
||||||
|
// # Every op NAMES its cluster
|
||||||
|
//
|
||||||
|
// Every In type that touches an apiserver carries a `Cluster` field, because there
|
||||||
|
// is no ambient cluster in this program: lux runs lux-k8s, zoo runs zoo-k8s, a
|
||||||
|
// customer runs their own, and "the cluster" is not a fact any function may
|
||||||
|
// assume. A function that can be called without saying which cluster would be a
|
||||||
|
// function that reads whichever one the process happened to be started next to.
|
||||||
|
//
|
||||||
|
// # There is no Org field in this file
|
||||||
|
//
|
||||||
|
// The tenant a call acts for rides the CALLER — forwarded from the gateway's
|
||||||
|
// assertion with zip.Ctx.Forward, or stated once by a background job with
|
||||||
|
// zip.WithCaller — and the callee re-derives it with zip.Tenant. An org in the
|
||||||
|
// argument is an org the caller CHOSE, and a caller that can name the org can
|
||||||
|
// read another tenant's cluster. `Org` appears only on REPLIES, where it is a
|
||||||
|
// property of the thing described rather than a claim by the asker.
|
||||||
|
//
|
||||||
|
// # There is no namespace default either
|
||||||
|
//
|
||||||
|
// A namespaced op states its namespace, and the registry row for the cluster
|
||||||
|
// bounds which namespaces that registration may address (registry.Cluster's
|
||||||
|
// Namespaces). A tenant naming a namespace outside its bound is refused before
|
||||||
|
// the apiserver is dialed, and refused again by the credential's own RBAC.
|
||||||
|
//
|
||||||
|
// # The wire is ZAP; the tags are the document's vocabulary
|
||||||
|
//
|
||||||
|
// These types cross as ZAP messages: a field IS its offset and no name travels.
|
||||||
|
// The compatibility rule is therefore structural — APPEND FIELDS AT THE END, and
|
||||||
|
// only at the end. Reordering, inserting or retyping one changes what every
|
||||||
|
// existing peer reads.
|
||||||
|
package plane
|
||||||
|
|
||||||
|
// The op names.
|
||||||
|
//
|
||||||
|
// Read as a set they are the whole Kubernetes authority this deployment holds:
|
||||||
|
// nine reads and a bounded list of mutations, each one named so its blast radius
|
||||||
|
// is auditable. There is deliberately no "run any k8s call" op — a generic
|
||||||
|
// passthrough relocates the credential problem behind an RPC instead of solving
|
||||||
|
// it, and the union RBAC in the repo's LLM.md could not then be written at all.
|
||||||
|
const (
|
||||||
|
// clusters — the registry, this app's first-class concept.
|
||||||
|
ClustersList = "k8s_clusters_list"
|
||||||
|
ClustersGet = "k8s_clusters_get"
|
||||||
|
ClustersRegister = "k8s_clusters_register"
|
||||||
|
ClustersDeregister = "k8s_clusters_deregister"
|
||||||
|
ClustersStatus = "k8s_clusters_status"
|
||||||
|
|
||||||
|
// nodes
|
||||||
|
NodesList = "k8s_nodes_list"
|
||||||
|
NodesCordon = "k8s_nodes_cordon"
|
||||||
|
NodesVolumeStats = "k8s_nodes_volume_stats"
|
||||||
|
|
||||||
|
// namespaces
|
||||||
|
NamespacesList = "k8s_namespaces_list"
|
||||||
|
NamespacesGet = "k8s_namespaces_get"
|
||||||
|
NamespacesEnsure = "k8s_namespaces_ensure"
|
||||||
|
NamespacesDelete = "k8s_namespaces_delete"
|
||||||
|
|
||||||
|
// pods
|
||||||
|
PodsList = "k8s_pods_list"
|
||||||
|
PodsLogs = "k8s_pods_logs"
|
||||||
|
|
||||||
|
// volumes — PersistentVolume and PersistentVolumeClaim.
|
||||||
|
VolumesListClaims = "k8s_volumes_list_claims"
|
||||||
|
VolumesListPersistent = "k8s_volumes_list_persistent"
|
||||||
|
VolumesEnsureClaim = "k8s_volumes_ensure_claim"
|
||||||
|
VolumesExpandClaim = "k8s_volumes_expand_claim"
|
||||||
|
|
||||||
|
// workloads — read-only projections.
|
||||||
|
WorkloadsListDeployments = "k8s_workloads_list_deployments"
|
||||||
|
WorkloadsGetDeployment = "k8s_workloads_get_deployment"
|
||||||
|
WorkloadsListReplicaSets = "k8s_workloads_list_replicasets"
|
||||||
|
WorkloadsListStatefulSets = "k8s_workloads_list_statefulsets"
|
||||||
|
WorkloadsListServices = "k8s_workloads_list_services"
|
||||||
|
WorkloadsListIngresses = "k8s_workloads_list_ingresses"
|
||||||
|
WorkloadsListEvents = "k8s_workloads_list_events"
|
||||||
|
WorkloadsListConfigMaps = "k8s_workloads_list_configmaps"
|
||||||
|
|
||||||
|
// jobs
|
||||||
|
JobsCreate = "k8s_jobs_create"
|
||||||
|
JobsList = "k8s_jobs_list"
|
||||||
|
JobsGet = "k8s_jobs_get"
|
||||||
|
|
||||||
|
// resources — the CLOSED custom-resource kind set.
|
||||||
|
ResourcesList = "k8s_resources_list"
|
||||||
|
ResourcesGet = "k8s_resources_get"
|
||||||
|
ResourcesWatch = "k8s_resources_watch"
|
||||||
|
ResourcesApply = "k8s_resources_apply"
|
||||||
|
ResourcesDelete = "k8s_resources_delete"
|
||||||
|
|
||||||
|
// access
|
||||||
|
AccessCan = "k8s_access_can"
|
||||||
|
|
||||||
|
// metrics
|
||||||
|
MetricsPods = "k8s_metrics_pods"
|
||||||
|
)
|
||||||
|
|
||||||
|
// KMS is the ONE peer this app calls, and the two op tokens it must spell to
|
||||||
|
// reach it. They belong to the KMS app and are declared identically in cloud's
|
||||||
|
// own plane package; when HIP-0106's `github.com/hanzoai/plane` extraction lands
|
||||||
|
// (migration step 2), both spellings collapse into that module and these three
|
||||||
|
// declarations are deleted in the same change. Until then this is the ONE place
|
||||||
|
// in this repo that names them, so there is exactly one line to move.
|
||||||
|
const (
|
||||||
|
KMS = "kms"
|
||||||
|
KMSGet = "kms_get"
|
||||||
|
KMSPut = "kms_put"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SecretIn names one secret in KMS and carries its value on a write. It is
|
||||||
|
// byte-compatible with the KMS app's own SecretIn, which is what makes the call
|
||||||
|
// work: a ZAP field is its offset, so the two declarations must stay structurally
|
||||||
|
// identical, and appending to either without the other is the break.
|
||||||
|
type SecretIn struct {
|
||||||
|
Ref string `json:"ref"`
|
||||||
|
Value []byte `json:"value,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Secret is one secret's value as KMS returns it.
|
||||||
|
type Secret struct {
|
||||||
|
Value []byte `json:"value,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// NoArgs is the input of an op that takes none. The registry read it serves is
|
||||||
|
// scoped by the CALLER and by nothing the caller could type.
|
||||||
|
type NoArgs struct{}
|
||||||
|
|
||||||
|
// ClusterRef names one registered cluster and nothing else.
|
||||||
|
//
|
||||||
|
// Every In type below carries a `Cluster` field of its own rather than embedding
|
||||||
|
// this one. That is not repetition for its own sake: zip binds a URL value onto
|
||||||
|
// the TOP LEVEL of an input only, so a cluster name promoted through an embedded
|
||||||
|
// struct would never bind from `?cluster=` or from a `:cluster` path segment, and
|
||||||
|
// every op would silently read the empty cluster. The field is spelled per type
|
||||||
|
// so that the wire actually carries it.
|
||||||
|
type ClusterRef struct {
|
||||||
|
// Cluster is the registered cluster's name — "hanzo-k8s", "lux-k8s",
|
||||||
|
// "zoo-k8s", "bootnode-k8s", or any name an org registered its own cluster
|
||||||
|
// under. It is resolved against the CALLING ORG's registry, so two orgs may
|
||||||
|
// use the same name for two different clusters and neither can reach the
|
||||||
|
// other's.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- clusters --------------------------------------------------------------
|
||||||
|
|
||||||
|
// Registered is one registered cluster as the registry holds it. The kubeconfig
|
||||||
|
// is NOT here and has no field: it is sealed in KMS under a ref this app derives
|
||||||
|
// from the owner and the name, and it is resolved at use and never returned.
|
||||||
|
type Registered struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
// Org is which org OWNS this cluster. On a reply that is a property of the
|
||||||
|
// record, not a claim by the caller.
|
||||||
|
Org string `json:"org"`
|
||||||
|
Provider string `json:"provider"`
|
||||||
|
Endpoint string `json:"endpoint,omitempty"`
|
||||||
|
Nodes int `json:"nodes"`
|
||||||
|
NvidiaGPU int `json:"nvidiaGpu"`
|
||||||
|
AmdGPU int `json:"amdGpu"`
|
||||||
|
// Namespaces bounds what this registration may address. Empty means the
|
||||||
|
// whole cluster, which is what an org registering its OWN cluster gets.
|
||||||
|
Namespaces []string `json:"namespaces,omitempty"`
|
||||||
|
Registered string `json:"registered"`
|
||||||
|
Default bool `json:"default"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Clusters is every cluster the calling org may reach.
|
||||||
|
type Clusters struct {
|
||||||
|
Clusters []Registered `json:"clusters"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// RegisterIn attaches a cluster to the calling org's registry.
|
||||||
|
type RegisterIn struct {
|
||||||
|
// Name is what every later op will call this cluster. Unique within the org.
|
||||||
|
Name string `json:"name" validate:"required"`
|
||||||
|
// Kubeconfig is the credential, in kubeconfig form. It is validated, used
|
||||||
|
// once to prove the cluster is reachable, sealed in KMS, and then dropped:
|
||||||
|
// it is never written to disk, never logged and never returned.
|
||||||
|
Kubeconfig string `json:"kubeconfig" validate:"required"`
|
||||||
|
// Provider labels where the cluster came from ("doks", "byo", "k3s", …).
|
||||||
|
Provider string `json:"provider,omitempty"`
|
||||||
|
// Namespaces bounds this registration to a namespace set. Omit it when the
|
||||||
|
// org owns the whole cluster; state it when the credential is a scoped
|
||||||
|
// ServiceAccount on a shared cluster.
|
||||||
|
Namespaces []string `json:"namespaces,omitempty"`
|
||||||
|
// Default marks the cluster an org's workloads target when none is named by
|
||||||
|
// a human — the ONE place a default is legitimate, because it is the org's
|
||||||
|
// own recorded choice rather than a process's ambient neighbour.
|
||||||
|
Default bool `json:"default,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Deregistered reports whether a name was present to remove.
|
||||||
|
type Deregistered struct {
|
||||||
|
Removed bool `json:"removed"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reachable is a cluster's liveness as of this call.
|
||||||
|
type Reachable struct {
|
||||||
|
// Connected is whether the apiserver answered.
|
||||||
|
Connected bool `json:"connected"`
|
||||||
|
// Version is the server version string, when it answered.
|
||||||
|
Version string `json:"version,omitempty"`
|
||||||
|
// Reason is why it did not, verbatim. Absent on success.
|
||||||
|
Reason string `json:"reason,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- nodes -----------------------------------------------------------------
|
||||||
|
|
||||||
|
// Node is one worker node, projected to what the boards read: is it up, will it
|
||||||
|
// take work, and what accelerators does it offer.
|
||||||
|
type Node struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Ready bool `json:"ready"`
|
||||||
|
Schedulable bool `json:"schedulable"`
|
||||||
|
Region string `json:"region,omitempty"`
|
||||||
|
InstanceKind string `json:"instanceKind,omitempty"`
|
||||||
|
CPU string `json:"cpu,omitempty"`
|
||||||
|
Memory string `json:"memory,omitempty"`
|
||||||
|
NvidiaGPU int `json:"nvidiaGpu"`
|
||||||
|
AmdGPU int `json:"amdGpu"`
|
||||||
|
InternalIP string `json:"internalIp,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Nodes is a cluster's worker inventory.
|
||||||
|
type Nodes struct {
|
||||||
|
Nodes []Node `json:"nodes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CordonIn takes a node in or out of service.
|
||||||
|
type CordonIn struct {
|
||||||
|
// Cluster is the registered cluster the node belongs to.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Node is the node's name.
|
||||||
|
Node string `json:"node" validate:"required"`
|
||||||
|
// Schedulable is the state to leave the node in: false cordons it, true
|
||||||
|
// uncordons it.
|
||||||
|
Schedulable bool `json:"schedulable"`
|
||||||
|
// Drain evicts the node's pods after cordoning. Ignored when Schedulable is
|
||||||
|
// true, because draining a node you just returned to service is incoherent.
|
||||||
|
Drain bool `json:"drain,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Cordoned reports how many pods the drain actually evicted.
|
||||||
|
type Cordoned struct {
|
||||||
|
Evicted int `json:"evicted"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// VolumeUse is one mounted claim's consumption, as its kubelet reports it.
|
||||||
|
type VolumeUse struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
UsedBytes int64 `json:"usedBytes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// VolumeStats is every mounted claim's fill on the cluster, plus how much of the
|
||||||
|
// fleet answered. Coverage is load-bearing: a claim with no row is UNMEASURED,
|
||||||
|
// which is a different fact from empty, and a consumer that treats the two alike
|
||||||
|
// will condemn live data.
|
||||||
|
type VolumeStats struct {
|
||||||
|
Volumes []VolumeUse `json:"volumes"`
|
||||||
|
// NodesRead is how many kubelets answered; NodesTotal how many were asked.
|
||||||
|
NodesRead int `json:"nodesRead"`
|
||||||
|
NodesTotal int `json:"nodesTotal"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- namespaces ------------------------------------------------------------
|
||||||
|
|
||||||
|
// Namespace is one namespace, projected.
|
||||||
|
type Namespace struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Phase string `json:"phase,omitempty"`
|
||||||
|
Labels map[string]string `json:"labels,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Namespaces is the namespaces this registration may address.
|
||||||
|
type Namespaces struct {
|
||||||
|
Namespaces []Namespace `json:"namespaces"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// NamespaceRef addresses one namespace on one cluster.
|
||||||
|
type NamespaceRef struct {
|
||||||
|
// Cluster is the registered cluster the namespace lives on.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Namespace is the namespace's name.
|
||||||
|
Namespace string `json:"namespace" validate:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// EnsureNamespaceIn creates a namespace if it is absent.
|
||||||
|
//
|
||||||
|
// It replaces four separate get-then-create-if-missing implementations with one
|
||||||
|
// name. Idempotence is the whole point: a caller that has to write the ladder
|
||||||
|
// itself writes a race, because "get says absent" and "create" are two calls.
|
||||||
|
type EnsureNamespaceIn struct {
|
||||||
|
// Cluster is the registered cluster to create the namespace on.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Namespace is the namespace's name.
|
||||||
|
Namespace string `json:"namespace" validate:"required"`
|
||||||
|
// Labels are merged onto the namespace when it is created.
|
||||||
|
Labels map[string]string `json:"labels,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensured reports whether the act created the object or found it already there.
|
||||||
|
// Two different facts, so two different answers: an idempotent call that cannot
|
||||||
|
// say which one it did leaves the caller unable to log a creation.
|
||||||
|
type Ensured struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Created bool `json:"created"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Deleted reports whether the object was present to delete.
|
||||||
|
type Deleted struct {
|
||||||
|
Removed bool `json:"removed"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- pods ------------------------------------------------------------------
|
||||||
|
|
||||||
|
// PodsIn selects pods on one cluster.
|
||||||
|
type PodsIn struct {
|
||||||
|
// Cluster is the registered cluster to read.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Namespace narrows to one namespace. Empty means every namespace this
|
||||||
|
// registration may address, which for a bounded registration is its own set
|
||||||
|
// and never the whole cluster.
|
||||||
|
Namespace string `json:"namespace,omitempty"`
|
||||||
|
// LabelSelector is a Kubernetes label selector, verbatim.
|
||||||
|
LabelSelector string `json:"labelSelector,omitempty"`
|
||||||
|
// NodeName narrows to the pods on one node — the field-selector variant the
|
||||||
|
// drain path needs, named rather than left to a raw selector string.
|
||||||
|
NodeName string `json:"nodeName,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pod is one pod, projected to placement, health, what it mounts and what it
|
||||||
|
// runs. Container ENV is deliberately absent: a pod spec carries injected
|
||||||
|
// secret material, and a read surface that returns it is an exfiltration door.
|
||||||
|
type Pod struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Phase string `json:"phase"`
|
||||||
|
Reason string `json:"reason,omitempty"`
|
||||||
|
Node string `json:"node,omitempty"`
|
||||||
|
Controller string `json:"controller,omitempty"`
|
||||||
|
Claims []string `json:"claims,omitempty"`
|
||||||
|
Images []string `json:"images,omitempty"`
|
||||||
|
Restarts int32 `json:"restarts"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pods is a page of pods.
|
||||||
|
type Pods struct {
|
||||||
|
Pods []Pod `json:"pods"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// LogsIn reads one container's log.
|
||||||
|
type LogsIn struct {
|
||||||
|
// Cluster is the registered cluster the pod runs on.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Namespace is the pod's namespace.
|
||||||
|
Namespace string `json:"namespace" validate:"required"`
|
||||||
|
// Pod is the pod's name.
|
||||||
|
Pod string `json:"pod" validate:"required"`
|
||||||
|
// Container names the container; empty takes the pod's only one.
|
||||||
|
Container string `json:"container,omitempty"`
|
||||||
|
// TailLines caps how many lines come back, newest last. Zero takes the
|
||||||
|
// server's default. A log read is unbounded by nature, so this is the bound.
|
||||||
|
TailLines int64 `json:"tailLines,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Logs is what the container printed.
|
||||||
|
type Logs struct {
|
||||||
|
Log string `json:"log"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- volumes ---------------------------------------------------------------
|
||||||
|
|
||||||
|
// Claim is one PersistentVolumeClaim, projected.
|
||||||
|
type Claim struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Phase string `json:"phase"`
|
||||||
|
Volume string `json:"volume,omitempty"`
|
||||||
|
// RequestGi is the size the claim asks for, in GiB.
|
||||||
|
RequestGi int `json:"requestGi"`
|
||||||
|
StorageClass string `json:"storageClass,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Claims is a page of claims.
|
||||||
|
type Claims struct {
|
||||||
|
Claims []Claim `json:"claims"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Volume is one PersistentVolume, projected to the identities by which it can be
|
||||||
|
// matched to the cloud device behind it.
|
||||||
|
type Volume struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Phase string `json:"phase"`
|
||||||
|
// Handle is the backing device's provider id, when the PV carries one.
|
||||||
|
Handle string `json:"handle,omitempty"`
|
||||||
|
ClaimNS string `json:"claimNamespace,omitempty"`
|
||||||
|
ClaimName string `json:"claimName,omitempty"`
|
||||||
|
CapacityGi int `json:"capacityGi"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Volumes is every PersistentVolume on the cluster. PVs are cluster-scoped, so
|
||||||
|
// this op is only served to a registration that addresses the whole cluster.
|
||||||
|
type Volumes struct {
|
||||||
|
Volumes []Volume `json:"volumes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// EnsureClaimIn creates a claim if it is absent.
|
||||||
|
type EnsureClaimIn struct {
|
||||||
|
// Cluster is the registered cluster to create the claim on.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Namespace is where the claim lives.
|
||||||
|
Namespace string `json:"namespace" validate:"required"`
|
||||||
|
// Name is the claim's name.
|
||||||
|
Name string `json:"name" validate:"required"`
|
||||||
|
// Gi is the storage to request, in GiB.
|
||||||
|
Gi int `json:"gi" validate:"required"`
|
||||||
|
// StorageClass names the class; empty takes the cluster's default.
|
||||||
|
StorageClass string `json:"storageClass,omitempty"`
|
||||||
|
// AccessMode is "ReadWriteOnce" (default) or "ReadWriteMany".
|
||||||
|
AccessMode string `json:"accessMode,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExpandClaimIn grows a claim. There is no shrink and no delete: a claim holds
|
||||||
|
// the only copy of a tenant's data, so this app can make one bigger and nothing
|
||||||
|
// more.
|
||||||
|
type ExpandClaimIn struct {
|
||||||
|
// Cluster is the registered cluster the claim lives on.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Namespace is the claim's namespace.
|
||||||
|
Namespace string `json:"namespace" validate:"required"`
|
||||||
|
// Name is the claim's name.
|
||||||
|
Name string `json:"name" validate:"required"`
|
||||||
|
// Gi is the new size in GiB. It must exceed the current request; the
|
||||||
|
// apiserver refuses a shrink and its message is surfaced verbatim.
|
||||||
|
Gi int `json:"gi" validate:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Expanded reports the size the claim now asks for.
|
||||||
|
type Expanded struct {
|
||||||
|
Gi int `json:"gi"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- workloads -------------------------------------------------------------
|
||||||
|
|
||||||
|
// Selector narrows a namespaced read.
|
||||||
|
type Selector struct {
|
||||||
|
// Cluster is the registered cluster to read.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Namespace narrows to one namespace; empty means every namespace this
|
||||||
|
// registration may address.
|
||||||
|
Namespace string `json:"namespace,omitempty"`
|
||||||
|
// LabelSelector is a Kubernetes label selector, verbatim.
|
||||||
|
LabelSelector string `json:"labelSelector,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Workload is one Deployment, ReplicaSet or StatefulSet, projected to the
|
||||||
|
// question every board asks: how many replicas are up, and what image is
|
||||||
|
// actually running. The RUNNING image is why this op exists — an App CR's status
|
||||||
|
// does not surface it, so the live workload is the only source.
|
||||||
|
type Workload struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
Replicas int32 `json:"replicas"`
|
||||||
|
Ready int32 `json:"ready"`
|
||||||
|
Updated int32 `json:"updated"`
|
||||||
|
Images []string `json:"images,omitempty"`
|
||||||
|
Owner string `json:"owner,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Workloads is a page of workloads.
|
||||||
|
type Workloads struct {
|
||||||
|
Workloads []Workload `json:"workloads"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// NamedIn addresses one namespaced object by name.
|
||||||
|
type NamedIn struct {
|
||||||
|
// Cluster is the registered cluster the object lives on.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Namespace is the object's namespace.
|
||||||
|
Namespace string `json:"namespace" validate:"required"`
|
||||||
|
// Name is the object's name.
|
||||||
|
Name string `json:"name" validate:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Service is one Service, projected to every identity by which it can be matched
|
||||||
|
// to the load balancer in front of it.
|
||||||
|
type Service struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Type string `json:"type"`
|
||||||
|
ClusterIP string `json:"clusterIp,omitempty"`
|
||||||
|
IPs []string `json:"ips,omitempty"`
|
||||||
|
// LBID is the provider's load-balancer id, from the cloud-controller's own
|
||||||
|
// annotation — the strongest link between a Service and a real balancer.
|
||||||
|
LBID string `json:"lbId,omitempty"`
|
||||||
|
Ports []int32 `json:"ports,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Services is a page of services.
|
||||||
|
type Services struct {
|
||||||
|
Services []Service `json:"services"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ingress is one Ingress, projected to the hosts it answers and whether each is
|
||||||
|
// covered by TLS.
|
||||||
|
type Ingress struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Hosts []string `json:"hosts,omitempty"`
|
||||||
|
TLSHosts []string `json:"tlsHosts,omitempty"`
|
||||||
|
Addresses []string `json:"addresses,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ingresses is a page of ingresses.
|
||||||
|
type Ingresses struct {
|
||||||
|
Ingresses []Ingress `json:"ingresses"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Event is one cluster event, projected. It is the only place a failure REASON
|
||||||
|
// for a workload that never produced a pod can be read.
|
||||||
|
type Event struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Type string `json:"type"`
|
||||||
|
Reason string `json:"reason,omitempty"`
|
||||||
|
Message string `json:"message,omitempty"`
|
||||||
|
Object string `json:"object,omitempty"`
|
||||||
|
Count int32 `json:"count"`
|
||||||
|
LastSeen int64 `json:"lastSeen"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Events is a page of events, newest last.
|
||||||
|
type Events struct {
|
||||||
|
Events []Event `json:"events"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ConfigMap is one ConfigMap. Data travels because a ConfigMap is by definition
|
||||||
|
// not secret material — a Secret is a different kind and this app does not read
|
||||||
|
// one back (see the repo's LLM.md on why).
|
||||||
|
type ConfigMap struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Data map[string]string `json:"data,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ConfigMaps is a page of config maps.
|
||||||
|
type ConfigMaps struct {
|
||||||
|
ConfigMaps []ConfigMap `json:"configMaps"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- jobs ------------------------------------------------------------------
|
||||||
|
|
||||||
|
// JobIn creates one batch Job.
|
||||||
|
type JobIn struct {
|
||||||
|
// Cluster is the registered cluster to run the job on.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Namespace is where the job runs.
|
||||||
|
Namespace string `json:"namespace" validate:"required"`
|
||||||
|
// Name is the job's name.
|
||||||
|
Name string `json:"name" validate:"required"`
|
||||||
|
// Image is the container image to run.
|
||||||
|
Image string `json:"image" validate:"required"`
|
||||||
|
// Command replaces the image's entrypoint. Empty keeps it.
|
||||||
|
Command []string `json:"command,omitempty"`
|
||||||
|
// Args are the arguments passed to the entrypoint.
|
||||||
|
Args []string `json:"args,omitempty"`
|
||||||
|
// Env is plain environment. Secret material does NOT belong here: seal it in
|
||||||
|
// KMS and project it with a scoped ServiceAccount instead.
|
||||||
|
Env map[string]string `json:"env,omitempty"`
|
||||||
|
// ServiceAccount runs the pod as a named ServiceAccount.
|
||||||
|
ServiceAccount string `json:"serviceAccount,omitempty"`
|
||||||
|
// Labels are stamped on the Job and its pods.
|
||||||
|
Labels map[string]string `json:"labels,omitempty"`
|
||||||
|
// BackoffLimit caps retries; zero means the Kubernetes default.
|
||||||
|
BackoffLimit int32 `json:"backoffLimit,omitempty"`
|
||||||
|
// TTLSeconds deletes the finished Job after this long. Zero leaves it, which
|
||||||
|
// on a busy namespace is how a Job list becomes unreadable.
|
||||||
|
TTLSeconds int32 `json:"ttlSeconds,omitempty"`
|
||||||
|
// Mounts attaches existing claims by name at a path.
|
||||||
|
Mounts []Mount `json:"mounts,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Mount attaches one existing PersistentVolumeClaim into a job's container.
|
||||||
|
type Mount struct {
|
||||||
|
Claim string `json:"claim"`
|
||||||
|
Path string `json:"path"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Job is one batch Job, projected to whether it finished and how it went.
|
||||||
|
type Job struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Active int32 `json:"active"`
|
||||||
|
Succeeded int32 `json:"succeeded"`
|
||||||
|
Failed int32 `json:"failed"`
|
||||||
|
// Phase is "running", "succeeded" or "failed" — the one word a caller wants,
|
||||||
|
// rolled here rather than re-derived from three counters by every consumer.
|
||||||
|
Phase string `json:"phase"`
|
||||||
|
StartedAt int64 `json:"startedAt,omitempty"`
|
||||||
|
EndedAt int64 `json:"endedAt,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Jobs is a page of jobs.
|
||||||
|
type Jobs struct {
|
||||||
|
Jobs []Job `json:"jobs"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- resources — the closed CR kind set ------------------------------------
|
||||||
|
|
||||||
|
// ResourceIn addresses custom resources of ONE registered kind.
|
||||||
|
//
|
||||||
|
// Kind is not free text: it is matched against this app's own allowlist, and a
|
||||||
|
// kind the allowlist does not carry is refused. Adding a kind is a code change
|
||||||
|
// here, which is exactly what keeps the set bounded and auditable — and is the
|
||||||
|
// difference between this op and the apply-arbitrary-YAML function it replaces.
|
||||||
|
type ResourceIn struct {
|
||||||
|
// Cluster is the registered cluster to read.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Kind is the registered kind: "App", "Application", "Datastore",
|
||||||
|
// "Validator", "TrainJob", "InferenceService", "IngressRoute", "Middleware"
|
||||||
|
// or "AppProject".
|
||||||
|
Kind string `json:"kind" validate:"required"`
|
||||||
|
// Namespace narrows to one namespace; empty means every namespace this
|
||||||
|
// registration may address. Ignored for a cluster-scoped kind.
|
||||||
|
Namespace string `json:"namespace,omitempty"`
|
||||||
|
// LabelSelector is a Kubernetes label selector, verbatim.
|
||||||
|
LabelSelector string `json:"labelSelector,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResourceRef addresses ONE custom resource by kind, namespace and name.
|
||||||
|
type ResourceRef struct {
|
||||||
|
// Cluster is the registered cluster the object lives on.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Kind is the registered kind.
|
||||||
|
Kind string `json:"kind" validate:"required"`
|
||||||
|
// Namespace is the object's namespace; empty for a cluster-scoped kind.
|
||||||
|
Namespace string `json:"namespace,omitempty"`
|
||||||
|
// Name is the object's name.
|
||||||
|
Name string `json:"name" validate:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resource is one custom resource: its coordinates and its whole body.
|
||||||
|
//
|
||||||
|
// Spec and Status travel as maps because their SHAPE is the CR author's domain,
|
||||||
|
// not this app's — projecting them here would make this app the schema owner of
|
||||||
|
// every kind it carries, and it would have to be edited whenever any of them
|
||||||
|
// grew a field. What this app owns is the KIND SET and the VERBS. That is the
|
||||||
|
// line between a bounded surface and a passthrough: the resource and the verb
|
||||||
|
// are named and typed, and the body is domain payload.
|
||||||
|
type Resource struct {
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
Namespace string `json:"namespace,omitempty"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Labels map[string]string `json:"labels,omitempty"`
|
||||||
|
Spec map[string]any `json:"spec,omitempty"`
|
||||||
|
Status map[string]any `json:"status,omitempty"`
|
||||||
|
// Version is the object's resourceVersion — the cursor a Watch resumes from.
|
||||||
|
Version string `json:"version,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resources is a page of custom resources, plus the cursor a Watch resumes from.
|
||||||
|
type Resources struct {
|
||||||
|
Resources []Resource `json:"resources"`
|
||||||
|
Version string `json:"version,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ApplyIn creates one custom resource or patches the one already there.
|
||||||
|
//
|
||||||
|
// It replaces five separate get-then-create-else-patch ladders with one name.
|
||||||
|
// The Spec is the caller's domain payload; the KIND is checked against the
|
||||||
|
// allowlist, so this cannot become "apply any object the cluster has".
|
||||||
|
type ApplyIn struct {
|
||||||
|
// Cluster is the registered cluster to apply to.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Kind is the registered kind.
|
||||||
|
Kind string `json:"kind" validate:"required"`
|
||||||
|
// Namespace is where the object lives; empty for a cluster-scoped kind.
|
||||||
|
Namespace string `json:"namespace,omitempty"`
|
||||||
|
// Name is the object's name.
|
||||||
|
Name string `json:"name" validate:"required"`
|
||||||
|
// Labels are merged onto the object.
|
||||||
|
Labels map[string]string `json:"labels,omitempty"`
|
||||||
|
// Spec is the desired spec, applied whole.
|
||||||
|
Spec map[string]any `json:"spec" validate:"required"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// WatchIn asks what changed since a cursor.
|
||||||
|
//
|
||||||
|
// It is a BOUNDED read, not a stream: it returns the events that arrived within
|
||||||
|
// WaitSeconds and the cursor to resume from. A stream cannot cross the call
|
||||||
|
// plane as a value, and rendering one — Server-Sent Events, a websocket — is the
|
||||||
|
// EDGE's concern. The consumer composes its own stream out of these answers,
|
||||||
|
// which is why the streaming shape is not duplicated in this app.
|
||||||
|
type WatchIn struct {
|
||||||
|
// Cluster is the registered cluster to watch.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Kind is the registered kind.
|
||||||
|
Kind string `json:"kind" validate:"required"`
|
||||||
|
// Namespace narrows to one namespace; empty means every namespace this
|
||||||
|
// registration may address.
|
||||||
|
Namespace string `json:"namespace,omitempty"`
|
||||||
|
// Version is the cursor to resume from — the Version a previous list or
|
||||||
|
// watch returned. Empty starts from the current state.
|
||||||
|
Version string `json:"version,omitempty"`
|
||||||
|
// WaitSeconds is how long to wait for a change before answering empty.
|
||||||
|
// Clamped to a bound this app owns, so a caller cannot pin a watch open.
|
||||||
|
WaitSeconds int `json:"waitSeconds,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Change is one watch event: what happened, and the object it happened to.
|
||||||
|
type Change struct {
|
||||||
|
// Type is "ADDED", "MODIFIED" or "DELETED".
|
||||||
|
Type string `json:"type"`
|
||||||
|
Resource Resource `json:"resource"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Changes is what happened since the cursor, and the cursor to resume from.
|
||||||
|
//
|
||||||
|
// Expired says the cursor is too old for the apiserver's history: the caller
|
||||||
|
// must LIST again rather than treat an empty page as "nothing changed", which is
|
||||||
|
// the bug that makes a watch-backed board silently stop updating.
|
||||||
|
type Changes struct {
|
||||||
|
Changes []Change `json:"changes"`
|
||||||
|
Version string `json:"version,omitempty"`
|
||||||
|
Expired bool `json:"expired,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- access ----------------------------------------------------------------
|
||||||
|
|
||||||
|
// CanIn asks the apiserver whether this app's own credential may do a thing.
|
||||||
|
//
|
||||||
|
// It is a SelfSubjectAccessReview: the answer comes from the cluster's RBAC
|
||||||
|
// rather than from a grant this app asserts about itself. Asking beats
|
||||||
|
// asserting — an out-of-tree ClusterRole that drifted is then a refusal a
|
||||||
|
// caller can degrade on, not a 403 in the middle of a mutation.
|
||||||
|
type CanIn struct {
|
||||||
|
// Cluster is the registered cluster to ask.
|
||||||
|
Cluster string `json:"cluster" validate:"required"`
|
||||||
|
// Verb is the Kubernetes verb: "get", "list", "create", "patch", "delete", …
|
||||||
|
Verb string `json:"verb" validate:"required"`
|
||||||
|
// Group is the API group; empty for the core group.
|
||||||
|
Group string `json:"group,omitempty"`
|
||||||
|
// Resource is the plural resource name ("pods", "resourcequotas").
|
||||||
|
Resource string `json:"resource" validate:"required"`
|
||||||
|
// Namespace scopes the question; empty asks cluster-wide.
|
||||||
|
Namespace string `json:"namespace,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Can is the apiserver's verdict, with its own reason when it gave one.
|
||||||
|
type Can struct {
|
||||||
|
Allowed bool `json:"allowed"`
|
||||||
|
Reason string `json:"reason,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- metrics ---------------------------------------------------------------
|
||||||
|
|
||||||
|
// PodMetric is one pod's live consumption, summed over its containers.
|
||||||
|
type PodMetric struct {
|
||||||
|
Namespace string `json:"namespace"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
// CPUMilli is CPU use in millicores; MemoryBytes is working-set memory.
|
||||||
|
CPUMilli int64 `json:"cpuMilli"`
|
||||||
|
MemoryBytes int64 `json:"memoryBytes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// PodMetrics is live consumption for the pods asked about.
|
||||||
|
//
|
||||||
|
// Available is false when the cluster has no metrics-server. That is a DIFFERENT
|
||||||
|
// fact from "every pod is idle", and a board that renders the two the same way
|
||||||
|
// tells an operator a healthy cluster is asleep.
|
||||||
|
type PodMetrics struct {
|
||||||
|
Pods []PodMetric `json:"pods"`
|
||||||
|
Available bool `json:"available"`
|
||||||
|
}
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
# The union of Kubernetes privilege this service needs — the first written statement of
|
||||||
|
# it anywhere in the fleet.
|
||||||
|
#
|
||||||
|
# Before this file existed, the only RBAC in tree granted a namespaced Role with
|
||||||
|
# `pods: [get, list, watch]`, which covered a writer-set membership check and NONE of the
|
||||||
|
# namespace-create, custom-resource-write, Job-create, node-patch, claim-patch or
|
||||||
|
# pod-evict operations the code performed. The rest was granted out of tree, so there was
|
||||||
|
# no auditable statement of the privilege the fleet actually held — and one consumer
|
||||||
|
# compensated at run time with a SelfSubjectAccessReview, asking the apiserver instead of
|
||||||
|
# asserting the grant.
|
||||||
|
#
|
||||||
|
# This file is that statement, and it is a CEILING: every verb here is one a named op in
|
||||||
|
# this repo performs, and every op in this repo performs only verbs here. `ops.Kinds()`
|
||||||
|
# and the `resources` rules below are gated against each other by the test suite, so a
|
||||||
|
# kind added to the code without a rule here fails the build rather than 403-ing in
|
||||||
|
# production.
|
||||||
|
#
|
||||||
|
# `""/secrets` is DELIBERATELY ABSENT and its absence is load-bearing. Material reaches a
|
||||||
|
# namespace as a KMSSecret custom resource the kms-operator resolves; nothing in this
|
||||||
|
# service reads a Secret back, because an RPC that returns a tenant's Secret is a
|
||||||
|
# credential-exfiltration endpoint however carefully it is gated. The standing rule is
|
||||||
|
# that credentials live in KMS, and this is what enforcing it looks like at the cluster.
|
||||||
|
#
|
||||||
|
# `nodes/proxy` is granted for ONE named op — nodes.volumeStats, the kubelet fill sweep —
|
||||||
|
# and appears in its own rule so that fact is visible in a diff. It is effectively
|
||||||
|
# arbitrary kubelet access, and it is the single largest privilege here. A deployment that
|
||||||
|
# does not use the volume board should delete that rule; nothing else needs it.
|
||||||
|
#
|
||||||
|
# On a SHARED cluster, do NOT grant this ClusterRole to a tenant. Bind a namespaced Role
|
||||||
|
# with the tenant's own namespaces instead, and register the cluster for that org with a
|
||||||
|
# `namespaces` bound — the registry refuses a namespace outside it before the apiserver is
|
||||||
|
# dialed, and this RBAC refuses it again.
|
||||||
|
apiVersion: rbac.authorization.k8s.io/v1
|
||||||
|
kind: ClusterRole
|
||||||
|
metadata:
|
||||||
|
name: hanzo-k8s
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: hanzo-k8s
|
||||||
|
app.kubernetes.io/managed-by: hanzo-cloud
|
||||||
|
rules:
|
||||||
|
# namespaces.{list,get,ensure,delete}
|
||||||
|
- apiGroups: [""]
|
||||||
|
resources: [namespaces]
|
||||||
|
verbs: [get, list, create, delete]
|
||||||
|
|
||||||
|
# pods.list, pods.logs, and the eviction half of nodes.cordon.
|
||||||
|
# `delete` is absent: a drain evicts, and an eviction respects PodDisruptionBudgets
|
||||||
|
# while a delete does not.
|
||||||
|
- apiGroups: [""]
|
||||||
|
resources: [pods, pods/log]
|
||||||
|
verbs: [get, list, watch]
|
||||||
|
- apiGroups: [""]
|
||||||
|
resources: [pods/eviction]
|
||||||
|
verbs: [create]
|
||||||
|
|
||||||
|
# workloads.listServices, workloads.listEvents, workloads.listConfigMaps
|
||||||
|
- apiGroups: [""]
|
||||||
|
resources: [services, events, configmaps]
|
||||||
|
verbs: [get, list, watch]
|
||||||
|
|
||||||
|
# volumes.listPersistent — cluster-scoped, read only. Nothing deletes a PersistentVolume.
|
||||||
|
- apiGroups: [""]
|
||||||
|
resources: [persistentvolumes]
|
||||||
|
verbs: [list]
|
||||||
|
|
||||||
|
# volumes.{listClaims,ensureClaim,expandClaim}. `delete` is absent on purpose: a claim
|
||||||
|
# holds the only copy of the tenant's data, so this service can make one bigger and
|
||||||
|
# nothing else.
|
||||||
|
- apiGroups: [""]
|
||||||
|
resources: [persistentvolumeclaims]
|
||||||
|
verbs: [get, list, create, patch]
|
||||||
|
|
||||||
|
# nodes.list and the cordon half of nodes.cordon. `patch` reaches only
|
||||||
|
# spec.unschedulable in this codebase — there is no generic node-patch op.
|
||||||
|
- apiGroups: [""]
|
||||||
|
resources: [nodes]
|
||||||
|
verbs: [get, list, watch, patch]
|
||||||
|
|
||||||
|
# nodes.volumeStats ONLY. See the note above.
|
||||||
|
- apiGroups: [""]
|
||||||
|
resources: [nodes/proxy]
|
||||||
|
verbs: [get]
|
||||||
|
|
||||||
|
# workloads.{listDeployments,getDeployment,listReplicaSets,listStatefulSets}
|
||||||
|
- apiGroups: [apps]
|
||||||
|
resources: [deployments, replicasets, statefulsets]
|
||||||
|
verbs: [get, list, watch]
|
||||||
|
|
||||||
|
# workloads.listIngresses
|
||||||
|
- apiGroups: [networking.k8s.io]
|
||||||
|
resources: [ingresses]
|
||||||
|
verbs: [get, list, watch]
|
||||||
|
|
||||||
|
# jobs.{create,list,get}
|
||||||
|
- apiGroups: [batch]
|
||||||
|
resources: [jobs]
|
||||||
|
verbs: [get, list, create]
|
||||||
|
|
||||||
|
# metrics.pods
|
||||||
|
- apiGroups: [metrics.k8s.io]
|
||||||
|
resources: [pods]
|
||||||
|
verbs: [list]
|
||||||
|
|
||||||
|
# access.can — asking the cluster what this credential may do, rather than asserting it.
|
||||||
|
- apiGroups: [authorization.k8s.io]
|
||||||
|
resources: [selfsubjectaccessreviews]
|
||||||
|
verbs: [create]
|
||||||
|
|
||||||
|
# resources.* over the CLOSED kind set. One rule per group, and the resource lists here
|
||||||
|
# are gated against ops.Kinds() by the test suite — two sources, one set.
|
||||||
|
- apiGroups: [hanzo.ai]
|
||||||
|
resources: [apps, datastores, ingressroutes, middlewares]
|
||||||
|
verbs: [get, list, watch, create, update, patch, delete]
|
||||||
|
- apiGroups: [apps.hanzo.ai]
|
||||||
|
resources: [applications]
|
||||||
|
verbs: [get, list, watch, create, update, patch, delete]
|
||||||
|
- apiGroups: [argoproj.io]
|
||||||
|
resources: [appprojects]
|
||||||
|
verbs: [get, list, watch, create, update, patch, delete]
|
||||||
|
- apiGroups: [node.lux.cloud]
|
||||||
|
resources: [luxnetworks]
|
||||||
|
verbs: [get, list, watch, create, update, patch, delete]
|
||||||
|
- apiGroups: [trainer.kubeflow.org]
|
||||||
|
resources: [trainjobs]
|
||||||
|
verbs: [get, list, watch, create, update, patch, delete]
|
||||||
|
- apiGroups: [serving.kserve.io]
|
||||||
|
resources: [inferenceservices]
|
||||||
|
verbs: [get, list, watch, create, update, patch, delete]
|
||||||
|
- apiGroups: [secrets.lux.network]
|
||||||
|
resources: [kmssecrets]
|
||||||
|
verbs: [get, list, watch, create, update, patch, delete]
|
||||||
@@ -0,0 +1,322 @@
|
|||||||
|
package registry
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
)
|
||||||
|
|
||||||
|
// This file is the adversarial suite. Every test in it is written from the attacker's
|
||||||
|
// side: a tenant that knows another tenant's cluster name, a tenant that knows another
|
||||||
|
// tenant's namespace, a caller with a header and no principal behind it.
|
||||||
|
//
|
||||||
|
// Each test names the guard it defeats when the guard is removed. Removing the org
|
||||||
|
// segment from indexRef makes TestForeignClusterNameIsNotFound and
|
||||||
|
// TestSameNameInTwoOrgsAreTwoClusters fail; removing the bound check from Bound.Scope
|
||||||
|
// makes TestForeignNamespaceIsRefused and TestBoundedListDoesNotWiden fail; removing
|
||||||
|
// the user-claim test from Tenant makes TestOrgHeaderWithoutPrincipalIsRefused fail.
|
||||||
|
// That mapping was verified by making each edit and observing the red, not assumed.
|
||||||
|
|
||||||
|
// memStore is an in-memory [Store]. It records every ref touched, because the ref
|
||||||
|
// NAMESPACE is the boundary this suite is really testing: if a call for one org can
|
||||||
|
// name another org's ref, no later check matters.
|
||||||
|
type memStore struct {
|
||||||
|
data map[string][]byte
|
||||||
|
got []string
|
||||||
|
put []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func newStore() *memStore { return &memStore{data: map[string][]byte{}} }
|
||||||
|
|
||||||
|
func (m *memStore) Get(_ context.Context, ref string) ([]byte, error) {
|
||||||
|
m.got = append(m.got, ref)
|
||||||
|
return m.data[ref], nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *memStore) Put(_ context.Context, ref string, value []byte) error {
|
||||||
|
m.put = append(m.put, ref)
|
||||||
|
if value == nil {
|
||||||
|
delete(m.data, ref)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
m.data[ref] = value
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// as is a context carrying a validated caller for one org. Both fields matter: an org
|
||||||
|
// with no user beside it is a claim the caller made about itself, which zip.Tenant
|
||||||
|
// refuses and this suite proves.
|
||||||
|
func as(org string) context.Context {
|
||||||
|
return zip.WithCaller(context.Background(), zip.Caller{Org: org, User: "u@" + org})
|
||||||
|
}
|
||||||
|
|
||||||
|
// seed writes an org's index and a usable credential straight into the store, so the
|
||||||
|
// authorization path can be exercised with no apiserver anywhere.
|
||||||
|
func seed(t *testing.T, s *memStore, org, name string, namespaces []string) {
|
||||||
|
t.Helper()
|
||||||
|
var list []Cluster
|
||||||
|
if raw := s.data[indexRef(org)]; len(raw) > 0 {
|
||||||
|
if err := json.Unmarshal(raw, &list); err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
list = append(list, Cluster{
|
||||||
|
Name: name, Org: org, Provider: "byo",
|
||||||
|
Endpoint: "https://" + name + "." + org + ".invalid:6443", Namespaces: namespaces,
|
||||||
|
})
|
||||||
|
raw, err := json.Marshal(list)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
s.data[indexRef(org)] = raw
|
||||||
|
s.data[credRef(org, name)] = kubeconfig("https://" + name + "." + org + ".invalid:6443")
|
||||||
|
}
|
||||||
|
|
||||||
|
// kubeconfig is a minimal, valid kubeconfig for a host. It carries no credential
|
||||||
|
// plugin, so SafeRESTConfig accepts it; nothing in these tests dials it.
|
||||||
|
func kubeconfig(host string) []byte {
|
||||||
|
return []byte(`apiVersion: v1
|
||||||
|
kind: Config
|
||||||
|
clusters:
|
||||||
|
- name: c
|
||||||
|
cluster:
|
||||||
|
server: ` + host + `
|
||||||
|
insecure-skip-tls-verify: true
|
||||||
|
contexts:
|
||||||
|
- name: c
|
||||||
|
context:
|
||||||
|
cluster: c
|
||||||
|
user: u
|
||||||
|
current-context: c
|
||||||
|
users:
|
||||||
|
- name: u
|
||||||
|
user:
|
||||||
|
token: t
|
||||||
|
`)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestForeignClusterNameIsNotFound(t *testing.T) {
|
||||||
|
t.Setenv(AllowPrivateHostsEnv, "1")
|
||||||
|
s := newStore()
|
||||||
|
seed(t, s, "alpha", "prod", nil)
|
||||||
|
r := New(s)
|
||||||
|
|
||||||
|
// beta knows the name. Knowing it must buy nothing.
|
||||||
|
if _, err := r.Reach(as("beta"), "prod"); err == nil {
|
||||||
|
t.Fatal("beta reached alpha's cluster by naming it — the org boundary is not being applied")
|
||||||
|
} else if _, ok := err.(ErrNoCluster); !ok {
|
||||||
|
t.Fatalf("want ErrNoCluster (a foreign cluster must be indistinguishable from an absent one), got %T: %v", err, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// And the refusal must not have touched alpha's material on the way.
|
||||||
|
for _, ref := range append(s.got, s.put...) {
|
||||||
|
if strings.Contains(ref, "orgs/alpha/") {
|
||||||
|
t.Fatalf("a call acting for beta touched %q", ref)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSameNameInTwoOrgsAreTwoClusters(t *testing.T) {
|
||||||
|
t.Setenv(AllowPrivateHostsEnv, "1")
|
||||||
|
s := newStore()
|
||||||
|
seed(t, s, "alpha", "prod", nil)
|
||||||
|
seed(t, s, "beta", "prod", nil)
|
||||||
|
r := New(s)
|
||||||
|
|
||||||
|
a, err := r.Reach(as("alpha"), "prod")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("alpha: %v", err)
|
||||||
|
}
|
||||||
|
b, err := r.Reach(as("beta"), "prod")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("beta: %v", err)
|
||||||
|
}
|
||||||
|
if a.Cluster.Endpoint == b.Cluster.Endpoint {
|
||||||
|
t.Fatalf("two orgs' clusters named %q resolved to one endpoint %q — a name is not a global identity",
|
||||||
|
"prod", a.Cluster.Endpoint)
|
||||||
|
}
|
||||||
|
if a.Rest.Host == b.Rest.Host {
|
||||||
|
t.Fatalf("both orgs got a client for %q — the credential is being shared across tenants", a.Rest.Host)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestForeignNamespaceIsRefused(t *testing.T) {
|
||||||
|
t.Setenv(AllowPrivateHostsEnv, "1")
|
||||||
|
s := newStore()
|
||||||
|
// beta is on the SHARED cluster with a scoped credential: it may address one
|
||||||
|
// namespace and no other.
|
||||||
|
seed(t, s, "beta", "shared", []string{"beta-prod"})
|
||||||
|
r := New(s)
|
||||||
|
|
||||||
|
b, err := r.Reach(as("beta"), "shared")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("beta: %v", err)
|
||||||
|
}
|
||||||
|
for _, ns := range []string{"alpha-prod", "kube-system", "hanzo", ""} {
|
||||||
|
if _, err := b.One(ns); err == nil {
|
||||||
|
t.Fatalf("beta acted on namespace %q outside its bound", ns)
|
||||||
|
} else if _, ok := err.(ErrNamespace); !ok {
|
||||||
|
t.Fatalf("namespace %q: want ErrNamespace, got %T: %v", ns, err, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got, err := b.One("beta-prod"); err != nil || got != "beta-prod" {
|
||||||
|
t.Fatalf("beta was refused its OWN namespace: %q %v", got, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBoundedListDoesNotWiden(t *testing.T) {
|
||||||
|
t.Setenv(AllowPrivateHostsEnv, "1")
|
||||||
|
s := newStore()
|
||||||
|
seed(t, s, "beta", "shared", []string{"beta-prod", "beta-stage"})
|
||||||
|
r := New(s)
|
||||||
|
b, err := r.Reach(as("beta"), "shared")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("beta: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// An omitted namespace on a bounded registration means "my namespaces", never
|
||||||
|
// "every namespace" — the empty string is what client-go reads as all of them, so
|
||||||
|
// leaking it here would turn every list op into a whole-cluster read.
|
||||||
|
scope, err := b.Scope("")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("scope: %v", err)
|
||||||
|
}
|
||||||
|
if len(scope) != 2 {
|
||||||
|
t.Fatalf("want the bound, got %v", scope)
|
||||||
|
}
|
||||||
|
for _, ns := range scope {
|
||||||
|
if ns == "" {
|
||||||
|
t.Fatal(`Scope("") returned the all-namespaces sentinel for a BOUNDED registration`)
|
||||||
|
}
|
||||||
|
if !strings.HasPrefix(ns, "beta-") {
|
||||||
|
t.Fatalf("scope leaked %q", ns)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Cluster-scoped reads cannot be narrowed at all, so a bounded registration must
|
||||||
|
// not be able to make one.
|
||||||
|
if err := b.WholeCluster(); err == nil {
|
||||||
|
t.Fatal("a namespace-bounded registration was allowed a cluster-scoped read")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUnboundedRegistrationReadsWholeCluster(t *testing.T) {
|
||||||
|
t.Setenv(AllowPrivateHostsEnv, "1")
|
||||||
|
s := newStore()
|
||||||
|
seed(t, s, "lux", "lux-k8s", nil)
|
||||||
|
r := New(s)
|
||||||
|
b, err := r.Reach(as("lux"), "lux-k8s")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("lux: %v", err)
|
||||||
|
}
|
||||||
|
// An org that registered its OWN cluster gave us the credential for all of it.
|
||||||
|
// Refusing it here would be a guard inventing a boundary the owner did not draw.
|
||||||
|
if err := b.WholeCluster(); err != nil {
|
||||||
|
t.Fatalf("owner of lux-k8s refused a cluster-scoped read: %v", err)
|
||||||
|
}
|
||||||
|
scope, err := b.Scope("")
|
||||||
|
if err != nil || len(scope) != 1 || scope[0] != "" {
|
||||||
|
t.Fatalf("want the all-namespaces sentinel for an unbounded registration, got %v %v", scope, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOrgHeaderWithoutPrincipalIsRefused(t *testing.T) {
|
||||||
|
s := newStore()
|
||||||
|
seed(t, s, "alpha", "prod", nil)
|
||||||
|
r := New(s)
|
||||||
|
|
||||||
|
// The full rule, not a non-empty check. Each of these is a caller stating
|
||||||
|
// something about itself that no validated principal backs.
|
||||||
|
for name, c := range map[string]zip.Caller{
|
||||||
|
"org header, no user": {Org: "alpha"},
|
||||||
|
"user, no org": {User: "u"},
|
||||||
|
"nothing at all": {},
|
||||||
|
"over the org bound": {Org: strings.Repeat("a", zip.MaxOrgLen+1), User: "u"},
|
||||||
|
"org escapes its ref": {Org: "alpha/../beta", User: "u"},
|
||||||
|
"org with a slash": {Org: "alpha/beta", User: "u"},
|
||||||
|
} {
|
||||||
|
ctx := zip.WithCaller(context.Background(), c)
|
||||||
|
if _, err := r.List(ctx); err == nil {
|
||||||
|
t.Fatalf("%s: the registry answered a call with no validated tenant", name)
|
||||||
|
} else if _, ok := err.(ErrNoTenant); !ok {
|
||||||
|
t.Fatalf("%s: want ErrNoTenant, got %T: %v", name, err, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestClusterNameCannotEscapeItsRef(t *testing.T) {
|
||||||
|
// A cluster name becomes a KMS ref segment, so a name containing a separator or a
|
||||||
|
// parent reference would address material outside the org's own prefix. This is the
|
||||||
|
// one place a name is checked, and Register is the only door it comes through.
|
||||||
|
for _, bad := range []string{
|
||||||
|
"../alpha/prod", "a/b", "prod/../../alpha", ".", "..", "-prod", "prod-",
|
||||||
|
"Prod", "pro d", "", strings.Repeat("p", MaxNameLen+1),
|
||||||
|
} {
|
||||||
|
if ValidName(bad) {
|
||||||
|
t.Fatalf("ValidName accepted %q, which can address another org's material", bad)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, ok := range []string{"prod", "hanzo-k8s", "lux-k8s", "zoo-k8s", "bootnode-k8s", "a1"} {
|
||||||
|
if !ValidName(ok) {
|
||||||
|
t.Fatalf("ValidName rejected the legitimate name %q", ok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRefsAreOrgPrefixed(t *testing.T) {
|
||||||
|
// The "orgs/<org>/" prefix is what makes the KMS app apply its OWN tenant rule to
|
||||||
|
// every ref this app resolves — a second refusal, in a second process. A ref shape
|
||||||
|
// that lost the prefix would silently drop that second gate, so it is asserted.
|
||||||
|
if got := indexRef("alpha"); got != "orgs/alpha/k8s/clusters" {
|
||||||
|
t.Fatalf("index ref %q is not under the org's KMS prefix", got)
|
||||||
|
}
|
||||||
|
if got := credRef("alpha", "prod"); got != "orgs/alpha/k8s/clusters/prod/kubeconfig" {
|
||||||
|
t.Fatalf("credential ref %q is not under the org's KMS prefix", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeregisterDestroysTheCredential(t *testing.T) {
|
||||||
|
t.Setenv(AllowPrivateHostsEnv, "1")
|
||||||
|
s := newStore()
|
||||||
|
seed(t, s, "beta", "prod", nil)
|
||||||
|
r := New(s)
|
||||||
|
ctx := as("beta")
|
||||||
|
|
||||||
|
removed, err := r.Deregister(ctx, "prod")
|
||||||
|
if err != nil || !removed {
|
||||||
|
t.Fatalf("deregister: %v %v", removed, err)
|
||||||
|
}
|
||||||
|
if v := s.data[credRef("beta", "prod")]; len(v) != 0 {
|
||||||
|
t.Fatalf("the sealed credential survived deregistration: %d bytes still readable", len(v))
|
||||||
|
}
|
||||||
|
if _, err := r.Reach(ctx, "prod"); err == nil {
|
||||||
|
t.Fatal("a deregistered cluster is still reachable")
|
||||||
|
}
|
||||||
|
// Twice is not an error: an operator retrying a deregistration must not be told
|
||||||
|
// something went wrong.
|
||||||
|
if removed, err := r.Deregister(ctx, "prod"); err != nil || removed {
|
||||||
|
t.Fatalf("second deregister: %v %v", removed, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeregisterCannotReachAcrossOrgs(t *testing.T) {
|
||||||
|
t.Setenv(AllowPrivateHostsEnv, "1")
|
||||||
|
s := newStore()
|
||||||
|
seed(t, s, "alpha", "prod", nil)
|
||||||
|
r := New(s)
|
||||||
|
|
||||||
|
// beta naming alpha's cluster must not remove it, and must not touch its material.
|
||||||
|
removed, err := r.Deregister(as("beta"), "prod")
|
||||||
|
if err != nil || removed {
|
||||||
|
t.Fatalf("beta deregistered alpha's cluster: %v %v", removed, err)
|
||||||
|
}
|
||||||
|
if len(s.data[credRef("alpha", "prod")]) == 0 {
|
||||||
|
t.Fatal("beta destroyed alpha's sealed credential")
|
||||||
|
}
|
||||||
|
if _, err := r.Reach(as("alpha"), "prod"); err != nil {
|
||||||
|
t.Fatalf("alpha lost its own cluster: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,94 @@
|
|||||||
|
package registry
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"net"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"k8s.io/client-go/rest"
|
||||||
|
"k8s.io/client-go/tools/clientcmd"
|
||||||
|
)
|
||||||
|
|
||||||
|
// AllowPrivateHostsEnv, when set, disables the non-routable-address rejection on
|
||||||
|
// an apiserver host. Off in production; set ONLY by a test whose apiserver runs on
|
||||||
|
// loopback.
|
||||||
|
const AllowPrivateHostsEnv = "K8S_ALLOW_PRIVATE_HOSTS"
|
||||||
|
|
||||||
|
// SafeRESTConfig parses a kubeconfig into a REST config, and is the ONLY place in
|
||||||
|
// this program that parses one. After the consolidation that created this repo,
|
||||||
|
// nobody else in the fleet holds a kubeconfig at all — which is what makes "one
|
||||||
|
// gate" a structural fact rather than a convention every new caller must remember.
|
||||||
|
//
|
||||||
|
// It REFUSES:
|
||||||
|
//
|
||||||
|
// - CREDENTIAL PLUGINS. rest.Config.ExecProvider runs a local binary with this
|
||||||
|
// process's full environment inherited, so a kubeconfig that carries one is
|
||||||
|
// remote code execution by whoever pasted it. AuthProvider likewise shells out.
|
||||||
|
// - SSRF TARGETS. The apiserver host is the ACTUAL dial target, so it is what
|
||||||
|
// gets guarded: https only, and never loopback, private, link-local, the
|
||||||
|
// instance-metadata address, unspecified or multicast. Guarding an endpoint
|
||||||
|
// that is not the one dialed is the shape of guard that does nothing.
|
||||||
|
//
|
||||||
|
// A pure DNS rebind after this check remains possible — client-go re-resolves at
|
||||||
|
// dial — and is the same accepted caveat every outbound-fetch guard in the fleet
|
||||||
|
// carries. It is stated rather than papered over.
|
||||||
|
func SafeRESTConfig(kubeconfig []byte) (*rest.Config, error) {
|
||||||
|
if len(kubeconfig) == 0 {
|
||||||
|
return nil, fmt.Errorf("kubeconfig is empty")
|
||||||
|
}
|
||||||
|
cfg, err := clientcmd.RESTConfigFromKubeConfig(kubeconfig)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("kubeconfig unusable: %w", err)
|
||||||
|
}
|
||||||
|
if cfg.ExecProvider != nil {
|
||||||
|
return nil, fmt.Errorf("kubeconfig uses an exec credential plugin, which is not permitted")
|
||||||
|
}
|
||||||
|
if cfg.AuthProvider != nil {
|
||||||
|
return nil, fmt.Errorf("kubeconfig uses an auth-provider plugin, which is not permitted")
|
||||||
|
}
|
||||||
|
if err := guardHost(cfg.Host); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return cfg, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func guardHost(rawHost string) error {
|
||||||
|
u, err := url.Parse(strings.TrimSpace(rawHost))
|
||||||
|
if err != nil || u.Host == "" {
|
||||||
|
return fmt.Errorf("cluster apiserver endpoint is not a valid URL")
|
||||||
|
}
|
||||||
|
if u.Scheme != "https" {
|
||||||
|
return fmt.Errorf("cluster apiserver endpoint must be https")
|
||||||
|
}
|
||||||
|
if os.Getenv(AllowPrivateHostsEnv) != "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
host := u.Hostname()
|
||||||
|
if host == "" || strings.EqualFold(host, "localhost") {
|
||||||
|
return fmt.Errorf("cluster apiserver host is not permitted")
|
||||||
|
}
|
||||||
|
var ips []net.IP
|
||||||
|
if ip := net.ParseIP(host); ip != nil {
|
||||||
|
ips = []net.IP{ip}
|
||||||
|
} else {
|
||||||
|
r, rerr := net.LookupIP(host)
|
||||||
|
if rerr != nil || len(r) == 0 {
|
||||||
|
return fmt.Errorf("cluster apiserver host does not resolve")
|
||||||
|
}
|
||||||
|
ips = r
|
||||||
|
}
|
||||||
|
for _, ip := range ips {
|
||||||
|
if blocked(ip) {
|
||||||
|
return fmt.Errorf("cluster apiserver resolves to a non-routable address")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func blocked(ip net.IP) bool {
|
||||||
|
return ip.IsLoopback() || ip.IsPrivate() || ip.IsLinkLocalUnicast() ||
|
||||||
|
ip.IsLinkLocalMulticast() || ip.IsInterfaceLocalMulticast() ||
|
||||||
|
ip.IsUnspecified() || ip.IsMulticast()
|
||||||
|
}
|
||||||
@@ -0,0 +1,591 @@
|
|||||||
|
// Package registry is the cluster registry — the core model of this app and the
|
||||||
|
// only door to a Kubernetes apiserver in this program.
|
||||||
|
//
|
||||||
|
// A cluster is a REGISTERED ENTITY: a name, a provider, an endpoint, a credential
|
||||||
|
// ref in KMS, and the org that owns it. Every operation NAMES its cluster, and
|
||||||
|
// the name is resolved through this registry against the tenant the call acts
|
||||||
|
// for. There is no ambient cluster and no in-cluster fallback: a function that
|
||||||
|
// could be called without saying which cluster would read whichever apiserver the
|
||||||
|
// process happened to be started next to, and that is precisely the assumption
|
||||||
|
// that cannot survive lux running lux-k8s while hanzo runs hanzo-k8s and a
|
||||||
|
// customer runs their own.
|
||||||
|
//
|
||||||
|
// # Multi-cluster is not a feature here, it is the shape
|
||||||
|
//
|
||||||
|
// [Registry.Reach] is the ONLY constructor of a Kubernetes client in this module,
|
||||||
|
// and it takes a name. That is the whole enforcement: no other function can
|
||||||
|
// obtain a clientset, so no other function can talk to a cluster it did not name.
|
||||||
|
// BYO-K8s, lux-k8s and hanzo-k8s are the same shape — three rows, one code path.
|
||||||
|
//
|
||||||
|
// # The credential never rests here
|
||||||
|
//
|
||||||
|
// A registration seals its kubeconfig in KMS under a ref DERIVED from the owning
|
||||||
|
// org and the cluster name, and stores no credential of its own. Reach resolves
|
||||||
|
// that ref at use, over the call plane, carrying the caller — so KMS re-enforces
|
||||||
|
// the tenant boundary independently of this package, and a registry row cannot
|
||||||
|
// name material it does not own because it does not name material at all. Nothing
|
||||||
|
// is written to disk, nothing is put in an environment variable, and nothing is
|
||||||
|
// logged.
|
||||||
|
//
|
||||||
|
// # Two independent gates, and neither is the apiserver's alone
|
||||||
|
//
|
||||||
|
// 1. The INDEX is per-org. A name is looked up only in the calling org's own
|
||||||
|
// index, so a name another org registered is NOT FOUND — not forbidden, which
|
||||||
|
// would confirm it exists.
|
||||||
|
// 2. The ROW bounds namespaces. A registration made with a scoped credential on a
|
||||||
|
// shared cluster carries the namespace set it may address, and [Bound.Scope]
|
||||||
|
// refuses anything outside it before the apiserver is dialed. The credential's
|
||||||
|
// own RBAC refuses it a second time.
|
||||||
|
package registry
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||||
|
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
|
||||||
|
"k8s.io/apimachinery/pkg/runtime/schema"
|
||||||
|
"k8s.io/client-go/dynamic"
|
||||||
|
"k8s.io/client-go/kubernetes"
|
||||||
|
"k8s.io/client-go/rest"
|
||||||
|
metricsv "k8s.io/metrics/pkg/client/clientset/versioned"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// probeTimeout bounds the one read a registration makes to prove a cluster is
|
||||||
|
// reachable with the credential it was handed. A cluster that cannot answer a
|
||||||
|
// node list in this long is not registered — failing at attach time is the only
|
||||||
|
// place the operator is still holding the kubeconfig and can fix it.
|
||||||
|
const probeTimeout = 15 * time.Second
|
||||||
|
|
||||||
|
// MaxNameLen bounds a cluster name. It is a DNS label bound because the name
|
||||||
|
// becomes a KMS ref segment and appears in log lines; an unbounded name is a
|
||||||
|
// store key an attacker chooses the size of.
|
||||||
|
const MaxNameLen = 63
|
||||||
|
|
||||||
|
// nodesGVR is the one coordinate this package reads for itself: a registration
|
||||||
|
// proves reachability by counting nodes and accelerators, which is also the
|
||||||
|
// inventory every fleet board wants.
|
||||||
|
var nodesGVR = schema.GroupVersionResource{Version: "v1", Resource: "nodes"}
|
||||||
|
|
||||||
|
// Cluster is a registered cluster as the index holds it.
|
||||||
|
//
|
||||||
|
// There is no Kubeconfig field and no Credential field, deliberately. The
|
||||||
|
// material is in KMS and its ref is DERIVED from Org and Name (see credRef) —
|
||||||
|
// storing the ref as data would let one edited row name another org's material,
|
||||||
|
// which is the same defect as letting a caller name its own org.
|
||||||
|
type Cluster struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
// Org is the org that owns this registration. A property of the record, set
|
||||||
|
// from the validated caller at register time and never from an argument.
|
||||||
|
Org string `json:"org"`
|
||||||
|
Provider string `json:"provider"`
|
||||||
|
Endpoint string `json:"endpoint,omitempty"`
|
||||||
|
Nodes int `json:"nodes"`
|
||||||
|
NvidiaGPU int `json:"nvidiaGpu"`
|
||||||
|
AmdGPU int `json:"amdGpu"`
|
||||||
|
Namespaces []string `json:"namespaces,omitempty"`
|
||||||
|
Registered string `json:"registered"`
|
||||||
|
Default bool `json:"default"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// View projects a record onto the wire type. It exists so the plane contract and
|
||||||
|
// the stored record can differ without either leaking into the other.
|
||||||
|
func (c Cluster) View() plane.Registered {
|
||||||
|
return plane.Registered{
|
||||||
|
Name: c.Name, Org: c.Org, Provider: c.Provider, Endpoint: c.Endpoint,
|
||||||
|
Nodes: c.Nodes, NvidiaGPU: c.NvidiaGPU, AmdGPU: c.AmdGPU,
|
||||||
|
Namespaces: c.Namespaces, Registered: c.Registered, Default: c.Default,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Store is where sealed material and the index live. It is an interface for ONE
|
||||||
|
// reason: a test must be able to exercise the tenancy guards without a KMS, and
|
||||||
|
// the guards are the part that must never be mocked away.
|
||||||
|
//
|
||||||
|
// The production implementation is the KMS peer over the call plane ([Sealed]).
|
||||||
|
type Store interface {
|
||||||
|
// Get returns the value at ref, or (nil, nil) when there is none. An absent
|
||||||
|
// value and an unreachable store are DIFFERENT answers: the first is empty,
|
||||||
|
// the second is an error, and collapsing them would make an outage look like
|
||||||
|
// an org with no clusters.
|
||||||
|
Get(ctx context.Context, ref string) ([]byte, error)
|
||||||
|
// Put writes value at ref.
|
||||||
|
Put(ctx context.Context, ref string, value []byte) error
|
||||||
|
}
|
||||||
|
|
||||||
|
// Registry is the per-deployment cluster registry.
|
||||||
|
type Registry struct {
|
||||||
|
store Store
|
||||||
|
|
||||||
|
mu sync.Mutex
|
||||||
|
bound map[string]*Bound // "org/name" -> live clients
|
||||||
|
}
|
||||||
|
|
||||||
|
// New opens a registry over a store.
|
||||||
|
func New(store Store) *Registry {
|
||||||
|
return &Registry{store: store, bound: map[string]*Bound{}}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bound is one cluster, resolved for one tenant: the record, the credential's
|
||||||
|
// clients, and the namespace bound the record carries.
|
||||||
|
//
|
||||||
|
// It is the only way to reach an apiserver in this module, and it cannot be
|
||||||
|
// constructed without a cluster name and a validated tenant.
|
||||||
|
type Bound struct {
|
||||||
|
Cluster Cluster
|
||||||
|
Typed kubernetes.Interface
|
||||||
|
Dyn dynamic.Interface
|
||||||
|
Metrics metricsv.Interface
|
||||||
|
Rest *rest.Config
|
||||||
|
}
|
||||||
|
|
||||||
|
// Scope resolves the namespaces an op may read, given the namespace it asked for.
|
||||||
|
//
|
||||||
|
// This is the ONE place the namespace boundary is enforced, and every namespaced
|
||||||
|
// op goes through it:
|
||||||
|
//
|
||||||
|
// - a named namespace inside the bound resolves to itself;
|
||||||
|
// - a named namespace outside the bound is REFUSED (not silently narrowed — a
|
||||||
|
// caller that asked for the wrong thing must be told, or it will believe an
|
||||||
|
// empty answer);
|
||||||
|
// - no namespace, on a bounded registration, resolves to the whole bound;
|
||||||
|
// - no namespace, on an unbounded registration, resolves to [""] — every
|
||||||
|
// namespace, which only an org that registered its OWN cluster ever gets.
|
||||||
|
func (b *Bound) Scope(ns string) ([]string, error) {
|
||||||
|
ns = strings.TrimSpace(ns)
|
||||||
|
bound := b.Cluster.Namespaces
|
||||||
|
if len(bound) == 0 {
|
||||||
|
if ns == "" {
|
||||||
|
return []string{metav1.NamespaceAll}, nil
|
||||||
|
}
|
||||||
|
return []string{ns}, nil
|
||||||
|
}
|
||||||
|
if ns == "" {
|
||||||
|
return append([]string(nil), bound...), nil
|
||||||
|
}
|
||||||
|
for _, allowed := range bound {
|
||||||
|
if allowed == ns {
|
||||||
|
return []string{ns}, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil, ErrNamespace{Cluster: b.Cluster.Name, Namespace: ns}
|
||||||
|
}
|
||||||
|
|
||||||
|
// One resolves exactly one namespace for an op that mutates or addresses a single
|
||||||
|
// object. An unbounded registration takes the caller's word; a bounded one must
|
||||||
|
// have the namespace named and inside its bound, because "act on all of them" is
|
||||||
|
// not a thing a mutation may mean.
|
||||||
|
func (b *Bound) One(ns string) (string, error) {
|
||||||
|
ns = strings.TrimSpace(ns)
|
||||||
|
if ns == "" {
|
||||||
|
return "", ErrNamespace{Cluster: b.Cluster.Name, Namespace: ""}
|
||||||
|
}
|
||||||
|
got, err := b.Scope(ns)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return got[0], nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// WholeCluster refuses a registration that does not address the whole cluster.
|
||||||
|
//
|
||||||
|
// Cluster-scoped reads — PersistentVolumes, Nodes, a cordon — cannot be narrowed
|
||||||
|
// to a namespace, so a scoped registration must not be able to make them at all.
|
||||||
|
// Answering them for a bounded row would hand a tenant on a shared cluster the
|
||||||
|
// node inventory of every other tenant on it.
|
||||||
|
func (b *Bound) WholeCluster() error {
|
||||||
|
if len(b.Cluster.Namespaces) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return ErrScope{Cluster: b.Cluster.Name}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ErrNoTenant is the refusal when no validated tenant rode along with the call.
|
||||||
|
//
|
||||||
|
// It is the FULL rule, not a non-empty check: zip.Tenant requires a validated
|
||||||
|
// user claim beside the org, refuses an empty org and refuses one over
|
||||||
|
// zip.MaxOrgLen. An org that arrived without a validated principal is a claim the
|
||||||
|
// caller made about itself.
|
||||||
|
type ErrNoTenant struct{}
|
||||||
|
|
||||||
|
func (ErrNoTenant) Error() string {
|
||||||
|
return "no validated tenant on this call: an org header without a validated principal is a claim, not an identity"
|
||||||
|
}
|
||||||
|
|
||||||
|
// ErrNoCluster is the refusal when a name is not in the calling org's registry.
|
||||||
|
//
|
||||||
|
// It says NOT FOUND rather than forbidden on purpose. A cluster another org
|
||||||
|
// registered must be indistinguishable from one that does not exist, or the
|
||||||
|
// refusal itself becomes an oracle for enumerating other tenants' cluster names.
|
||||||
|
type ErrNoCluster struct{ Name string }
|
||||||
|
|
||||||
|
func (e ErrNoCluster) Error() string {
|
||||||
|
return fmt.Sprintf("no cluster named %q is registered for this org", e.Name)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ErrNamespace is the refusal when a call names a namespace outside the bound its
|
||||||
|
// registration carries.
|
||||||
|
type ErrNamespace struct{ Cluster, Namespace string }
|
||||||
|
|
||||||
|
func (e ErrNamespace) Error() string {
|
||||||
|
if e.Namespace == "" {
|
||||||
|
return fmt.Sprintf("cluster %q requires a namespace: this registration is bounded to a namespace set", e.Cluster)
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("namespace %q is outside what this org's registration of cluster %q may address", e.Namespace, e.Cluster)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ErrScope is the refusal when a cluster-scoped read is asked of a registration
|
||||||
|
// bounded to namespaces.
|
||||||
|
type ErrScope struct{ Cluster string }
|
||||||
|
|
||||||
|
func (e ErrScope) Error() string {
|
||||||
|
return fmt.Sprintf("cluster %q is registered to this org with a namespace bound, so cluster-scoped reads are not available", e.Cluster)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ErrName is the refusal when a cluster name could not be a KMS ref segment.
|
||||||
|
type ErrName struct{ Name string }
|
||||||
|
|
||||||
|
func (e ErrName) Error() string {
|
||||||
|
return fmt.Sprintf("cluster name %q is not usable: 1-%d characters of a-z, 0-9 and '-', starting and ending alphanumeric", e.Name, MaxNameLen)
|
||||||
|
}
|
||||||
|
|
||||||
|
// List returns every cluster the tenant on this call may reach.
|
||||||
|
func (r *Registry) List(ctx context.Context) ([]Cluster, error) {
|
||||||
|
org, err := Tenant(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return r.index(ctx, org)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get returns one record by name, or ErrNoCluster.
|
||||||
|
func (r *Registry) Get(ctx context.Context, name string) (Cluster, error) {
|
||||||
|
org, err := Tenant(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return Cluster{}, err
|
||||||
|
}
|
||||||
|
list, err := r.index(ctx, org)
|
||||||
|
if err != nil {
|
||||||
|
return Cluster{}, err
|
||||||
|
}
|
||||||
|
for _, c := range list {
|
||||||
|
if c.Name == name {
|
||||||
|
return c, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return Cluster{}, ErrNoCluster{Name: name}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Register attaches a cluster to the calling org's registry.
|
||||||
|
//
|
||||||
|
// The kubeconfig is validated by [SafeRESTConfig], USED to prove the cluster
|
||||||
|
// answers, sealed in KMS, and then dropped. Registration is idempotent on name
|
||||||
|
// within the org: re-registering re-seals the credential and refreshes the
|
||||||
|
// inventory, which is how a rotated credential lands without a second verb.
|
||||||
|
//
|
||||||
|
// A cluster is registered into the CALLER's own scope and no other. There is no
|
||||||
|
// op that registers a cluster for a different org, which is what makes
|
||||||
|
// cross-org reach impossible rather than merely checked: to reach a cluster an
|
||||||
|
// org must hold a credential for it, sealed under its own KMS ref, and KMS
|
||||||
|
// refuses a peer asking for another org's material.
|
||||||
|
func (r *Registry) Register(ctx context.Context, in *plane.RegisterIn) (Cluster, error) {
|
||||||
|
org, err := Tenant(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return Cluster{}, err
|
||||||
|
}
|
||||||
|
name := strings.TrimSpace(in.Name)
|
||||||
|
if !ValidName(name) {
|
||||||
|
return Cluster{}, ErrName{Name: in.Name}
|
||||||
|
}
|
||||||
|
kube := []byte(in.Kubeconfig)
|
||||||
|
cfg, err := SafeRESTConfig(kube)
|
||||||
|
if err != nil {
|
||||||
|
return Cluster{}, err
|
||||||
|
}
|
||||||
|
dyn, err := dynamic.NewForConfig(cfg)
|
||||||
|
if err != nil {
|
||||||
|
return Cluster{}, fmt.Errorf("kubeconfig unusable: %w", err)
|
||||||
|
}
|
||||||
|
inv, err := inventory(ctx, dyn)
|
||||||
|
if err != nil {
|
||||||
|
return Cluster{}, fmt.Errorf("cluster unreachable with this credential: %w", err)
|
||||||
|
}
|
||||||
|
if err := r.store.Put(ctx, credRef(org, name), kube); err != nil {
|
||||||
|
return Cluster{}, fmt.Errorf("seal credential: %w", err)
|
||||||
|
}
|
||||||
|
rec := Cluster{
|
||||||
|
Name: name, Org: org, Provider: firstNonEmpty(in.Provider, "byo"),
|
||||||
|
Endpoint: cfg.Host, Nodes: inv.nodes, NvidiaGPU: inv.nvidia, AmdGPU: inv.amd,
|
||||||
|
Namespaces: cleanNamespaces(in.Namespaces),
|
||||||
|
Registered: time.Now().UTC().Format(time.RFC3339), Default: in.Default,
|
||||||
|
}
|
||||||
|
list, err := r.index(ctx, org)
|
||||||
|
if err != nil {
|
||||||
|
return Cluster{}, err
|
||||||
|
}
|
||||||
|
if err := r.writeIndex(ctx, org, upsert(list, rec)); err != nil {
|
||||||
|
return Cluster{}, err
|
||||||
|
}
|
||||||
|
r.forget(org, name)
|
||||||
|
return rec, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Deregister detaches a cluster from the calling org's registry and destroys the
|
||||||
|
// sealed credential by overwriting it.
|
||||||
|
//
|
||||||
|
// Overwrite rather than delete because the store's contract is get/put: a
|
||||||
|
// credential nothing can read is gone for every purpose this app has, and adding
|
||||||
|
// a third verb to reach the same end would be a second way to do one thing.
|
||||||
|
func (r *Registry) Deregister(ctx context.Context, name string) (bool, error) {
|
||||||
|
org, err := Tenant(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
list, err := r.index(ctx, org)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
kept := make([]Cluster, 0, len(list))
|
||||||
|
found := false
|
||||||
|
for _, c := range list {
|
||||||
|
if c.Name == name {
|
||||||
|
found = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
kept = append(kept, c)
|
||||||
|
}
|
||||||
|
if !found {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
if err := r.writeIndex(ctx, org, kept); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if err := r.store.Put(ctx, credRef(org, name), nil); err != nil {
|
||||||
|
return false, fmt.Errorf("destroy credential: %w", err)
|
||||||
|
}
|
||||||
|
r.forget(org, name)
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reach resolves a named cluster for the tenant on this call and returns its
|
||||||
|
// clients. It is the ONE constructor of a Kubernetes client in this module.
|
||||||
|
func (r *Registry) Reach(ctx context.Context, name string) (*Bound, error) {
|
||||||
|
org, err := Tenant(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
name = strings.TrimSpace(name)
|
||||||
|
if name == "" {
|
||||||
|
return nil, ErrNoCluster{Name: name}
|
||||||
|
}
|
||||||
|
rec, err := r.Get(ctx, name)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
key := org + "/" + name
|
||||||
|
r.mu.Lock()
|
||||||
|
b, ok := r.bound[key]
|
||||||
|
r.mu.Unlock()
|
||||||
|
if ok {
|
||||||
|
// The record may have moved (a namespace bound tightened, an inventory
|
||||||
|
// refreshed) while the clients stayed valid. Take the fresh record and
|
||||||
|
// keep the connections: a stale BOUND is a stale authorization.
|
||||||
|
b2 := *b
|
||||||
|
b2.Cluster = rec
|
||||||
|
return &b2, nil
|
||||||
|
}
|
||||||
|
raw, err := r.store.Get(ctx, credRef(org, name))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("resolve credential for cluster %q: %w", name, err)
|
||||||
|
}
|
||||||
|
if len(raw) == 0 {
|
||||||
|
return nil, fmt.Errorf("cluster %q is registered but its credential is gone from KMS; re-register it", name)
|
||||||
|
}
|
||||||
|
cfg, err := SafeRESTConfig(raw)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
cfg.UserAgent = "hanzo-k8s"
|
||||||
|
// client-go's default 5 QPS / burst 10 throttles a per-node fan-out badly: a
|
||||||
|
// 17-node kubelet sweep spent over a second queued, and at 100 nodes the
|
||||||
|
// queue alone approaches the read timeout. The read count is bounded by the
|
||||||
|
// op, so the real concurrency bound belongs there, not here.
|
||||||
|
cfg.QPS, cfg.Burst = 50, 100
|
||||||
|
typed, err := kubernetes.NewForConfig(cfg)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("cluster %q: %w", name, err)
|
||||||
|
}
|
||||||
|
dyn, err := dynamic.NewForConfig(cfg)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("cluster %q: %w", name, err)
|
||||||
|
}
|
||||||
|
mx, err := metricsv.NewForConfig(cfg)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("cluster %q: %w", name, err)
|
||||||
|
}
|
||||||
|
b = &Bound{Cluster: rec, Typed: typed, Dyn: dyn, Metrics: mx, Rest: cfg}
|
||||||
|
r.mu.Lock()
|
||||||
|
r.bound[key] = b
|
||||||
|
r.mu.Unlock()
|
||||||
|
return b, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// forget drops a cached client so the next Reach rebuilds it from KMS. Called
|
||||||
|
// whenever the credential behind it may have changed.
|
||||||
|
func (r *Registry) forget(org, name string) {
|
||||||
|
r.mu.Lock()
|
||||||
|
delete(r.bound, org+"/"+name)
|
||||||
|
r.mu.Unlock()
|
||||||
|
}
|
||||||
|
|
||||||
|
// index reads one org's records. An absent index is an empty registry, which is
|
||||||
|
// the correct answer for an org that has registered nothing.
|
||||||
|
func (r *Registry) index(ctx context.Context, org string) ([]Cluster, error) {
|
||||||
|
raw, err := r.store.Get(ctx, indexRef(org))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read cluster registry: %w", err)
|
||||||
|
}
|
||||||
|
if len(raw) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
var list []Cluster
|
||||||
|
if err := json.Unmarshal(raw, &list); err != nil {
|
||||||
|
return nil, fmt.Errorf("cluster registry for this org is corrupt: %w", err)
|
||||||
|
}
|
||||||
|
// The stored Org is authoritative for display, but a row that disagrees with
|
||||||
|
// the index it was read from is a defect: normalise to the index's own org so
|
||||||
|
// a hand-edited row cannot make a record claim a different owner.
|
||||||
|
for i := range list {
|
||||||
|
list[i].Org = org
|
||||||
|
}
|
||||||
|
return list, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Registry) writeIndex(ctx context.Context, org string, list []Cluster) error {
|
||||||
|
raw, err := json.Marshal(list)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return r.store.Put(ctx, indexRef(org), raw)
|
||||||
|
}
|
||||||
|
|
||||||
|
// indexRef is where an org's registry lives. The "orgs/<org>/" prefix is not
|
||||||
|
// decoration: the KMS app authorizes a ref that names a tenant against the
|
||||||
|
// tenant the call acts for, so this shape buys a second, independent refusal of
|
||||||
|
// a cross-org read from a process that is not this one.
|
||||||
|
func indexRef(org string) string { return "orgs/" + org + "/k8s/clusters" }
|
||||||
|
|
||||||
|
// credRef is where one cluster's kubeconfig is sealed. DERIVED, never stored: a
|
||||||
|
// ref that came out of a record could be edited to name another org's material.
|
||||||
|
func credRef(org, name string) string {
|
||||||
|
return "orgs/" + org + "/k8s/clusters/" + name + "/kubeconfig"
|
||||||
|
}
|
||||||
|
|
||||||
|
// ValidName reports whether a cluster name may be used. It must be a DNS label:
|
||||||
|
// the name becomes a KMS ref segment, so a name containing '/' or ".." would
|
||||||
|
// address material outside the org's own prefix.
|
||||||
|
func ValidName(name string) bool {
|
||||||
|
if name == "" || len(name) > MaxNameLen {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i := 0; i < len(name); i++ {
|
||||||
|
c := name[i]
|
||||||
|
switch {
|
||||||
|
case c >= 'a' && c <= 'z', c >= '0' && c <= '9':
|
||||||
|
case c == '-' && i > 0 && i < len(name)-1:
|
||||||
|
default:
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// cleanNamespaces normalises a namespace bound: trimmed, deduplicated, order
|
||||||
|
// preserved, empties dropped. An empty result means "the whole cluster", so a
|
||||||
|
// bound of only blanks must not silently become one — a caller that meant to
|
||||||
|
// bound and typed nothing gets no bound, which is why every op that can widen is
|
||||||
|
// gated on WholeCluster as well.
|
||||||
|
func cleanNamespaces(in []string) []string {
|
||||||
|
seen := map[string]bool{}
|
||||||
|
out := make([]string, 0, len(in))
|
||||||
|
for _, ns := range in {
|
||||||
|
ns = strings.TrimSpace(ns)
|
||||||
|
if ns == "" || seen[ns] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seen[ns] = true
|
||||||
|
out = append(out, ns)
|
||||||
|
}
|
||||||
|
if len(out) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
type inv struct{ nodes, nvidia, amd int }
|
||||||
|
|
||||||
|
// inventory is the reachability proof AND the fleet inventory, in one read. It
|
||||||
|
// is deliberately the same call for both: a "registered but unreachable" row is
|
||||||
|
// a row that will fail at first use, so the attach refuses instead.
|
||||||
|
func inventory(ctx context.Context, dyn dynamic.Interface) (inv, error) {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, probeTimeout)
|
||||||
|
defer cancel()
|
||||||
|
ul, err := dyn.Resource(nodesGVR).List(ctx, metav1.ListOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return inv{}, err
|
||||||
|
}
|
||||||
|
out := inv{nodes: len(ul.Items)}
|
||||||
|
for _, n := range ul.Items {
|
||||||
|
alloc, found, _ := unstructured.NestedMap(n.Object, "status", "allocatable")
|
||||||
|
if !found {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out.nvidia += QuantityInt(alloc["nvidia.com/gpu"])
|
||||||
|
out.amd += QuantityInt(alloc["amd.com/gpu"])
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// QuantityInt reads a Kubernetes quantity that is a plain count. An accelerator
|
||||||
|
// count is always an integer; anything else is zero rather than an error,
|
||||||
|
// because a node that reports a malformed GPU count must not fail the whole
|
||||||
|
// inventory.
|
||||||
|
func QuantityInt(v any) int {
|
||||||
|
s := strings.TrimSpace(fmt.Sprint(v))
|
||||||
|
if s == "" || s == "<nil>" {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
n := 0
|
||||||
|
for i := 0; i < len(s); i++ {
|
||||||
|
if s[i] < '0' || s[i] > '9' {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
n = n*10 + int(s[i]-'0')
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
func upsert(list []Cluster, rec Cluster) []Cluster {
|
||||||
|
for i, c := range list {
|
||||||
|
if c.Name == rec.Name {
|
||||||
|
list[i] = rec
|
||||||
|
return list
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return append(list, rec)
|
||||||
|
}
|
||||||
|
|
||||||
|
func firstNonEmpty(vals ...string) string {
|
||||||
|
for _, v := range vals {
|
||||||
|
if strings.TrimSpace(v) != "" {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
package registry
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
|
||||||
|
"github.com/hanzoai/k8s/plane"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Sealed is the production [Store]: the KMS peer, reached over the call plane.
|
||||||
|
//
|
||||||
|
// Every credential this app holds lives there and nowhere else. There is no
|
||||||
|
// kubeconfig on disk, no credential in an environment variable, and no path by
|
||||||
|
// which one is printed — the material is fetched at the moment of use, held for
|
||||||
|
// the length of one client construction, and never returned to a caller.
|
||||||
|
//
|
||||||
|
// The call carries the CALLER, so KMS applies its own tenant rule to the ref: a
|
||||||
|
// ref under "orgs/<org>/" is served only to a call acting for that org. That is a
|
||||||
|
// second refusal, in a second process, of the thing this package already refuses —
|
||||||
|
// and the reason the ref shape is derived rather than stored.
|
||||||
|
type Sealed struct{}
|
||||||
|
|
||||||
|
// Get reads one sealed value. An absent ref is (nil, nil): a registry with no
|
||||||
|
// index yet is an org with no clusters, which is not an error. An unreachable KMS
|
||||||
|
// IS an error, because answering "no clusters" during a KMS outage would make a
|
||||||
|
// tenant's whole fleet look deregistered.
|
||||||
|
func (Sealed) Get(ctx context.Context, ref string) ([]byte, error) {
|
||||||
|
out, err := zip.Ask[plane.SecretIn, plane.Secret](ctx, plane.KMS, plane.KMSGet, &plane.SecretIn{Ref: ref})
|
||||||
|
if err != nil {
|
||||||
|
if absent(err) {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return out.Value, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Put writes one sealed value. A nil value is a legitimate write: it destroys the
|
||||||
|
// material at that ref, which is how a deregistration disposes of a credential
|
||||||
|
// without this app needing a delete verb KMS does not offer.
|
||||||
|
func (Sealed) Put(ctx context.Context, ref string, value []byte) error {
|
||||||
|
_, err := zip.Ask[plane.SecretIn, plane.Secret](ctx, plane.KMS, plane.KMSPut, &plane.SecretIn{Ref: ref, Value: value})
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// absent reports whether an error means "there is nothing at that ref" rather
|
||||||
|
// than "the store did not answer". Only a 404 counts: every other status —
|
||||||
|
// including a 403, which means the ref belongs to another tenant — must surface,
|
||||||
|
// because treating a refusal as an empty registry is how a tenancy failure turns
|
||||||
|
// into a silently empty board.
|
||||||
|
func absent(err error) bool {
|
||||||
|
var he *zip.HTTPError
|
||||||
|
return errors.As(err, &he) && he.Status == http.StatusNotFound
|
||||||
|
}
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
package registry
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/zap-proto/zip"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Tenant is the org this call may act for, or a refusal.
|
||||||
|
//
|
||||||
|
// It is zip.Tenant plus one thing zip cannot know: the org becomes a KMS ref
|
||||||
|
// SEGMENT here, so a value containing '/' would address material outside the
|
||||||
|
// org's own prefix. That is a property of this app's storage, not of the
|
||||||
|
// framework's identity model, so it is checked here and only here.
|
||||||
|
//
|
||||||
|
// zip.Tenant is the FULL rule and not a non-empty check — no validated user claim
|
||||||
|
// means the org that rode along is a claim the caller made about itself, an empty
|
||||||
|
// org is a refusal, and an org over zip.MaxOrgLen is a refusal because the value
|
||||||
|
// is retained past the request as a store key. The org is used VERBATIM: trimmed,
|
||||||
|
// never lower-cased and never truncated, because folding collapses "acme",
|
||||||
|
// "ACME" and a 32-character prefix into one bucket, which is itself a
|
||||||
|
// cross-tenant break.
|
||||||
|
func Tenant(ctx context.Context) (string, error) {
|
||||||
|
org, ok := zip.Tenant(ctx)
|
||||||
|
if !ok {
|
||||||
|
return "", ErrNoTenant{}
|
||||||
|
}
|
||||||
|
if strings.ContainsAny(org, "/\\") || strings.Contains(org, "..") {
|
||||||
|
return "", ErrNoTenant{}
|
||||||
|
}
|
||||||
|
return org, nil
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user