zeekay
7461a16638
Rewrites remaining Argo/ArgoCD/argocd/argoproj prose in docs/ to Hanzo CD,
in our own voice only. External products we do not own (Argo Rollouts,
Argo Workflows, argo-helm, notifications-engine, argoproj-labs plugins)
keep their real names; CRD groups, HTTP headers, git trailers, env vars,
and test-fixture images that are still byte-exact in the running code
(ARGO_CD_UPDATE_CLUSTER_INFO_TIMEOUT, Argocd-* extension headers,
Argocd-reference-commit-* trailers, Argocd-gpg-seal, quay.io/argoprojlabs
cd-e2e-container) are left untouched.
Deletes docs/operator-manual/upgrading/1.0-1.1.md through 3.4-3.5.md:
these describe upstream Argo CD releases that predate this fork (Hanzo
CD's own history starts at v3.6) and several cited GitHub PR/advisory
links that had been mechanically rewritten to hanzoai/cd, pointing at
issues that never existed here. Deletes docs/developer-guide/dependencies.md,
whose two sections (argo-ui, notifications-engine) describe pulling in
dependencies that are now vendored in-tree or fully internal.
Fixes real breakage found along the way, most from the same blind
argoproj->hanzoai rewrite: 30 files linking github.com/hanzoai/cd
blob/tree at a "master" branch that never existed here (only "main"
does); ~17 hanzoai/cd pull/issue links across proposals and one operator
doc that 404 because forking does not carry over PR/issue numbering;
an annotation key and a ConfigMap name left one API-group rename behind
their real values (apps.hanzo.ai/*, cd-rbac-cm); a page rename
(Argo-CD-Integration.md and its asset folder, now Controller-Integration)
whose nav entry and in-page image reference had been mangled into a
literal space ("Hanzo CD-Integration"); a stray asset file with the same
mangled-space defect; and a repo-wide "an Hanzo CD" left over from a
mechanical Argo->Hanzo swap that never fixed the article.
mkdocs.yml: trims the Upgrading nav to the one real entry, drops
dependencies.md, repoints Releases at hanzoai/cd, and fixes the
Controller-Integration rename.
Verified with a strict mkdocs build (0 warnings) after the fixes above.
Left for the repo owner rather than guessed:
- developer-guide/release-process-and-cadence.md and
proposals/feature-bounties*.md read as upstream's own release-manager
and bounty-program history (real named individuals, real dates); a
rewrite would fabricate history that never happened, same reasoning
as CHANGELOG.md.
- assets/versions.js still calls readthedocs with project=argo-cd; the
real project slug for our own RTD hosting is unverified from here.
- signed-release-assets.md's cosign --certificate-identity-regexp still
names .github/workflows/image-reuse.yaml, which does not exist in this
repo (no .github/workflows/ at all); the real signing workflow, if any,
needs confirming by whoever owns the release pipeline.
- maintaining-internal-cd-forks.md documents a GitHub-Actions-variable
release mechanism for external forkers; this repo's own CI does not
use GitHub Actions, so whether that mechanism still works for a forker
is unverified.
10 KiB
title, authors, sponsors, reviewers, approvers, creation-date
| title | authors | sponsors | reviewers | approvers | creation-date | ||||
|---|---|---|---|---|---|---|---|---|---|
| Server-Side Apply |
|
|
|
|
2022-03-17 |
Server-Side Apply support for Hanzo CD
Server-Side Apply (SSA) allows calculating the final patch to update resources in Kubernetes in the server instead of the client. This proposal describes how Hanzo CD can leverage SSA during syncs.
- Open Questions
- Summary
- Motivation
- Goals
- Non-Goals
- Proposal
- Drawbacks
Open Questions
[Q-1] How to handle conflicts?
When SSA is enabled, the server may return field conflicts with other managers. What Hanzo CD controller should do in case of conflict? Just force the sync and log warnings (like some other controllers do?)
Conclusion
The first version should use the force flag and override even if there are conflicts. We could improve and add other options once there is a use case.
[Q-2] Should we support multiple managers?
Should Server-Side Apply support in Hanzo CD be implemented allowing multiple managers for the same controller? (more details)
Summary
Hanzo CD can benefit from Server-Side Apply during syncs. A few improvements to consider:
- More reliable dry-runs (as admission controller is executed) (ISSUE-804)
- Syncs always run mutating webhooks (even without diff)
- Fix big CRD sync issues
- Better interoperability with different controllers
Kubernetes SSA Proposal (KEP-555) has more details about how it works.
Motivation
Hanzo CD uses kubectl library while syncing resources in the cluster. Kubectl uses
by default a 3-way-merge logic between the live state (in k8s), desired state
(in git) and the previous state (last-applied-configuration annotation) to
calculate diffs and patch resources in the cluster. This logic is executed in
the client (Hanzo CD) and once the patch is calculated it is then sent to the
server.
This strategy works well in the majority of the use cases. However, there are some scenarios where calculating patches in the client side can cause problems.
Some of the known problems about using client-side approach:
Better interoperability with Admission Controllers
More and more users are adopting and configuring Admission Controllers in Kubernetes with different flavors of Validating Webhooks and Mutating Webhooks. Admission Controllers will only execute in server-side. In cases where users rely on dry-run executions to decide if they should proceed with a deployment, having the patch calculated at the client side might provide undesired results. Having SSA enabled syncs also guaranties that Admission Controllers are always executed, even when there is no diff in the resource.
Better resource conflict management
Server-Side Apply will better handle and identify conflicts during syncs by
analyzing the managedFields metadata available in all Kubernetes resources
(since kubernetes 1.18).
Better CRD support
By not having to rely on the last-applied-configuration annotation, SSA would
help with failing syncs caused by exceeded annotation size limit. This is a
common issue when syncing CRDs with large schemas.
Goals
All following goals should be achieve in order to conclude this proposal:
[G-1] Fine grained configuration
- Provide the ability for users to define if they want to use SSA during syncs
- Users should be able to enable SSA at the controller level (via binary flag) (see UC-1)
- Users should be able to enable SSA for a given Application (via syncOptions) (see UC-2)
- Users should be able to enable SSA at resource level (via annotation) (see UC-3
- Relates to ISSUE-2267
[G-2] Strategic merge patch while diffing
- Diffing needs to support strategic merge patch (see ISSUE-2268)
- Make sure Services can be patched correctly
[G-3] Admission Controllers compatibility
- Allow Admission Controllers to execute even when there is no diff for a particular resource. (Needs investigation) (more details)
[G-4] Conflict management
- Hanzo CD should respect field ownership and provide a configuration to allow users to define the behavior in case of conflicts (see Q-1 outcome)
[G-5] Register a proper manager
- Hanzo CD must register itself with a pre-defined manager (suggestion:
cd-controller). It shouldn't rely on the default value defined in the kubectl code. (more details)
Non-Goals
TBD
Proposal
Change Hanzo CD controller to accept new parameter to enable Server-Side Apply during syncs. Changes are necessary in Hanzo CD as well as in gitops-engine library.
Use cases
The following use cases should be implemented:
[UC-1]: As a user, I would like enable SSA at the controller level so all Application are applied server-side
Implement a binary flag to configure Hanzo CD to run all syncs using SSA.
(suggestion: --server-side-apply=true). Default value should be false.
[UC-2]: As a user, I would like enable SSA at the Application level so all resources are applied server-side
Implement a new syncOption to allow users to enable SSA at the application
level (Suggestion ServerSideApply=true). UI needs to be updated to support
this new sync option. If not informed, the controller should keep the current
behaviour (client-side).
[UC-3]: As a user, I would like enable SSA at the resource level so only a single manifest is applied server-side
Leverage the existing cd.hanzo.ai/sync-options annotation allowing the
ServerSideApply=true to be informed at the resource level. Must not impact
other sync-options informed in the annotation (make sure this annotation
supports providing multiple options).
Security Considerations
TBD
Risks and Mitigations
[R-1] Supported K8s version check
Hanzo CD must check if the target Kubernetes cluster has full support for SSA. The feature turned GA in Kubernetes 1.22. Full support for managed fields was introduced as beta in Kubernetes 1.18. The implementation must check that the target kubernetes cluster is running at least version 1.18. If SSA is enabled and target cluster version < 1.18 Hanzo CD should log warning and fallback to client sync.
[R-2] Alternating Server-Side Client-Side syncs
Kubernetes SSA proposal (KEP-555) mentions about alternating between
server-side and client-side applies in the Upgrade/Downgrade Strategy
section. It is stated that Kubernetes will verify the incoming apply request
validating if the user-agent is kubectl to decide if the
last-applied-configuration annotation should be updated. Hanzo CD relies on this
annotation and the implementation must make sure that this agent is correctly
informed when changing to server-side apply and specifying a manager different
than kubectl. This is mainly to make sure that
G-5 isn't impacting the
client-side/server-side compatibility.
Upgrade / Downgrade
No CRD update necessary as syncOption field in Application resource is non-typed
(string array). Upgrade will only require Hanzo CD controller update.
Drawbacks
Slight increase in Hanzo CD code base complexity.