mirror of
https://github.com/hanzoai/deploy.git
synced 2026-10-07 08:44:50 +00:00
41536738c8
* feat(api): hard-fork CRD API group argoproj.io -> deploy.hanzo.ai
Hanzo CD hard-fork, phase 1.1. Single source of truth is
pkg/apis/application/register.go's Group constant (Application/AppProject/
ApplicationSet all derive their FullName from it); every other Go call-site
that hardcoded the literal group string is updated to match: the 3 generated
informer GVRs, the app-of-apps nested-Application detector in types.go, the
resource-tree Application-kind cache populator, the 3 NotFound error
constructors in server/application, and the applicationset-controller's
leader-election Lease name/ID (kept in lockstep across Go source + RBAC
resourceNames, or leader election would get RBAC-denied on startup).
CRD YAMLs (application/appproject/applicationset) and every RBAC apiGroups
rule that actually governs our 3 CRDs are repointed to deploy.hanzo.ai, across
both manifests/base/* (namespaced Roles) and manifests/cluster-rbac/* (cluster
scope). Two adjacent things that share the "argoproj.io" string but are NOT
ours are deliberately left untouched: gitops-engine's health-check dispatch
for Argo Workflows' `Workflow` kind, and argocd-server-clusterrole's second
apiGroups block granting read on Argo Workflows' `workflows` resource — both
belong to a separate, unrelated Argoproj-family project we are not forking.
Also untouched, per explicit scope: annotation/label keys (argocd.argoproj.io/*)
and finalizers (*.argoproj.io) that track already-applied live resources, and
argocd-cm.yaml's resource.customizations.ignoreResourceUpdates keys (a narrow
app-of-apps diffing nicety, not build- or RBAC-relevant).
CGO_ENABLED=0 go build ./... verified green.
* ci: wire canonical hanzo.yml + cicd.yml (Hanzo CD image build)
Phase 1.3. Follows the same one-way pattern every hanzo/lux/zoo repo uses:
hanzo.yml declares the image (root Dockerfile -> ghcr.io/hanzoai/deploy) and a
fast go-build sanity test (the multi-stage Dockerfile build is the slow, real
gate); cicd.yml is the ~10-line trigger shim importing hanzoai/ci's reusable
build workflow on the arc pool, secrets from KMS only.
No `deploy:` block yet, matching the hanzoai/cms build-only-first-cutover
pattern: this repo's own live hanzo-cd namespace runs the whole Hanzo fleet's
GitOps, so the prod image/CRD swap is a deliberate separate action (see the
[HELD] universe PR), never an automatic rollout on merge. Every PR/push still
builds and pushes an immutable ghcr.io/hanzoai/deploy:sha-<7>-amd64 image.
* feat(ui): rebrand Argo CD -> Hanzo CD (title, favicon, logo, sidebar)
Phase 1.2. User-facing "Argo CD"/"ArgoCD" text replaced with "Hanzo CD"
across the login page, help modal, settings screens, version-info panel,
sync-options warning, fullscreen-logs title, app.tsx page titles, and
index.html's <title>/noscript text (dropped the now-wrong upstream CLI
doc link rather than invent a hanzo.ai URL that doesn't exist).
Logo/favicon assets replaced with the canonical Hanzo mark (never
hand-drawn, copied from ~/work/hanzo/logo/dist): the two real usages of
Argo's logo collapse onto one Hanzo glyph (mono + white variants) at the
same 5 call sites (login icon + footer, login.scss $logoPath/
$logoNegativePath, sidebar text-logo), preserving the existing CSS
invert() filters so both the light login panel and dark sidebar still
render correctly. Old argologo.svg/argo_o.svg/argo.png deleted (no
longer referenced). favicon-{16x16,32x32}.png added at
ui/src/assets/favicon/ (index.html's links to that path were previously
dead - the directory didn't exist in this checkout). package.json name
argo-cd-ui -> hanzo-cd-ui.
Left alone, deliberately: the two Kubernetes resource-kind icon sets
under ui/src/assets/images/resources/ (unrelated to branding), and
upstream doc/project links (argoproj.io, argo-cd.readthedocs.io) since
we have no equivalent hanzo.ai docs to point to yet - except the login
footer logo's outbound link, which now points to hanzo.ai instead of
argoproj.io (a Hanzo-branded mark linking to the upstream project it
was forked from read as an obvious leftover).
* feat(module): rename Go module github.com/argoproj/argo-cd/v3 -> github.com/hanzoai/deploy
Phase 2. No version suffix on the new path (org policy: never publish our
own Go modules above v1.x.x / with a v2+ suffix) — this fork starts a fresh
v0/v1 module identity rather than carrying the v3 the upstream fork point
had reached.
Mechanical, symmetric rewrite: 704 .go files (import statements only; two
disjoint substitutions applied specific-first so the nested gitops-engine
module path never double-matches), plus go.mod (module line, the
gitops-engine require+replace pair, version reset to v0.0.0-... since a
suffix-less path can't carry a v3 semver), gitops-engine/go.mod, Makefile
(ldflags PACKAGE + 2 TEST_MODULE filters), .golangci.yaml, .goreleaser.yaml,
.mockery.yaml (+ gitops-engine's), Procfile, the codegen/proto generation
scripts (hack/generate-proto.sh, update-codegen.sh, update-openapi.sh), and
20 .proto files' go_package options. Also one OpenTelemetry tracer name
string in gitops-engine/pkg/sync/sync_context.go (a cosmestic label, not an
import, but worth keeping consistent).
CGO_ENABLED=0 go build ./... verified green for both the root module and
the nested gitops-engine module; go vet clean on both; go mod tidy run
(network available) to reconcile go.sum, dependency set unchanged.
Left alone, deliberately: doc-only mentions (docs/snyk/*.html scan reports,
gitops-engine/README.md), a comment-only reference in
.github/workflows/release.yaml, and one Renovate bot config line — none are
build- or codegen-load-bearing. Also left alone: comments citing upstream
GitHub issue URLs (github.com/argoproj/argo-cd/issues/NNNNN) and a handful
of test fixtures using "https://github.com/argoproj/argo-cd.git" as a
representative example repo URL — both are accurate historical/test-data
references, not stale self-references.
---------
Co-authored-by: Hanzo Dev <[email protected]>
879 lines
33 KiB
Go
879 lines
33 KiB
Go
package oci
|
|
|
|
import (
|
|
"archive/tar"
|
|
"bytes"
|
|
"compress/gzip"
|
|
"context"
|
|
"crypto/tls"
|
|
"encoding/json"
|
|
"errors"
|
|
"io"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"net/url"
|
|
"os"
|
|
"path/filepath"
|
|
"slices"
|
|
"testing"
|
|
|
|
"github.com/opencontainers/go-digest"
|
|
"github.com/opencontainers/image-spec/specs-go"
|
|
imagev1 "github.com/opencontainers/image-spec/specs-go/v1"
|
|
"github.com/stretchr/testify/assert"
|
|
"github.com/stretchr/testify/require"
|
|
"oras.land/oras-go/v2"
|
|
"oras.land/oras-go/v2/content"
|
|
"oras.land/oras-go/v2/content/memory"
|
|
|
|
utilio "github.com/hanzoai/deploy/util/io"
|
|
"github.com/hanzoai/deploy/util/io/files"
|
|
)
|
|
|
|
type layerConf struct {
|
|
desc imagev1.Descriptor
|
|
bytes []byte
|
|
}
|
|
|
|
func generateManifest(t *testing.T, store *memory.Store, layerDescs ...layerConf) string {
|
|
t.Helper()
|
|
return generateManifestWithConfig(t, store, imagev1.MediaTypeImageConfig, layerDescs...)
|
|
}
|
|
|
|
func generateManifestWithConfig(t *testing.T, store *memory.Store, configMediaType string, layerDescs ...layerConf) string {
|
|
t.Helper()
|
|
configBlob := []byte("Hello config")
|
|
configDesc := content.NewDescriptorFromBytes(configMediaType, configBlob)
|
|
|
|
var layers []imagev1.Descriptor
|
|
|
|
for _, layer := range layerDescs {
|
|
layers = append(layers, layer.desc)
|
|
}
|
|
|
|
manifestBlob, err := json.Marshal(imagev1.Manifest{
|
|
Config: configDesc,
|
|
Layers: layers,
|
|
Versioned: specs.Versioned{SchemaVersion: 2},
|
|
})
|
|
require.NoError(t, err)
|
|
manifestDesc := content.NewDescriptorFromBytes(imagev1.MediaTypeImageManifest, manifestBlob)
|
|
|
|
for _, layer := range layerDescs {
|
|
require.NoError(t, store.Push(t.Context(), layer.desc, bytes.NewReader(layer.bytes)))
|
|
}
|
|
|
|
require.NoError(t, store.Push(t.Context(), configDesc, bytes.NewReader(configBlob)))
|
|
require.NoError(t, store.Push(t.Context(), manifestDesc, bytes.NewReader(manifestBlob)))
|
|
require.NoError(t, store.Tag(t.Context(), manifestDesc, manifestDesc.Digest.String()))
|
|
|
|
return manifestDesc.Digest.String()
|
|
}
|
|
|
|
func createGzippedTarWithContent(t *testing.T, filename, content string) []byte {
|
|
t.Helper()
|
|
var buf bytes.Buffer
|
|
gzw := gzip.NewWriter(&buf)
|
|
tw := tar.NewWriter(gzw)
|
|
|
|
require.NoError(t, tw.WriteHeader(&tar.Header{
|
|
Name: filename,
|
|
Mode: 0o644,
|
|
Size: int64(len(content)),
|
|
}))
|
|
_, err := tw.Write([]byte(content))
|
|
require.NoError(t, err)
|
|
require.NoError(t, tw.Close())
|
|
require.NoError(t, gzw.Close())
|
|
|
|
return buf.Bytes()
|
|
}
|
|
|
|
func addFileToDirectory(t *testing.T, dir, filename, content string) {
|
|
t.Helper()
|
|
|
|
filePath := filepath.Join(dir, filename)
|
|
err := os.WriteFile(filePath, []byte(content), 0o644)
|
|
require.NoError(t, err)
|
|
}
|
|
|
|
func Test_nativeOCIClient_Extract(t *testing.T) {
|
|
cacheDir := utilio.NewRandomizedTempPaths(t.TempDir())
|
|
|
|
type fields struct {
|
|
repoURL string
|
|
tagsFunc func(context.Context, string) (tags []string, err error)
|
|
allowedMediaTypes []string
|
|
}
|
|
type args struct {
|
|
manifestMaxExtractedSize int64
|
|
disableManifestMaxExtractedSize bool
|
|
digestFunc func(*memory.Store) string
|
|
postValidationFunc func(string, string, Client, fields, args)
|
|
}
|
|
tests := []struct {
|
|
name string
|
|
fields fields
|
|
args args
|
|
expectedError error
|
|
}{
|
|
{
|
|
name: "extraction fails due to size limit",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{imagev1.MediaTypeImageLayerGzip},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
layerBlob := createGzippedTarWithContent(t, "some-path", "some content")
|
|
return generateManifest(t, store, layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, layerBlob), layerBlob})
|
|
},
|
|
manifestMaxExtractedSize: 10,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
expectedError: errors.New("cannot extract contents of oci image with revision sha256:1b6dfd71e2b35c2f35dffc39007c2276f3c0e235cbae4c39cba74bd406174e22: failed to perform \"Push\" on destination: could not decompress layer: error while iterating on tar reader: unexpected EOF"),
|
|
},
|
|
{
|
|
name: "extraction fails due to multiple content layers",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{imagev1.MediaTypeImageLayerGzip},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
layerBlob := createGzippedTarWithContent(t, "some-path", "some content")
|
|
otherLayerBlob := createGzippedTarWithContent(t, "some-other-path", "some other content")
|
|
return generateManifest(t, store, layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, layerBlob), layerBlob}, layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, otherLayerBlob), otherLayerBlob})
|
|
},
|
|
manifestMaxExtractedSize: 1000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
expectedError: errors.New("expected only a single oci content layer, got 2"),
|
|
},
|
|
{
|
|
name: "extraction with multiple layers, but just a single content layer",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{imagev1.MediaTypeImageLayerGzip},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
layerBlob := createGzippedTarWithContent(t, "some-path", "some content")
|
|
return generateManifest(t, store, layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, layerBlob), layerBlob}, layerConf{content.NewDescriptorFromBytes("application/vnd.cncf.helm.chart.provenance.v1.prov", []byte{}), []byte{}})
|
|
},
|
|
manifestMaxExtractedSize: 1000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
},
|
|
{
|
|
name: "extraction fails due to invalid media type",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{"application/vnd.different.media.type"},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
layerBlob := "Hello layer"
|
|
return generateManifest(t, store, layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, []byte(layerBlob)), []byte(layerBlob)})
|
|
},
|
|
manifestMaxExtractedSize: 1000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
expectedError: errors.New("oci layer media type application/vnd.oci.image.layer.v1.tar+gzip is not in the list of allowed media types"),
|
|
},
|
|
{
|
|
name: "extraction fails due to non-existent digest",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{"application/vnd.cncf.helm.chart.content.v1.tar+gzip"},
|
|
},
|
|
args: args{
|
|
digestFunc: func(_ *memory.Store) string {
|
|
return "sha256:nonexistentdigest"
|
|
},
|
|
manifestMaxExtractedSize: 1000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
expectedError: errors.New("error resolving oci manifest for digest sha256:nonexistentdigest: sha256:nonexistentdigest: not found"),
|
|
},
|
|
{
|
|
name: "extraction with helm chart",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{"application/vnd.cncf.helm.chart.content.v1.tar+gzip"},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
chartDir := t.TempDir()
|
|
chartName := "mychart"
|
|
|
|
parent := filepath.Join(chartDir, "parent")
|
|
require.NoError(t, os.Mkdir(parent, 0o755))
|
|
|
|
chartPath := filepath.Join(parent, chartName)
|
|
require.NoError(t, os.Mkdir(chartPath, 0o755))
|
|
|
|
addFileToDirectory(t, chartPath, "Chart.yaml", "some content")
|
|
|
|
temp, err := os.CreateTemp(t.TempDir(), "")
|
|
require.NoError(t, err)
|
|
defer temp.Close()
|
|
_, err = files.Tgz(parent, nil, nil, temp)
|
|
require.NoError(t, err)
|
|
_, err = temp.Seek(0, io.SeekStart)
|
|
require.NoError(t, err)
|
|
all, err := io.ReadAll(temp)
|
|
|
|
require.NoError(t, err)
|
|
|
|
return generateManifest(t, store, layerConf{content.NewDescriptorFromBytes("application/vnd.cncf.helm.chart.content.v1.tar+gzip", all), all})
|
|
},
|
|
postValidationFunc: func(_, path string, _ Client, _ fields, _ args) {
|
|
tempDir, err := files.CreateTempDir(os.TempDir())
|
|
defer os.RemoveAll(tempDir)
|
|
require.NoError(t, err)
|
|
chartDir, err := os.ReadDir(path)
|
|
require.NoError(t, err)
|
|
require.Len(t, chartDir, 1)
|
|
require.Equal(t, "Chart.yaml", chartDir[0].Name())
|
|
chartYaml, err := os.Open(filepath.Join(path, chartDir[0].Name()))
|
|
require.NoError(t, err)
|
|
contents, err := io.ReadAll(chartYaml)
|
|
require.NoError(t, err)
|
|
require.Equal(t, "some content", string(contents))
|
|
},
|
|
manifestMaxExtractedSize: 10000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
},
|
|
{
|
|
name: "extraction with standard gzip layer",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{imagev1.MediaTypeImageLayerGzip},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
layerBlob := createGzippedTarWithContent(t, "foo.yaml", "some content")
|
|
return generateManifest(t, store, layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, layerBlob), layerBlob})
|
|
},
|
|
postValidationFunc: func(_, path string, _ Client, _ fields, _ args) {
|
|
manifestDir, err := os.ReadDir(path)
|
|
require.NoError(t, err)
|
|
require.Len(t, manifestDir, 1)
|
|
require.Equal(t, "foo.yaml", manifestDir[0].Name())
|
|
f, err := os.Open(filepath.Join(path, manifestDir[0].Name()))
|
|
require.NoError(t, err)
|
|
contents, err := io.ReadAll(f)
|
|
require.NoError(t, err)
|
|
require.Equal(t, "some content", string(contents))
|
|
},
|
|
manifestMaxExtractedSize: 1000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
},
|
|
{
|
|
name: "extraction with docker rootfs tar.gzip layer",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{"application/vnd.docker.image.rootfs.diff.tar.gzip"},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
layerBlob := createGzippedTarWithContent(t, "foo.yaml", "some content")
|
|
return generateManifest(t, store, layerConf{content.NewDescriptorFromBytes("application/vnd.docker.image.rootfs.diff.tar.gzip", layerBlob), layerBlob})
|
|
},
|
|
postValidationFunc: func(_, path string, _ Client, _ fields, _ args) {
|
|
manifestDir, err := os.ReadDir(path)
|
|
require.NoError(t, err)
|
|
require.Len(t, manifestDir, 1)
|
|
require.Equal(t, "foo.yaml", manifestDir[0].Name())
|
|
f, err := os.Open(filepath.Join(path, manifestDir[0].Name()))
|
|
require.NoError(t, err)
|
|
contents, err := io.ReadAll(f)
|
|
require.NoError(t, err)
|
|
require.Equal(t, "some content", string(contents))
|
|
},
|
|
manifestMaxExtractedSize: 1000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
},
|
|
{
|
|
name: "extraction with standard gzip layer using cache",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{imagev1.MediaTypeImageLayerGzip},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
layerBlob := createGzippedTarWithContent(t, "foo.yaml", "some content")
|
|
return generateManifest(t, store, layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, layerBlob), layerBlob})
|
|
},
|
|
manifestMaxExtractedSize: 1000,
|
|
disableManifestMaxExtractedSize: false,
|
|
postValidationFunc: func(sha string, _ string, _ Client, fields fields, args args) {
|
|
store := memory.New()
|
|
c := newClientWithLock(fields.repoURL, globalLock, store, fields.tagsFunc, func(_ context.Context) error {
|
|
return nil
|
|
}, fields.allowedMediaTypes,
|
|
WithImagePaths(cacheDir),
|
|
WithManifestMaxExtractedSize(args.manifestMaxExtractedSize),
|
|
WithDisableManifestMaxExtractedSize(args.disableManifestMaxExtractedSize),
|
|
WithEventHandlers(fakeEventHandlers(t, fields.repoURL)))
|
|
_, gotCloser, err := c.Extract(t.Context(), sha)
|
|
require.NoError(t, err)
|
|
require.NoError(t, gotCloser.Close())
|
|
},
|
|
},
|
|
},
|
|
{
|
|
name: "helm chart with multiple layers (provenance + chart content) should succeed",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{"application/vnd.cncf.helm.chart.content.v1.tar+gzip"},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
chartDir := t.TempDir()
|
|
chartName := "mychart"
|
|
|
|
parent := filepath.Join(chartDir, "parent")
|
|
require.NoError(t, os.Mkdir(parent, 0o755))
|
|
|
|
chartPath := filepath.Join(parent, chartName)
|
|
require.NoError(t, os.Mkdir(chartPath, 0o755))
|
|
|
|
addFileToDirectory(t, chartPath, "Chart.yaml", "helm chart content")
|
|
|
|
temp, err := os.CreateTemp(t.TempDir(), "")
|
|
require.NoError(t, err)
|
|
defer temp.Close()
|
|
_, err = files.Tgz(parent, nil, nil, temp)
|
|
require.NoError(t, err)
|
|
_, err = temp.Seek(0, io.SeekStart)
|
|
require.NoError(t, err)
|
|
chartBlob, err := io.ReadAll(temp)
|
|
require.NoError(t, err)
|
|
|
|
// Create provenance layer
|
|
provenanceBlob := []byte("provenance data")
|
|
|
|
return generateManifestWithConfig(t, store, "application/vnd.cncf.helm.config.v1+json",
|
|
layerConf{content.NewDescriptorFromBytes("application/vnd.cncf.helm.chart.content.v1.tar+gzip", chartBlob), chartBlob},
|
|
layerConf{content.NewDescriptorFromBytes("application/vnd.cncf.helm.chart.provenance.v1.prov", provenanceBlob), provenanceBlob})
|
|
},
|
|
postValidationFunc: func(_, path string, _ Client, _ fields, _ args) {
|
|
// Verify only chart content was extracted, not provenance
|
|
chartDir, err := os.ReadDir(path)
|
|
require.NoError(t, err)
|
|
require.Len(t, chartDir, 1)
|
|
require.Equal(t, "Chart.yaml", chartDir[0].Name())
|
|
|
|
chartYaml, err := os.Open(filepath.Join(path, chartDir[0].Name()))
|
|
require.NoError(t, err)
|
|
contents, err := io.ReadAll(chartYaml)
|
|
require.NoError(t, err)
|
|
require.Equal(t, "helm chart content", string(contents))
|
|
},
|
|
manifestMaxExtractedSize: 10000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
},
|
|
{
|
|
name: "helm chart with multiple layers (attestation + provenance + chart content) should succeed",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{"application/vnd.cncf.helm.chart.content.v1.tar+gzip"},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
chartDir := t.TempDir()
|
|
chartName := "mychart"
|
|
|
|
parent := filepath.Join(chartDir, "parent")
|
|
require.NoError(t, os.Mkdir(parent, 0o755))
|
|
|
|
chartPath := filepath.Join(parent, chartName)
|
|
require.NoError(t, os.Mkdir(chartPath, 0o755))
|
|
|
|
addFileToDirectory(t, chartPath, "Chart.yaml", "multi-layer chart")
|
|
addFileToDirectory(t, chartPath, "values.yaml", "key: value")
|
|
|
|
temp, err := os.CreateTemp(t.TempDir(), "")
|
|
require.NoError(t, err)
|
|
defer temp.Close()
|
|
_, err = files.Tgz(parent, nil, nil, temp)
|
|
require.NoError(t, err)
|
|
_, err = temp.Seek(0, io.SeekStart)
|
|
require.NoError(t, err)
|
|
chartBlob, err := io.ReadAll(temp)
|
|
require.NoError(t, err)
|
|
|
|
// Create multiple non-content layers
|
|
attestationBlob := []byte("attestation data")
|
|
provenanceBlob := []byte("provenance data")
|
|
|
|
return generateManifestWithConfig(t, store, "application/vnd.cncf.helm.config.v1+json",
|
|
layerConf{content.NewDescriptorFromBytes("application/vnd.in-toto+json", attestationBlob), attestationBlob},
|
|
layerConf{content.NewDescriptorFromBytes("application/vnd.cncf.helm.chart.content.v1.tar+gzip", chartBlob), chartBlob},
|
|
layerConf{content.NewDescriptorFromBytes("application/vnd.cncf.helm.chart.provenance.v1.prov", provenanceBlob), provenanceBlob})
|
|
},
|
|
postValidationFunc: func(_, path string, _ Client, _ fields, _ args) {
|
|
// Verify only chart content was extracted
|
|
chartDir, err := os.ReadDir(path)
|
|
require.NoError(t, err)
|
|
require.Len(t, chartDir, 2) // Chart.yaml and values.yaml
|
|
|
|
files := make(map[string]bool)
|
|
for _, f := range chartDir {
|
|
files[f.Name()] = true
|
|
}
|
|
require.True(t, files["Chart.yaml"])
|
|
require.True(t, files["values.yaml"])
|
|
|
|
// Ensure no provenance or attestation files were extracted
|
|
require.False(t, files["provenance"])
|
|
require.False(t, files["attestation"])
|
|
},
|
|
manifestMaxExtractedSize: 10000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
},
|
|
{
|
|
name: "helm chart with only provenance layer should fail (no chart content)",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{"application/vnd.cncf.helm.chart.content.v1.tar+gzip"},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
provenanceBlob := []byte("provenance data")
|
|
return generateManifestWithConfig(t, store, "application/vnd.cncf.helm.config.v1+json",
|
|
layerConf{content.NewDescriptorFromBytes("application/vnd.cncf.helm.chart.provenance.v1.prov", provenanceBlob), provenanceBlob})
|
|
},
|
|
manifestMaxExtractedSize: 1000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
expectedError: errors.New("expected only a single oci content layer, got 0"),
|
|
},
|
|
{
|
|
name: "non-helm OCI with multiple content layers should still fail",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{imagev1.MediaTypeImageLayerGzip},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
layerBlob1 := createGzippedTarWithContent(t, "file1.yaml", "content1")
|
|
layerBlob2 := createGzippedTarWithContent(t, "file2.yaml", "content2")
|
|
// Using standard image config, not Helm config
|
|
return generateManifest(t, store,
|
|
layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, layerBlob1), layerBlob1},
|
|
layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, layerBlob2), layerBlob2})
|
|
},
|
|
manifestMaxExtractedSize: 1000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
expectedError: errors.New("expected only a single oci content layer, got 2"),
|
|
},
|
|
{
|
|
name: "helm chart with extra content layer should succeed and ignore extra layer",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{"application/vnd.cncf.helm.chart.content.v1.tar+gzip", imagev1.MediaTypeImageLayerGzip},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
chartDir := t.TempDir()
|
|
chartName := "mychart"
|
|
|
|
parent := filepath.Join(chartDir, "parent")
|
|
require.NoError(t, os.Mkdir(parent, 0o755))
|
|
|
|
chartPath := filepath.Join(parent, chartName)
|
|
require.NoError(t, os.Mkdir(chartPath, 0o755))
|
|
|
|
addFileToDirectory(t, chartPath, "Chart.yaml", "chart with extra docker layer")
|
|
|
|
temp, err := os.CreateTemp(t.TempDir(), "")
|
|
require.NoError(t, err)
|
|
defer temp.Close()
|
|
_, err = files.Tgz(parent, nil, nil, temp)
|
|
require.NoError(t, err)
|
|
_, err = temp.Seek(0, io.SeekStart)
|
|
require.NoError(t, err)
|
|
chartBlob, err := io.ReadAll(temp)
|
|
require.NoError(t, err)
|
|
|
|
// Extra OCI layer that Docker/some registries add
|
|
extraLayerBlob := createGzippedTarWithContent(t, "extra.txt", "extra layer content")
|
|
|
|
// Helm chart with proper Helm content layer + extra OCI layer that should be ignored
|
|
return generateManifestWithConfig(t, store, "application/vnd.cncf.helm.config.v1+json",
|
|
layerConf{content.NewDescriptorFromBytes("application/vnd.cncf.helm.chart.content.v1.tar+gzip", chartBlob), chartBlob},
|
|
layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, extraLayerBlob), extraLayerBlob})
|
|
},
|
|
postValidationFunc: func(_, path string, _ Client, _ fields, _ args) {
|
|
// Verify only Helm chart content was extracted, not the extra OCI layer
|
|
chartDir, err := os.ReadDir(path)
|
|
require.NoError(t, err)
|
|
require.Len(t, chartDir, 1)
|
|
require.Equal(t, "Chart.yaml", chartDir[0].Name())
|
|
|
|
chartYaml, err := os.Open(filepath.Join(path, chartDir[0].Name()))
|
|
require.NoError(t, err)
|
|
contents, err := io.ReadAll(chartYaml)
|
|
require.NoError(t, err)
|
|
require.Equal(t, "chart with extra docker layer", string(contents))
|
|
},
|
|
manifestMaxExtractedSize: 10000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
},
|
|
{
|
|
name: "helm chart with extra OCI layer + provenance should extract only helm chart content",
|
|
fields: fields{
|
|
allowedMediaTypes: []string{"application/vnd.cncf.helm.chart.content.v1.tar+gzip", imagev1.MediaTypeImageLayerGzip},
|
|
},
|
|
args: args{
|
|
digestFunc: func(store *memory.Store) string {
|
|
chartDir := t.TempDir()
|
|
chartName := "mychart"
|
|
|
|
parent := filepath.Join(chartDir, "parent")
|
|
require.NoError(t, os.Mkdir(parent, 0o755))
|
|
|
|
chartPath := filepath.Join(parent, chartName)
|
|
require.NoError(t, os.Mkdir(chartPath, 0o755))
|
|
|
|
templatesPath := filepath.Join(chartPath, "templates")
|
|
require.NoError(t, os.Mkdir(templatesPath, 0o755))
|
|
|
|
addFileToDirectory(t, chartPath, "Chart.yaml", "multi-layer helm chart")
|
|
addFileToDirectory(t, templatesPath, "deployment.yaml", "apiVersion: apps/v1")
|
|
|
|
temp, err := os.CreateTemp(t.TempDir(), "")
|
|
require.NoError(t, err)
|
|
defer temp.Close()
|
|
_, err = files.Tgz(parent, nil, nil, temp)
|
|
require.NoError(t, err)
|
|
_, err = temp.Seek(0, io.SeekStart)
|
|
require.NoError(t, err)
|
|
chartBlob, err := io.ReadAll(temp)
|
|
require.NoError(t, err)
|
|
|
|
provenanceBlob := []byte("provenance data")
|
|
extraLayerBlob := createGzippedTarWithContent(t, "extra.txt", "extra oci layer")
|
|
|
|
// Helm chart with: Helm content layer + extra OCI layer + provenance
|
|
// Only the Helm content layer should be extracted
|
|
return generateManifestWithConfig(t, store, "application/vnd.cncf.helm.config.v1+json",
|
|
layerConf{content.NewDescriptorFromBytes("application/vnd.cncf.helm.chart.content.v1.tar+gzip", chartBlob), chartBlob},
|
|
layerConf{content.NewDescriptorFromBytes(imagev1.MediaTypeImageLayerGzip, extraLayerBlob), extraLayerBlob},
|
|
layerConf{content.NewDescriptorFromBytes("application/vnd.cncf.helm.chart.provenance.v1.prov", provenanceBlob), provenanceBlob})
|
|
},
|
|
postValidationFunc: func(_, path string, _ Client, _ fields, _ args) {
|
|
// Verify only Helm chart content was extracted
|
|
entries, err := os.ReadDir(path)
|
|
require.NoError(t, err)
|
|
require.Len(t, entries, 2) // Chart.yaml and templates dir
|
|
|
|
files := make(map[string]bool)
|
|
for _, e := range entries {
|
|
files[e.Name()] = true
|
|
}
|
|
require.True(t, files["Chart.yaml"])
|
|
require.True(t, files["templates"])
|
|
|
|
// Verify Chart.yaml content
|
|
chartYaml, err := os.ReadFile(filepath.Join(path, "Chart.yaml"))
|
|
require.NoError(t, err)
|
|
require.YAMLEq(t, "multi-layer helm chart", string(chartYaml))
|
|
|
|
// Verify templates/deployment.yaml exists
|
|
deploymentYaml, err := os.ReadFile(filepath.Join(path, "templates", "deployment.yaml"))
|
|
require.NoError(t, err)
|
|
require.YAMLEq(t, "apiVersion: apps/v1", string(deploymentYaml))
|
|
|
|
// Ensure extra OCI layer and provenance were not extracted
|
|
require.False(t, files["extra.txt"])
|
|
require.False(t, files["provenance"])
|
|
},
|
|
manifestMaxExtractedSize: 10000,
|
|
disableManifestMaxExtractedSize: false,
|
|
},
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
store := memory.New()
|
|
sha := tt.args.digestFunc(store)
|
|
|
|
c := newClientWithLock(tt.fields.repoURL, globalLock, store, tt.fields.tagsFunc, func(_ context.Context) error {
|
|
return nil
|
|
}, tt.fields.allowedMediaTypes,
|
|
WithImagePaths(cacheDir),
|
|
WithManifestMaxExtractedSize(tt.args.manifestMaxExtractedSize),
|
|
WithDisableManifestMaxExtractedSize(tt.args.disableManifestMaxExtractedSize),
|
|
WithEventHandlers(fakeEventHandlers(t, tt.fields.repoURL)))
|
|
path, gotCloser, err := c.Extract(t.Context(), sha)
|
|
|
|
if tt.expectedError != nil {
|
|
require.EqualError(t, err, tt.expectedError.Error())
|
|
return
|
|
}
|
|
|
|
require.NoError(t, err)
|
|
require.NotEmpty(t, path)
|
|
require.NotNil(t, gotCloser)
|
|
|
|
exists, err := fileExists(path)
|
|
require.True(t, exists)
|
|
require.NoError(t, err)
|
|
|
|
if tt.args.postValidationFunc != nil {
|
|
tt.args.postValidationFunc(sha, path, c, tt.fields, tt.args)
|
|
}
|
|
|
|
require.NoError(t, gotCloser.Close())
|
|
|
|
exists, err = fileExists(path)
|
|
require.False(t, exists)
|
|
require.NoError(t, err)
|
|
})
|
|
}
|
|
}
|
|
|
|
func Test_nativeOCIClient_ResolveRevision(t *testing.T) {
|
|
store := memory.New()
|
|
data := []byte("")
|
|
descriptor := imagev1.Descriptor{
|
|
MediaType: "",
|
|
Digest: digest.FromBytes(data),
|
|
}
|
|
require.NoError(t, store.Push(t.Context(), descriptor, bytes.NewReader(data)))
|
|
require.NoError(t, store.Tag(t.Context(), descriptor, "latest"))
|
|
require.NoError(t, store.Tag(t.Context(), descriptor, "1.2.0"))
|
|
require.NoError(t, store.Tag(t.Context(), descriptor, "v1.2.0"))
|
|
require.NoError(t, store.Tag(t.Context(), descriptor, descriptor.Digest.String()))
|
|
|
|
type fields struct {
|
|
repoURL string
|
|
repo oras.ReadOnlyTarget
|
|
tagsFunc func(context.Context, string) (tags []string, err error)
|
|
allowedMediaTypes []string
|
|
}
|
|
tests := []struct {
|
|
name string
|
|
fields fields
|
|
revision string
|
|
noCache bool
|
|
expectedDigest string
|
|
expectedError error
|
|
}{
|
|
{
|
|
name: "resolve semantic version constraint",
|
|
revision: "^1.0.0",
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{"1.0.0", "1.1.0", "1.2.0", "2.0.0"}, nil
|
|
}},
|
|
expectedDigest: descriptor.Digest.String(),
|
|
},
|
|
{
|
|
name: "resolve exact version",
|
|
revision: "1.2.0",
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{"1.0.0", "1.1.0", "1.2.0", "2.0.0"}, nil
|
|
}},
|
|
expectedDigest: descriptor.Digest.String(),
|
|
},
|
|
{
|
|
name: "resolve digest directly",
|
|
revision: descriptor.Digest.String(),
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{}, errors.New("this should not be invoked")
|
|
}},
|
|
expectedDigest: descriptor.Digest.String(),
|
|
},
|
|
{
|
|
name: "no matching version for constraint",
|
|
revision: "^3.0.0",
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{"1.0.0", "1.1.0", "1.2.0", "2.0.0"}, nil
|
|
}},
|
|
expectedError: errors.New("no version for constraints: version matching constraint not found in 4 tags"),
|
|
},
|
|
{
|
|
name: "error fetching tags",
|
|
revision: "^1.0.0",
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{}, errors.New("some random error")
|
|
}},
|
|
expectedError: errors.New("error fetching tags: failed to get tags: some random error"),
|
|
},
|
|
{
|
|
name: "error resolving digest",
|
|
revision: "sha256:abc123",
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{"1.0.0", "1.1.0", "1.2.0", "2.0.0"}, nil
|
|
}},
|
|
expectedError: errors.New("cannot get digest for revision sha256:abc123: sha256:abc123: not found"),
|
|
},
|
|
{
|
|
name: "resolve latest tag",
|
|
revision: "latest",
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{"1.0.0", "1.1.0", "1.2.0", "2.0.0", "latest"}, nil
|
|
}},
|
|
expectedDigest: descriptor.Digest.String(),
|
|
},
|
|
{
|
|
name: "resolve with complex semver constraint",
|
|
revision: ">=1.0.0 <2.0.0",
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{"0.9.0", "1.0.0", "1.1.0", "1.2.0", "2.0.0", "2.1.0"}, nil
|
|
}},
|
|
expectedDigest: descriptor.Digest.String(),
|
|
},
|
|
{
|
|
name: "resolve with only non-semver tags",
|
|
revision: "^1.0.0",
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{"latest", "stable", "prod", "dev"}, nil
|
|
}},
|
|
expectedError: errors.New("no version for constraints: version matching constraint not found in 4 tags"),
|
|
},
|
|
{
|
|
name: "resolve explicit tag",
|
|
revision: "v1.2.0",
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{}, errors.New("this should not be invoked")
|
|
}},
|
|
expectedError: nil,
|
|
expectedDigest: descriptor.Digest.String(),
|
|
},
|
|
{
|
|
name: "resolve with empty tag list",
|
|
revision: "^1.0.0",
|
|
fields: fields{repo: store, tagsFunc: func(context.Context, string) (tags []string, err error) {
|
|
return []string{}, nil
|
|
}},
|
|
expectedError: errors.New("no version for constraints: version matching constraint not found in 0 tags"),
|
|
},
|
|
}
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
c := newClientWithLock(tt.fields.repoURL, globalLock, tt.fields.repo, tt.fields.tagsFunc, func(_ context.Context) error {
|
|
return nil
|
|
}, tt.fields.allowedMediaTypes, WithEventHandlers(fakeEventHandlers(t, tt.fields.repoURL)))
|
|
got, err := c.ResolveRevision(t.Context(), tt.revision, tt.noCache)
|
|
if tt.expectedError != nil {
|
|
require.EqualError(t, err, tt.expectedError.Error())
|
|
return
|
|
}
|
|
|
|
require.NoError(t, err)
|
|
if got != tt.expectedDigest {
|
|
t.Errorf("ResolveRevision() got = %v, expectedDigest %v", got, tt.expectedDigest)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func Test_nativeOCIClient_DigestMetadata(t *testing.T) {
|
|
// Regression test for https://github.com/argoproj/argo-cd/issues/27521:
|
|
// DigestMetadata must work on any replica without relying on a local tar
|
|
// populated by a prior Extract.
|
|
pushManifest := func(t *testing.T, store *memory.Store, annotations map[string]string) string {
|
|
t.Helper()
|
|
configBlob := []byte("config")
|
|
configDesc := content.NewDescriptorFromBytes("application/vnd.cncf.helm.config.v1+json", configBlob)
|
|
manifestBlob, err := json.Marshal(imagev1.Manifest{
|
|
Versioned: specs.Versioned{SchemaVersion: 2},
|
|
Config: configDesc,
|
|
Annotations: annotations,
|
|
})
|
|
require.NoError(t, err)
|
|
manifestDesc := content.NewDescriptorFromBytes(imagev1.MediaTypeImageManifest, manifestBlob)
|
|
require.NoError(t, store.Push(t.Context(), configDesc, bytes.NewReader(configBlob)))
|
|
require.NoError(t, store.Push(t.Context(), manifestDesc, bytes.NewReader(manifestBlob)))
|
|
require.NoError(t, store.Tag(t.Context(), manifestDesc, manifestDesc.Digest.String()))
|
|
return manifestDesc.Digest.String()
|
|
}
|
|
|
|
t.Run("succeeds without any local tar cache", func(t *testing.T) {
|
|
store := memory.New()
|
|
annotations := map[string]string{
|
|
"org.opencontainers.image.version": "1.2.3",
|
|
"org.opencontainers.image.authors": "[email protected]",
|
|
}
|
|
sha := pushManifest(t, store, annotations)
|
|
|
|
// Note: no WithImagePaths — there is no local cache directory at all,
|
|
// mirroring a repo-server replica that never ran Extract for this digest.
|
|
c := newClientWithLock("repo", globalLock, store, nil, func(_ context.Context) error { return nil }, nil,
|
|
WithEventHandlers(fakeEventHandlers(t, "repo")))
|
|
|
|
manifest, err := c.DigestMetadata(t.Context(), sha)
|
|
require.NoError(t, err)
|
|
require.NotNil(t, manifest)
|
|
assert.Equal(t, annotations, manifest.Annotations)
|
|
})
|
|
|
|
t.Run("propagates not-found errors from the remote", func(t *testing.T) {
|
|
store := memory.New()
|
|
c := newClientWithLock("repo", globalLock, store, nil, func(_ context.Context) error { return nil }, nil,
|
|
WithEventHandlers(fakeEventHandlers(t, "repo")))
|
|
|
|
_, err := c.DigestMetadata(t.Context(), "sha256:0000000000000000000000000000000000000000000000000000000000000000")
|
|
require.Error(t, err)
|
|
})
|
|
}
|
|
|
|
func TestNewClientUsesHTTP2(t *testing.T) {
|
|
t.Parallel()
|
|
t.Run("should negotiate HTTP/2 when TLS is configured", func(t *testing.T) {
|
|
t.Parallel()
|
|
var requestProtos []string
|
|
server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
requestProtos = append(requestProtos, r.Proto)
|
|
t.Logf("called %s with proto %s", r.URL.Path, r.Proto)
|
|
w.WriteHeader(http.StatusOK)
|
|
}))
|
|
// httptest.NewTLSServer only advertises http/1.1 in ALPN, so we must
|
|
// configure the server to also offer h2 for HTTP/2 negotiation to work.
|
|
server.TLS = &tls.Config{NextProtos: []string{"h2", "http/1.1"}}
|
|
server.StartTLS()
|
|
t.Cleanup(server.Close)
|
|
|
|
serverURL, err := url.Parse(server.URL)
|
|
require.NoError(t, err)
|
|
|
|
// NewClient expects oci://host/path format.
|
|
repoURL := "oci://" + serverURL.Host + "/myorg/myrepo"
|
|
client, err := NewClient(repoURL, Creds{InsecureSkipVerify: true}, "", "", nil,
|
|
WithEventHandlers(fakeEventHandlers(t, serverURL.Host+"/myorg/myrepo")))
|
|
require.NoError(t, err)
|
|
|
|
// TestRepo pings the registry's /v2/ endpoint, exercising the transport.
|
|
_, _ = client.TestRepo(t.Context())
|
|
|
|
require.NotEmpty(t, requestProtos, "expected at least one request to the server")
|
|
hasHTTP2 := slices.Contains(requestProtos, "HTTP/2.0")
|
|
assert.True(t, hasHTTP2, "expected at least one HTTP/2 request, but got protocols: %v", requestProtos)
|
|
})
|
|
}
|
|
|
|
func fakeEventHandlers(t *testing.T, repoURL string) EventHandlers {
|
|
t.Helper()
|
|
return EventHandlers{
|
|
OnExtract: func(repo string) func() { return func() { require.Equal(t, repoURL, repo) } },
|
|
OnResolveRevision: func(repo string) func() { return func() { require.Equal(t, repoURL, repo) } },
|
|
OnDigestMetadata: func(repo string) func() { return func() { require.Equal(t, repoURL, repo) } },
|
|
OnTestRepo: func(repo string) func() { return func() { require.Equal(t, repoURL, repo) } },
|
|
OnGetTags: func(repo string) func() { return func() { require.Equal(t, repoURL, repo) } },
|
|
OnGetTagsFail: func(repo string) func() {
|
|
return func() { require.Equal(t, repoURL, repo) }
|
|
},
|
|
OnTestRepoFail: func(repo string) func() {
|
|
return func() { require.Equal(t, repoURL, repo) }
|
|
},
|
|
OnExtractFail: func(repo string) func(revision string) {
|
|
return func(_ string) { require.Equal(t, repoURL, repo) }
|
|
},
|
|
OnResolveRevisionFail: func(repo string) func(revision string) {
|
|
return func(_ string) { require.Equal(t, repoURL, repo) }
|
|
},
|
|
OnDigestMetadataFail: func(repo string) func(revision string) {
|
|
return func(_ string) { require.Equal(t, repoURL, repo) }
|
|
},
|
|
}
|
|
}
|