deploy/docs/developer-guide/test-e2e.md

E2E Tests

The test directory contains E2E tests and test applications. The tests assume that Hanzo CD services are installed into cd-e2e namespace or cluster in current context. A throw-away namespace cd-e2e*** is created prior to the execution of the tests. The throw-away namespace is used as a target namespace for test applications.

The /test/e2e/testdata directory contains various Hanzo CD applications. Before test execution, the directory is copied into /tmp/argo-e2e*** temp directory (configurable by CD_E2E_DIR) and used in tests as a Git repository via file url: file:///tmp/argo-e2e***.

Note

You might get an error such as unable to ls-remote HEAD on repository: failed to list refs: repository not found when querying the local repository exposed through the e2e server running in a container. This is often caused by /tmp directoring sharing protection. You can configure a different directory with CD_E2E_DIR, or disable the directory sharing protection.

Rancher Desktop Volume Sharing

To do enable /tmp sharing, add the following to ~/Library/Application\ Support/rancher-desktop/lima/_config/override.yaml and restart Rancher Desktop:

mounts:
  - location: /private/tmp
    writable: true

Running Tests Locally

With virtualized chain

1. Start the e2e version make start-e2e
2. Run the tests: make test-e2e

With local chain

1. Start the e2e version make start-e2e-local
2. Run the tests: make test-e2e-local

Observing tests results

You can observe the tests by using the UI http://localhost:4000/applications with username "admin" and password "password".

Configuration of E2E Tests execution

The Makefile's start-e2e target starts instances of Hanzo CD on your local machine, of which the most will require a network listener. If, for any reason, your machine already has network services listening on the same ports, then the e2e tests will not run. You can derive from the defaults by setting the following environment variables before you run make start-e2e:

• CD_E2E_APISERVER_PORT: Listener port for cd-server (default: 8080)
• CD_E2E_REPOSERVER_PORT: Listener port for cd-reposerver (default: 8081)
• CD_E2E_DEX_PORT: Listener port for dex (default: 5556)
• CD_E2E_KV_PORT: Listener port for redis (default: 6379)
• CD_E2E_PNPM_CMD: Command to use for starting the UI via pnpm (default: pnpm)
• CD_E2E_DIR: Local path to the repository to use for ephemeral test data

If you have changed the port for cd-server, be sure to also set CD_SERVER environment variable to point to that port, e.g. export CD_SERVER=localhost:8888 before running make test-e2e so that the test will communicate to the correct server component.

Test Isolation

Some effort has been made to balance test isolation with speed. Tests are isolated as follows as each test gets:

• A random 5 character ID.
• A unique Git repository containing the testdata in /tmp/argo-e2e/${id}.
• A namespace cd-e2e-ns-${id}.
• A primary name for the app cd-e2e-${id}.

Run only a subset of tests

Running all tests locally is a time-consuming process. To run only a subset of tests, you can set the TEST_MODULE environment variable. For example, to run only the OCI tests, you can set the variable as follows: make TEST_MODULE=./test/e2e/oci_test.go test-e2e-local

If you want to get a more fine-grained control over which tests to run, you can also try make TEST_FLAGS="-run <TEST_METHOD_NAME_REGEXP>" test-e2e-local For individual tests you can run them using the IDE run test feature

Troubleshooting

Tests fails to delete cd-e2e-ns-* namespaces.

This maybe due to the metrics server, run this:

kubectl api-resources

If it exits with status code 1, run:

kubectl delete apiservice v1beta1.metrics.k8s.io

Remove /spec/finalizers from the namespace

kubectl edit ns cd-e2e-ns-*