16 KiB
name, category, description, tags, version
| name | category | description | tags | version | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| cloud-kubernetes-deployment | cloud | Deploy and manage applications on Kubernetes with production best practices |
|
1.0.0 |
Kubernetes Deployment
Deploy and manage containerized applications on Kubernetes with production best practices for high availability, security, and scalability.
Hanzo Production Reality (read first)
For Hanzo/Lux services this generic skill is background knowledge only —
deployment is prescribed by HIP-0117 (one binary, three modes) and the
operator. See skills/hanzo-cloud-architecture/SKILL.md.
cloud serve— single process, NO Kubernetes. The whole cloud (IAM, KMS, o11y, tasks, console, ...) in one binary with per-tenant SQLite. This is local dev and single-node/edge production.cloud cluster init— the binary fetches k3s, bootstraps the cluster, installs the operator, and hands reconciliation toservices.hanzo.aiCRs. Devs do NOT hand-write Deployments; the operator reconciles CRs. (Decided per HIP-0117; staged.)helm install cloud ./helm/cloud— BYO Kubernetes, same image.
Rules that override the generic advice below:
- You deploy by editing a
services.hanzo.aiCR (or universe manifests), never bykubectl create deployment. - CI/CD is Gitea Actions on git.hanzo.ai (GH-Actions-compatible workflows on our runners); GitHub is the OSS mirror. Never build images locally.
- Images:
ghcr.io/hanzoai/*(Hanzo) /ghcr.io/luxfi/*(Lux) — never mix. - Secrets come from KMS (KMSSecret CRDs), never plain K8s Secrets in manifests.
- Ingress is
hanzoai/ingress— no nginx, no caddy. - 8 Kubernetes clusters are operated; visor autoscales fleets across providers (BYO provider).
Overview
This skill covers Kubernetes deployment strategies, including:
- Pod and Deployment Management: Creating and managing workloads
- Service Discovery: Exposing and accessing applications
- Rolling Updates: Zero-downtime deployments and rollbacks
- Resource Management: CPU, memory, and storage configuration
- Health Checks: Liveness, readiness, and startup probes
- Configuration: ConfigMaps, Secrets, and environment management
- Networking: Services, Ingress, and NetworkPolicies
- Security: RBAC, Pod Security Standards, security contexts
- Autoscaling: HPA, VPA, and cluster autoscaling
- GitOps: ArgoCD and Flux workflows
- Helm: Package management and templating
When to Use
- Deploying containerized applications to Kubernetes
- Managing application lifecycle in Kubernetes
- Implementing CI/CD for Kubernetes deployments
- Setting up production-ready Kubernetes workloads
- Troubleshooting Kubernetes deployments
- Migrating applications to Kubernetes
Prerequisites
- Docker and containerization basics
- YAML syntax
- Basic networking concepts
- Command-line proficiency
- Access to a Kubernetes cluster (or local testing with kind/minikube)
Key Concepts
Kubernetes Resources
Workloads:
- Pod: Smallest deployable unit (one or more containers)
- Deployment: Manages ReplicaSets for stateless apps
- StatefulSet: Manages stateful workloads with stable identities
- DaemonSet: Runs pods on all (or selected) nodes
- Job/CronJob: Batch processing and scheduled tasks
Services:
- ClusterIP: Internal cluster communication
- NodePort: Expose on node IPs
- LoadBalancer: Cloud provider load balancer
- Ingress: HTTP/HTTPS routing
Configuration:
- ConfigMap: Non-sensitive configuration data
- Secret: Sensitive data (base64 encoded)
- PersistentVolume/PVC: Storage management
Security:
- ServiceAccount: Pod identity
- Role/ClusterRole: RBAC permissions
- NetworkPolicy: Pod network isolation
- PodSecurityPolicy/Standards: Pod security controls
Deployment Strategies
Rolling Update (Default):
- Gradual replacement of old pods with new ones
- Zero downtime
- Configurable with maxSurge and maxUnavailable
Recreate:
- Delete all old pods before creating new ones
- Causes downtime
- Useful when incompatible versions can't coexist
Canary:
- Deploy new version to small subset
- Monitor metrics before full rollout
- Manual traffic splitting
Blue-Green:
- Deploy new version alongside old
- Switch traffic all at once
- Easy rollback
Basic Usage
Create Deployment
# From command line
kubectl create deployment web --image=nginx:1.25 --replicas=3
# From YAML
kubectl apply -f deployment.yaml
deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
spec:
replicas: 3
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
containers:
- name: nginx
image: nginx:1.25
ports:
- containerPort: 80
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 200m
memory: 256Mi
Expose with Service
# Create service
kubectl expose deployment web --port=80 --type=LoadBalancer
# From YAML
kubectl apply -f service.yaml
service.yaml:
apiVersion: v1
kind: Service
metadata:
name: web
spec:
selector:
app: web
ports:
- port: 80
targetPort: 80
type: LoadBalancer
Update Deployment
# Update image
kubectl set image deployment/web nginx=nginx:1.26
# Edit deployment
kubectl edit deployment/web
# Apply changes from file
kubectl apply -f deployment.yaml
# Monitor rollout
kubectl rollout status deployment/web
Rollback Deployment
# View history
kubectl rollout history deployment/web
# Rollback to previous version
kubectl rollout undo deployment/web
# Rollback to specific revision
kubectl rollout undo deployment/web --to-revision=2
Advanced Patterns
Production-Ready Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
labels:
app: api
version: v1
spec:
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
selector:
matchLabels:
app: api
template:
metadata:
labels:
app: api
version: v1
spec:
serviceAccountName: api
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 2000
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- api
topologyKey: kubernetes.io/hostname
containers:
- name: api
image: myapp/api:v1.0.0
ports:
- name: http
containerPort: 8080
env:
- name: DB_HOST
valueFrom:
configMapKeyRef:
name: api-config
key: db.host
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: api-secrets
key: db.password
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 2
memory: 2Gi
startupProbe:
httpGet:
path: /healthz/startup
port: http
periodSeconds: 10
failureThreshold: 30
livenessProbe:
httpGet:
path: /healthz/live
port: http
periodSeconds: 10
failureThreshold: 3
readinessProbe:
httpGet:
path: /healthz/ready
port: http
periodSeconds: 5
failureThreshold: 3
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
volumeMounts:
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir: {}
Horizontal Pod Autoscaler
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: api-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: api
minReplicas: 3
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80
ConfigMap and Secret
# ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
name: api-config
data:
db.host: "postgres.default.svc.cluster.local"
db.port: "5432"
app.yaml: |
server:
port: 8080
timeout: 30s
---
# Secret
# ❌ BAD: Hardcoded credentials - example only, never do this in production
# In production, use sealed-secrets, external-secrets-operator, or your cloud provider's secret management
apiVersion: v1
kind: Secret
metadata:
name: api-secrets
type: Opaque
stringData:
db.password: "changeme123" # Example only - use secret management in production
api.key: "secret-api-key" # Example only - use secret management in production
Common Commands
# Get resources
kubectl get deployments
kubectl get pods
kubectl get services
kubectl get all
# Describe resource
kubectl describe deployment web
kubectl describe pod web-abc123
# View logs
kubectl logs deployment/web
kubectl logs -f pod/web-abc123 # Follow logs
kubectl logs pod/web-abc123 --previous # Previous container
# Execute commands in pod
kubectl exec -it pod/web-abc123 -- /bin/sh
# Port forwarding
kubectl port-forward deployment/web 8080:80
# Scale deployment
kubectl scale deployment/web --replicas=5
# Delete resources
kubectl delete deployment web
kubectl delete -f deployment.yaml
Troubleshooting
Pod Not Starting
# Check pod status
kubectl get pods
kubectl describe pod <pod-name>
# Common issues:
# - ImagePullBackOff: Check image name and registry credentials
# - CrashLoopBackOff: Check logs for application errors
# - Pending: Check resource availability and node constraints
# View events
kubectl get events --sort-by='.lastTimestamp'
# Check logs
kubectl logs <pod-name>
Service Not Accessible
# Check service and endpoints
kubectl get service web
kubectl get endpoints web
# Test DNS
kubectl run -it --rm debug --image=busybox --restart=Never -- nslookup web
# Test connectivity
kubectl run -it --rm debug --image=curlimages/curl --restart=Never -- curl http://web
Deployment Not Rolling Out
# Check rollout status
kubectl rollout status deployment/web
# View deployment events
kubectl describe deployment web
# Check replica sets
kubectl get rs -l app=web
# Pause and resume
kubectl rollout pause deployment/web
kubectl rollout resume deployment/web
Best Practices
Resource Management
- Always set resource requests and limits
- Use appropriate QoS class (Guaranteed for critical workloads)
- Monitor actual usage and adjust accordingly
Health Checks
- Always define readiness probes
- Use startup probes for slow-starting applications
- Keep probe endpoints lightweight
Security
- Run as non-root user
- Use read-only root filesystem
- Drop all capabilities unless needed
- Implement NetworkPolicies
- Use RBAC with least privilege
- Scan images for vulnerabilities
High Availability
- Run multiple replicas (minimum 3 for critical services)
- Use pod anti-affinity to spread across nodes/zones
- Define PodDisruptionBudgets
- Use topology spread constraints
Configuration
- Externalize configuration with ConfigMaps/Secrets
- Use immutable ConfigMaps/Secrets in production
- Never commit secrets to Git
- Version configuration with application
Deployment
- Use specific image tags (never :latest)
- Implement gradual rollouts (canary/blue-green)
- Monitor during rollout
- Have rollback plan
- Test in staging first
Related Skills
containers-docker: Container fundamentalscloud-aws-eks: AWS Kubernetes servicecloud-gcp-gke: Google Kubernetes Engineobservability-prometheus: Metrics collectioncicd-github-actions: CI/CD automationinfrastructure-terraform: Infrastructure as Code
Level 3: Resources
Comprehensive resources for Kubernetes deployment are available in the resources/ directory:
Reference Material
resources/REFERENCE.md (1,800+ lines)
Complete reference covering:
- Kubernetes architecture and fundamentals
- Pod lifecycle and multi-container patterns
- Deployment strategies and rollout management
- Service networking and discovery
- Resource management and QoS
- Health checks configuration
- ConfigMaps and Secrets management
- Persistent storage (PVs, PVCs, StorageClasses)
- Networking (Services, Ingress, NetworkPolicies)
- Security (RBAC, Pod Security Standards, security contexts)
- Autoscaling (HPA, VPA, Cluster Autoscaler)
- GitOps patterns (ArgoCD, Flux)
- Helm charts and package management
- Production best practices
- Troubleshooting guide
- Common pitfalls and anti-patterns
Executable Scripts
resources/scripts/validate_manifests.py
Validate Kubernetes YAML manifests for correctness and best practices:
- YAML syntax validation
- Kubernetes API schema compliance
- Security best practices checks
- Resource configuration validation
- Common misconfiguration detection
- kubectl dry-run validation
- JSON and colored output options
# Validate single file
./validate_manifests.py deployment.yaml
# Validate directory
./validate_manifests.py --strict manifests/
# JSON output
./validate_manifests.py --json deployment.yaml
resources/scripts/analyze_deployment.py
Analyze deployments for issues and optimization opportunities:
- Configuration issue detection
- Security vulnerability scanning
- Resource optimization recommendations
- High availability concerns
- Production readiness assessment
- Live cluster analysis support
# Analyze file
./analyze_deployment.py deployment.yaml
# Analyze live cluster
./analyze_deployment.py --cluster --namespace production
# JSON output
./analyze_deployment.py --json --cluster
resources/scripts/test_deployment.sh
Test deployments in local Kubernetes cluster:
- Creates test cluster (kind/minikube)
- Deploys manifests
- Validates rollout
- Runs health checks
- Executes smoke tests
- Automatic cleanup
# Test with kind
./test_deployment.sh deployment.yaml
# Test with minikube
./test_deployment.sh --provider minikube manifests/
# Keep cluster for debugging
./test_deployment.sh --no-cleanup deployment.yaml
Examples
resources/examples/manifests/basic-deployment/
Simple deployment example:
- Basic Deployment with nginx
- ClusterIP Service
- Minimal configuration for learning
resources/examples/manifests/production-deployment/
Production-ready deployment:
- Full Deployment with all best practices
- LoadBalancer Service with annotations
- ConfigMap and Secret management
- HorizontalPodAutoscaler
- PodDisruptionBudget
- Security contexts and RBAC
- Multi-container patterns
- Health checks and lifecycle hooks
resources/examples/helm/sample-chart/
Complete Helm chart structure:
- Chart.yaml with metadata and dependencies
- values.yaml with sensible defaults
- Templates for all resources
- Helper templates (_helpers.tpl)
- Configurable security and resources
resources/examples/kustomize/
Kustomize overlays for environments:
base/: Base manifestsoverlays/dev/: Development environment customizationsoverlays/prod/: Production environment customizations- Demonstrates patching and configuration management
resources/examples/cicd/github-actions-deploy.yml
Complete GitHub Actions workflow:
- Build and push Docker images
- Validate manifests
- Deploy to staging (on PR)
- Deploy to production (on main)
- Rollback on failure
- Smoke tests
Learning Path
- Start: Basic deployment and service creation
- Intermediate: Add health checks, resource limits, ConfigMaps
- Advanced: Implement autoscaling, GitOps, production patterns
- Expert: Multi-cluster, service mesh, advanced networking