Files
skills/public/cryptography-certificate-management.md

20 KiB

name, description
name description
cryptography-certificate-management Certificate lifecycle management including rotation, renewal, monitoring, automation with Let's Encrypt and ACME

Certificate Management

Scope: Certificate rotation, renewal, monitoring, Let's Encrypt, ACME protocol, automation Lines: ~350 Last Updated: 2025-10-27 Format Version: 1.0 (Atomic)


When to Use This Skill

Activate this skill when:

  • Managing certificate expiration
  • Automating certificate renewal
  • Implementing Let's Encrypt
  • Monitoring certificate health
  • Rotating certificates
  • Setting up ACME clients
  • Handling certificate revocation
  • Planning certificate infrastructure

Certificate Lifecycle

1. Generation → Create private key and CSR
2. Issuance  → CA signs certificate
3. Deployment → Install on servers
4. Monitoring → Track expiration
5. Renewal   → Get new certificate before expiry
6. Rotation  → Replace old with new
7. Revocation → Invalidate if compromised (optional)

Let's Encrypt & ACME

Overview

Let's Encrypt:

  • Free, automated certificate authority
  • 90-day certificate lifetime
  • ACME protocol for automation
  • Rate limits apply

ACME Protocol:

1. Account creation
2. Domain validation (HTTP-01, DNS-01, TLS-ALPN-01)
3. Certificate request
4. Certificate issuance
5. Certificate renewal

Certbot

Installation:

# Ubuntu/Debian
sudo apt install certbot python3-certbot-nginx

# macOS
brew install certbot

HTTP-01 Challenge (webroot):

# Obtain certificate
sudo certbot certonly --webroot \
    -w /var/www/html \
    -d example.com \
    -d www.example.com

# Certificates stored in:
# /etc/letsencrypt/live/example.com/fullchain.pem
# /etc/letsencrypt/live/example.com/privkey.pem

Nginx plugin (automatic):

# Obtain and install certificate
sudo certbot --nginx -d example.com -d www.example.com

# Certbot automatically:
# - Validates domain
# - Obtains certificate
# - Updates nginx config
# - Reloads nginx

DNS-01 Challenge (wildcards):

# For wildcard certificates
sudo certbot certonly --manual \
    --preferred-challenges dns \
    -d '*.example.com' \
    -d example.com

# Add TXT record to DNS:
# _acme-challenge.example.com → [provided value]

Auto-renewal:

# Test renewal
sudo certbot renew --dry-run

# Setup cron (or systemd timer)
0 0,12 * * * certbot renew --quiet --post-hook "systemctl reload nginx"

acme.sh

Alternative ACME client:

# Install
⚠️ **SECURITY**: Piping curl to shell is dangerous. For production:
```bash
# Download script first
curl -O https://get.acme.sh
# Verify checksum
sha256sum get.acme.sh
# Review content
less get.acme.sh
# Then execute
bash get.acme.sh

For development/learning only:

curl https://get.acme.sh | sh

# Issue certificate (HTTP validation)
acme.sh --issue -d example.com -w /var/www/html

# Issue wildcard (DNS validation with Cloudflare)
export CF_Token="your-cloudflare-api-token"  # Placeholder - replace with actual API token from Cloudflare dashboard
acme.sh --issue --dns dns_cf -d example.com -d '*.example.com'

# Install certificate
acme.sh --install-cert -d example.com \
    --key-file /etc/ssl/private/example.com.key \
    --fullchain-file /etc/ssl/certs/example.com.crt \
    --reloadcmd "systemctl reload nginx"

# Auto-renewal (automatic after install)
acme.sh --cron

Automation Patterns

Pattern 1: Systemd Timer for Renewal

Service file (/etc/systemd/system/certbot-renew.service):

[Unit]
Description=Certbot Renewal

[Service]
Type=oneshot
ExecStart=/usr/bin/certbot renew --quiet --deploy-hook "systemctl reload nginx"

Timer file (/etc/systemd/system/certbot-renew.timer):

[Unit]
Description=Certbot Renewal Timer

[Timer]
OnCalendar=daily
RandomizedDelaySec=1h
Persistent=true

[Install]
WantedBy=timers.target

Enable:

sudo systemctl enable --now certbot-renew.timer
sudo systemctl list-timers certbot-renew.timer

Pattern 2: Kubernetes cert-manager

Install cert-manager:

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml

ClusterIssuer (Let's Encrypt):

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx

Certificate resource:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-com-tls
  namespace: default
spec:
  secretName: example-com-tls
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
  - example.com
  - www.example.com

Ingress annotation (automatic):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - example.com
    secretName: example-com-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: example
            port:
              number: 80

Pattern 3: Custom Automation Script

Python renewal script:

#!/usr/bin/env python3
import subprocess
import ssl
import socket
from datetime import datetime, timedelta
import smtplib
from email.message import EmailMessage

def check_certificate_expiry(hostname, port=443):
    """Check days until certificate expires"""
    context = ssl.create_default_context()
    with socket.create_connection((hostname, port)) as sock:
        with context.wrap_socket(sock, server_hostname=hostname) as ssock:
            cert = ssock.getpeercert()
            not_after = datetime.strptime(cert['notAfter'], '%b %d %H:%M:%S %Y %Z')
            days_remaining = (not_after - datetime.now()).days
            return days_remaining

def renew_certificate(domain):
    """Renew certificate using certbot"""
    result = subprocess.run(
        ['certbot', 'renew', '--cert-name', domain, '--quiet'],
        capture_output=True,
        text=True
    )
    return result.returncode == 0

def send_alert(subject, body):
    """Send email alert"""
    msg = EmailMessage()
    msg['Subject'] = subject
    msg['From'] = '[email protected]'
    msg['To'] = '[email protected]'
    msg.set_content(body)

    with smtplib.SMTP('localhost') as s:
        s.send_message(msg)

def main():
    domains = ['example.com', 'api.example.com']

    for domain in domains:
        try:
            days = check_certificate_expiry(domain)
            print(f"{domain}: {days} days until expiry")

            # Renew if < 30 days remaining
            if days < 30:
                print(f"Renewing {domain}...")
                if renew_certificate(domain):
                    send_alert(
                        f"Certificate renewed: {domain}",
                        f"Successfully renewed certificate for {domain}"
                    )
                else:
                    send_alert(
                        f"Certificate renewal FAILED: {domain}",
                        f"Failed to renew certificate for {domain}"
                    )

            # Alert if < 14 days and renewal failed
            elif days < 14:
                send_alert(
                    f"Certificate expiring soon: {domain}",
                    f"Certificate for {domain} expires in {days} days"
                )

        except Exception as e:
            send_alert(
                f"Certificate check failed: {domain}",
                f"Error checking {domain}: {str(e)}"
            )

if __name__ == '__main__':
    main()

Monitoring

Check Expiration

OpenSSL:

# Check expiration date
echo | openssl s_client -connect example.com:443 2>/dev/null | \
    openssl x509 -noout -dates

# Days until expiry
echo | openssl s_client -connect example.com:443 2>/dev/null | \
    openssl x509 -noout -enddate | \
    awk -F= '{print $2}' | xargs -I {} date -d {} +%s | \
    awk -v now=$(date +%s) '{print int(($1-now)/86400)}'

Prometheus Exporter:

# ssl_exporter
apiVersion: v1
kind: Service
metadata:
  name: ssl-exporter
spec:
  ports:
  - port: 9219
  selector:
    app: ssl-exporter
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ssl-exporter
spec:
  template:
    spec:
      containers:
      - name: ssl-exporter
        image: ribbybibby/ssl-exporter:latest
        args:
        - --config.file=/config/ssl-exporter.yaml

Prometheus alert:

groups:
- name: certificates
  rules:
  - alert: CertificateExpiringSoon
    expr: ssl_cert_not_after - time() < 86400 * 30
    labels:
      severity: warning
    annotations:
      summary: "Certificate expiring in < 30 days"
      description: "Certificate for {{ $labels.instance }} expires in {{ $value | humanizeDuration }}"

Monitoring Services

SSL Labs API:

import requests
import time

def check_ssl_labs(domain):
    """Check SSL configuration with SSL Labs"""
    api_url = "https://api.ssllabs.com/api/v3/analyze"

    # Start scan
    requests.get(api_url, params={'host': domain, 'startNew': 'on'})

    # Poll for results
    while True:
        resp = requests.get(api_url, params={'host': domain}).json()
        if resp['status'] == 'READY':
            grade = resp['endpoints'][0]['grade']
            return grade
        time.sleep(30)

grade = check_ssl_labs('example.com')
print(f"SSL Labs grade: {grade}")

Certificate Rotation

Zero-Downtime Rotation

Strategy:

1. Obtain new certificate
2. Deploy new certificate alongside old
3. Update configuration to use new
4. Reload/restart services
5. Verify new certificate active
6. Remove old certificate

Nginx example:

# 1. Obtain new certificate
certbot certonly --webroot -w /var/www/html -d example.com

# 2. Test configuration
nginx -t

# 3. Reload nginx (no downtime)
systemctl reload nginx

# 4. Verify
echo | openssl s_client -connect example.com:443 2>/dev/null | \
    openssl x509 -noout -dates

Multi-Server Rotation

Ansible playbook:

---
- name: Rotate certificates
  hosts: webservers
  tasks:
    - name: Copy new certificate
      copy:
        src: "{{ item }}"
        dest: "/etc/ssl/certs/"
        mode: 0644
      loop:
        - example.com.crt
        - example.com-chain.crt

    - name: Copy private key
      copy:
        src: example.com.key
        dest: /etc/ssl/private/
        mode: 0600

    - name: Reload nginx
      systemd:
        name: nginx
        state: reloaded

    - name: Verify certificate
      shell: |
        echo | openssl s_client -connect localhost:443 2>/dev/null | \
        openssl x509 -noout -dates
      register: cert_check
      changed_when: false

    - name: Display certificate info
      debug:
        var: cert_check.stdout

Best Practices

1. Renewal Window

# ✅ Good: Renew 30 days before expiry
if [ $days_remaining -lt 30 ]; then
    renew_certificate
fi

# ❌ Bad: Wait until last minute
if [ $days_remaining -lt 3 ]; then
    renew_certificate
fi

2. Monitoring

# ✅ Good: Multiple monitoring approaches
- Prometheus metrics
- Periodic checks
- Alert 30 days before expiry

# ❌ Bad: No monitoring
# (certificate expires, site goes down)

3. Backup Certificates

# ✅ Good: Backup before rotation
cp /etc/ssl/certs/example.com.crt /backup/
certbot renew

# ❌ Bad: No backup
certbot renew
# (if renewal fails, no fallback)

Troubleshooting

Issue 1: Rate Limits

Let's Encrypt limits:

- 50 certificates per domain per week
- 5 duplicate certificates per week
- 300 new orders per account per 3 hours

Solution: Use staging environment for testing

certbot --staging -d example.com

Issue 2: Renewal Failure

Check renewal status:

certbot certificates

Common issues:

  • Webroot path wrong
  • DNS not propagated (DNS-01)
  • Firewall blocking (HTTP-01)
  • Rate limit exceeded

Debug:

certbot renew --dry-run --debug

  • cryptography-pki-fundamentals - Certificate basics
  • cryptography-tls-configuration - TLS setup
  • infrastructure-automation - Deployment automation
  • observability-metrics-monitoring - Monitoring setup

Level 3: Resources

Location: ~/.hanzo/skills/cryptography/certificate-management/resources/

This skill includes comprehensive Level 3 resources for production certificate management implementations:

REFERENCE.md (~3,100 lines)

Comprehensive technical reference covering:

  • Fundamentals: Digital certificates, X.509 standard, trust chains, PKI concepts
  • Certificate Components: Subject, SANs, extensions, key usage, certificate formats (PEM, DER, PKCS#12)
  • Certificate Lifecycle: Generation, CSR creation, issuance, deployment, monitoring, renewal, rotation, revocation
  • CA Hierarchies: Root/intermediate CAs, public vs private CAs, trust models
  • ACME Protocol: Challenge types (HTTP-01, DNS-01, TLS-ALPN-01), ACME flow, client implementations
  • Let's Encrypt: Rate limits, staging environment, trust chain, wildcard certificates
  • Certificate Automation: Certbot usage, acme.sh, renewal hooks, systemd timers
  • Kubernetes cert-manager: Installation, issuers, certificate resources, ingress annotations
  • Mutual TLS (mTLS): Server/client configuration, service mesh integration, zero-trust patterns
  • Certificate Revocation: CRL, OCSP, OCSP stapling, must-staple extension
  • Certificate Transparency: CT logs, monitoring, unauthorized certificate detection
  • Certificate Pinning: Public key pinning, HPKP (deprecated), mobile implementations
  • Monitoring: Prometheus exporters, expiration tracking, alerting strategies
  • Best Practices: Lifetime management, key protection, automation, testing, backup/recovery
  • Common Issues: Troubleshooting expired certs, chain issues, key mismatches, SAN errors
  • Security: Private key protection, validation requirements, weak cryptography avoidance
  • Compliance: PCI-DSS, HIPAA, SOC2, GDPR certificate requirements
  • Tools: OpenSSL, certbot, acme.sh, step CLI, cfssl, cert-manager, SSL Labs, testssl.sh

Scripts (3 production-ready tools)

validate_certificates.py (580+ lines) - Comprehensive certificate validation

  • Validates certificates from remote hosts or local files
  • Checks expiration dates with configurable thresholds
  • Detects weak algorithms (MD5, SHA-1, weak RSA keys)
  • Validates certificate chains and hostname matching
  • SAN validation including wildcard matching
  • OCSP/CRL revocation checking support
  • Compliance validation (PCI-DSS, HIPAA, SOC2)
  • Batch validation from hosts file
  • JSON output for CI/CD integration
  • Detailed reporting with severity levels
  • Usage: ./validate_certificates.py --host example.com --compliance PCI-DSS --json

renew_certificates.py (450+ lines) - Automated ACME certificate renewal

  • ACME protocol automation (Let's Encrypt, ZeroSSL)
  • HTTP-01 and DNS-01 challenge support
  • Automatic renewal based on expiration thresholds
  • Pre/post/deploy hook support
  • Automatic backup before renewal
  • Rollback on renewal failure
  • Batch renewal of all expiring certificates
  • Dry-run mode for testing
  • Staging environment support
  • Custom ACME provider support
  • DNS provider integration (Cloudflare, Route53, etc.)
  • Usage: ./renew_certificates.py --domain example.com --dns-provider cloudflare --json

monitor_cert_expiry.sh (550+ lines) - Certificate expiration monitoring

  • Monitors certificates from remote hosts or local files
  • Configurable warning/critical thresholds
  • Multiple output formats (text, JSON, Prometheus, CSV)
  • Batch monitoring from hosts file
  • Directory scanning for certificate files
  • Email/webhook/Slack alerting
  • Certificate inventory export
  • OCSP revocation status checking
  • Prometheus metrics format for monitoring integration
  • Color-coded terminal output
  • Exit codes for CI/CD integration
  • Usage: ./monitor_cert_expiry.sh --hosts-file hosts.txt --format prometheus --alert-email [email protected]

Examples (6 production-ready implementations)

python/acme_automation.py - ACME protocol automation from scratch

  • Account registration and key management
  • Certificate ordering with ACME protocol
  • HTTP-01 challenge implementation
  • DNS-01 challenge for wildcard certificates
  • CSR generation with multiple SANs
  • Complete ACME workflow demonstration
  • Staging environment support
  • Production-ready patterns

python/mtls_server_client.py - Mutual TLS implementation

  • mTLS server with client certificate verification
  • mTLS client with certificate authentication
  • Certificate generation helper
  • TLS version and cipher suite configuration
  • Client certificate extraction and validation
  • Zero-trust authentication pattern
  • Service-to-service authentication example

python/prometheus_cert_exporter.py - Prometheus metrics exporter

  • Exports certificate expiration metrics
  • Multi-host monitoring support
  • Prometheus-compatible metrics format
  • Certificate metadata collection
  • Expiration time tracking
  • Validity status monitoring
  • Error counting and duration tracking
  • Health check endpoint
  • Production-ready monitoring integration

kubernetes/cert-manager-setup.yaml - Comprehensive cert-manager configuration

  • ClusterIssuers for Let's Encrypt (production and staging)
  • DNS-01 solvers (Cloudflare, Route53)
  • Certificate resources with SANs
  • Wildcard certificate examples
  • Ingress annotation automation
  • Namespace-scoped issuers
  • Private CA issuer configuration
  • mTLS client certificates
  • Certificate lifecycle management
  • Renewal configuration
  • Troubleshooting helpers

bash/certbot-automation.sh - Complete certbot automation suite

  • HTTP-01 webroot validation
  • Nginx plugin integration
  • Wildcard certificates (manual and automated DNS)
  • DNS provider integration (Cloudflare, Route53)
  • Systemd timer setup for auto-renewal
  • Pre/post/deploy hooks
  • Certificate revocation
  • Force renewal
  • Staging environment testing
  • Custom CSR usage
  • Certificate monitoring
  • Nginx installation automation
  • 18+ production examples

config/nginx-ssl-best-practices.conf - Production Nginx SSL configuration

  • Modern TLS 1.2/1.3 configuration
  • Strong cipher suite selection
  • OCSP stapling setup
  • HSTS and security headers
  • Let's Encrypt integration
  • HTTP to HTTPS redirect
  • Wildcard certificate support
  • mTLS configuration
  • Multiple domain examples
  • SSL Labs A+ rating configuration
  • Certificate renewal integration
  • PHP-FPM and reverse proxy examples

Quick Start

# Validate remote certificate
cd ~/.hanzo/skills/cryptography/certificate-management/resources/scripts
./validate_certificates.py --host example.com --check-revocation

# Renew certificate automatically
./renew_certificates.py --domain example.com --webroot /var/www/html --post-hook "systemctl reload nginx"

# Monitor certificate expiration
./monitor_cert_expiry.sh --hosts-file hosts.txt --threshold-warning 30 --json

# Export Prometheus metrics
./monitor_cert_expiry.sh --hosts-file hosts.txt --format prometheus > /var/lib/prometheus/cert_metrics.prom

# Run Python examples
cd ../examples/python
pip install cryptography acme josepy prometheus_client
python acme_automation.py http01
python mtls_server_client.py setup && python mtls_server_client.py server
python prometheus_cert_exporter.py --hosts example.com:443

# View comprehensive reference
cd ../
less REFERENCE.md

Integration Notes

CI/CD Integration:

# .github/workflows/certificate-check.yml
- name: Validate Certificates
  run: |
    ./scripts/validate_certificates.py \
      --batch-file production-hosts.txt \
      --compliance PCI-DSS \
      --json \
      --check-revocation

Monitoring Setup:

# Prometheus scrape config
scrape_configs:
  - job_name: 'ssl-certificates'
    static_configs:
      - targets: ['localhost:9117']
    scrape_interval: 60s

Alerting:

# Prometheus alert rules
- alert: CertificateExpiringSoon
  expr: ssl_certificate_expiry_seconds < 86400 * 30
  labels:
    severity: warning
  annotations:
    summary: "Certificate expiring in < 30 days"

Kubernetes Deployment:

# Install cert-manager
kubectl apply -f examples/kubernetes/cert-manager-setup.yaml

# Check certificate status
kubectl get certificates -A
kubectl describe certificate example-com-tls

Last Updated: 2025-10-27