20 KiB
name, description
| name | description |
|---|---|
| cryptography-certificate-management | Certificate lifecycle management including rotation, renewal, monitoring, automation with Let's Encrypt and ACME |
Certificate Management
Scope: Certificate rotation, renewal, monitoring, Let's Encrypt, ACME protocol, automation Lines: ~350 Last Updated: 2025-10-27 Format Version: 1.0 (Atomic)
When to Use This Skill
Activate this skill when:
- Managing certificate expiration
- Automating certificate renewal
- Implementing Let's Encrypt
- Monitoring certificate health
- Rotating certificates
- Setting up ACME clients
- Handling certificate revocation
- Planning certificate infrastructure
Certificate Lifecycle
1. Generation → Create private key and CSR
2. Issuance → CA signs certificate
3. Deployment → Install on servers
4. Monitoring → Track expiration
5. Renewal → Get new certificate before expiry
6. Rotation → Replace old with new
7. Revocation → Invalidate if compromised (optional)
Let's Encrypt & ACME
Overview
Let's Encrypt:
- Free, automated certificate authority
- 90-day certificate lifetime
- ACME protocol for automation
- Rate limits apply
ACME Protocol:
1. Account creation
2. Domain validation (HTTP-01, DNS-01, TLS-ALPN-01)
3. Certificate request
4. Certificate issuance
5. Certificate renewal
Certbot
Installation:
# Ubuntu/Debian
sudo apt install certbot python3-certbot-nginx
# macOS
brew install certbot
HTTP-01 Challenge (webroot):
# Obtain certificate
sudo certbot certonly --webroot \
-w /var/www/html \
-d example.com \
-d www.example.com
# Certificates stored in:
# /etc/letsencrypt/live/example.com/fullchain.pem
# /etc/letsencrypt/live/example.com/privkey.pem
Nginx plugin (automatic):
# Obtain and install certificate
sudo certbot --nginx -d example.com -d www.example.com
# Certbot automatically:
# - Validates domain
# - Obtains certificate
# - Updates nginx config
# - Reloads nginx
DNS-01 Challenge (wildcards):
# For wildcard certificates
sudo certbot certonly --manual \
--preferred-challenges dns \
-d '*.example.com' \
-d example.com
# Add TXT record to DNS:
# _acme-challenge.example.com → [provided value]
Auto-renewal:
# Test renewal
sudo certbot renew --dry-run
# Setup cron (or systemd timer)
0 0,12 * * * certbot renew --quiet --post-hook "systemctl reload nginx"
acme.sh
Alternative ACME client:
# Install
⚠️ **SECURITY**: Piping curl to shell is dangerous. For production:
```bash
# Download script first
curl -O https://get.acme.sh
# Verify checksum
sha256sum get.acme.sh
# Review content
less get.acme.sh
# Then execute
bash get.acme.sh
For development/learning only:
curl https://get.acme.sh | sh
# Issue certificate (HTTP validation)
acme.sh --issue -d example.com -w /var/www/html
# Issue wildcard (DNS validation with Cloudflare)
export CF_Token="your-cloudflare-api-token" # Placeholder - replace with actual API token from Cloudflare dashboard
acme.sh --issue --dns dns_cf -d example.com -d '*.example.com'
# Install certificate
acme.sh --install-cert -d example.com \
--key-file /etc/ssl/private/example.com.key \
--fullchain-file /etc/ssl/certs/example.com.crt \
--reloadcmd "systemctl reload nginx"
# Auto-renewal (automatic after install)
acme.sh --cron
Automation Patterns
Pattern 1: Systemd Timer for Renewal
Service file (/etc/systemd/system/certbot-renew.service):
[Unit]
Description=Certbot Renewal
[Service]
Type=oneshot
ExecStart=/usr/bin/certbot renew --quiet --deploy-hook "systemctl reload nginx"
Timer file (/etc/systemd/system/certbot-renew.timer):
[Unit]
Description=Certbot Renewal Timer
[Timer]
OnCalendar=daily
RandomizedDelaySec=1h
Persistent=true
[Install]
WantedBy=timers.target
Enable:
sudo systemctl enable --now certbot-renew.timer
sudo systemctl list-timers certbot-renew.timer
Pattern 2: Kubernetes cert-manager
Install cert-manager:
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml
ClusterIssuer (Let's Encrypt):
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: [email protected]
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: nginx
Certificate resource:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: example-com-tls
namespace: default
spec:
secretName: example-com-tls
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
dnsNames:
- example.com
- www.example.com
Ingress annotation (automatic):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- example.com
secretName: example-com-tls
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: example
port:
number: 80
Pattern 3: Custom Automation Script
Python renewal script:
#!/usr/bin/env python3
import subprocess
import ssl
import socket
from datetime import datetime, timedelta
import smtplib
from email.message import EmailMessage
def check_certificate_expiry(hostname, port=443):
"""Check days until certificate expires"""
context = ssl.create_default_context()
with socket.create_connection((hostname, port)) as sock:
with context.wrap_socket(sock, server_hostname=hostname) as ssock:
cert = ssock.getpeercert()
not_after = datetime.strptime(cert['notAfter'], '%b %d %H:%M:%S %Y %Z')
days_remaining = (not_after - datetime.now()).days
return days_remaining
def renew_certificate(domain):
"""Renew certificate using certbot"""
result = subprocess.run(
['certbot', 'renew', '--cert-name', domain, '--quiet'],
capture_output=True,
text=True
)
return result.returncode == 0
def send_alert(subject, body):
"""Send email alert"""
msg = EmailMessage()
msg['Subject'] = subject
msg['From'] = '[email protected]'
msg['To'] = '[email protected]'
msg.set_content(body)
with smtplib.SMTP('localhost') as s:
s.send_message(msg)
def main():
domains = ['example.com', 'api.example.com']
for domain in domains:
try:
days = check_certificate_expiry(domain)
print(f"{domain}: {days} days until expiry")
# Renew if < 30 days remaining
if days < 30:
print(f"Renewing {domain}...")
if renew_certificate(domain):
send_alert(
f"Certificate renewed: {domain}",
f"Successfully renewed certificate for {domain}"
)
else:
send_alert(
f"Certificate renewal FAILED: {domain}",
f"Failed to renew certificate for {domain}"
)
# Alert if < 14 days and renewal failed
elif days < 14:
send_alert(
f"Certificate expiring soon: {domain}",
f"Certificate for {domain} expires in {days} days"
)
except Exception as e:
send_alert(
f"Certificate check failed: {domain}",
f"Error checking {domain}: {str(e)}"
)
if __name__ == '__main__':
main()
Monitoring
Check Expiration
OpenSSL:
# Check expiration date
echo | openssl s_client -connect example.com:443 2>/dev/null | \
openssl x509 -noout -dates
# Days until expiry
echo | openssl s_client -connect example.com:443 2>/dev/null | \
openssl x509 -noout -enddate | \
awk -F= '{print $2}' | xargs -I {} date -d {} +%s | \
awk -v now=$(date +%s) '{print int(($1-now)/86400)}'
Prometheus Exporter:
# ssl_exporter
apiVersion: v1
kind: Service
metadata:
name: ssl-exporter
spec:
ports:
- port: 9219
selector:
app: ssl-exporter
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ssl-exporter
spec:
template:
spec:
containers:
- name: ssl-exporter
image: ribbybibby/ssl-exporter:latest
args:
- --config.file=/config/ssl-exporter.yaml
Prometheus alert:
groups:
- name: certificates
rules:
- alert: CertificateExpiringSoon
expr: ssl_cert_not_after - time() < 86400 * 30
labels:
severity: warning
annotations:
summary: "Certificate expiring in < 30 days"
description: "Certificate for {{ $labels.instance }} expires in {{ $value | humanizeDuration }}"
Monitoring Services
SSL Labs API:
import requests
import time
def check_ssl_labs(domain):
"""Check SSL configuration with SSL Labs"""
api_url = "https://api.ssllabs.com/api/v3/analyze"
# Start scan
requests.get(api_url, params={'host': domain, 'startNew': 'on'})
# Poll for results
while True:
resp = requests.get(api_url, params={'host': domain}).json()
if resp['status'] == 'READY':
grade = resp['endpoints'][0]['grade']
return grade
time.sleep(30)
grade = check_ssl_labs('example.com')
print(f"SSL Labs grade: {grade}")
Certificate Rotation
Zero-Downtime Rotation
Strategy:
1. Obtain new certificate
2. Deploy new certificate alongside old
3. Update configuration to use new
4. Reload/restart services
5. Verify new certificate active
6. Remove old certificate
Nginx example:
# 1. Obtain new certificate
certbot certonly --webroot -w /var/www/html -d example.com
# 2. Test configuration
nginx -t
# 3. Reload nginx (no downtime)
systemctl reload nginx
# 4. Verify
echo | openssl s_client -connect example.com:443 2>/dev/null | \
openssl x509 -noout -dates
Multi-Server Rotation
Ansible playbook:
---
- name: Rotate certificates
hosts: webservers
tasks:
- name: Copy new certificate
copy:
src: "{{ item }}"
dest: "/etc/ssl/certs/"
mode: 0644
loop:
- example.com.crt
- example.com-chain.crt
- name: Copy private key
copy:
src: example.com.key
dest: /etc/ssl/private/
mode: 0600
- name: Reload nginx
systemd:
name: nginx
state: reloaded
- name: Verify certificate
shell: |
echo | openssl s_client -connect localhost:443 2>/dev/null | \
openssl x509 -noout -dates
register: cert_check
changed_when: false
- name: Display certificate info
debug:
var: cert_check.stdout
Best Practices
1. Renewal Window
# ✅ Good: Renew 30 days before expiry
if [ $days_remaining -lt 30 ]; then
renew_certificate
fi
# ❌ Bad: Wait until last minute
if [ $days_remaining -lt 3 ]; then
renew_certificate
fi
2. Monitoring
# ✅ Good: Multiple monitoring approaches
- Prometheus metrics
- Periodic checks
- Alert 30 days before expiry
# ❌ Bad: No monitoring
# (certificate expires, site goes down)
3. Backup Certificates
# ✅ Good: Backup before rotation
cp /etc/ssl/certs/example.com.crt /backup/
certbot renew
# ❌ Bad: No backup
certbot renew
# (if renewal fails, no fallback)
Troubleshooting
Issue 1: Rate Limits
Let's Encrypt limits:
- 50 certificates per domain per week
- 5 duplicate certificates per week
- 300 new orders per account per 3 hours
Solution: Use staging environment for testing
certbot --staging -d example.com
Issue 2: Renewal Failure
Check renewal status:
certbot certificates
Common issues:
- Webroot path wrong
- DNS not propagated (DNS-01)
- Firewall blocking (HTTP-01)
- Rate limit exceeded
Debug:
certbot renew --dry-run --debug
Related Skills
cryptography-pki-fundamentals- Certificate basicscryptography-tls-configuration- TLS setupinfrastructure-automation- Deployment automationobservability-metrics-monitoring- Monitoring setup
Level 3: Resources
Location: ~/.hanzo/skills/cryptography/certificate-management/resources/
This skill includes comprehensive Level 3 resources for production certificate management implementations:
REFERENCE.md (~3,100 lines)
Comprehensive technical reference covering:
- Fundamentals: Digital certificates, X.509 standard, trust chains, PKI concepts
- Certificate Components: Subject, SANs, extensions, key usage, certificate formats (PEM, DER, PKCS#12)
- Certificate Lifecycle: Generation, CSR creation, issuance, deployment, monitoring, renewal, rotation, revocation
- CA Hierarchies: Root/intermediate CAs, public vs private CAs, trust models
- ACME Protocol: Challenge types (HTTP-01, DNS-01, TLS-ALPN-01), ACME flow, client implementations
- Let's Encrypt: Rate limits, staging environment, trust chain, wildcard certificates
- Certificate Automation: Certbot usage, acme.sh, renewal hooks, systemd timers
- Kubernetes cert-manager: Installation, issuers, certificate resources, ingress annotations
- Mutual TLS (mTLS): Server/client configuration, service mesh integration, zero-trust patterns
- Certificate Revocation: CRL, OCSP, OCSP stapling, must-staple extension
- Certificate Transparency: CT logs, monitoring, unauthorized certificate detection
- Certificate Pinning: Public key pinning, HPKP (deprecated), mobile implementations
- Monitoring: Prometheus exporters, expiration tracking, alerting strategies
- Best Practices: Lifetime management, key protection, automation, testing, backup/recovery
- Common Issues: Troubleshooting expired certs, chain issues, key mismatches, SAN errors
- Security: Private key protection, validation requirements, weak cryptography avoidance
- Compliance: PCI-DSS, HIPAA, SOC2, GDPR certificate requirements
- Tools: OpenSSL, certbot, acme.sh, step CLI, cfssl, cert-manager, SSL Labs, testssl.sh
Scripts (3 production-ready tools)
validate_certificates.py (580+ lines) - Comprehensive certificate validation
- Validates certificates from remote hosts or local files
- Checks expiration dates with configurable thresholds
- Detects weak algorithms (MD5, SHA-1, weak RSA keys)
- Validates certificate chains and hostname matching
- SAN validation including wildcard matching
- OCSP/CRL revocation checking support
- Compliance validation (PCI-DSS, HIPAA, SOC2)
- Batch validation from hosts file
- JSON output for CI/CD integration
- Detailed reporting with severity levels
- Usage:
./validate_certificates.py --host example.com --compliance PCI-DSS --json
renew_certificates.py (450+ lines) - Automated ACME certificate renewal
- ACME protocol automation (Let's Encrypt, ZeroSSL)
- HTTP-01 and DNS-01 challenge support
- Automatic renewal based on expiration thresholds
- Pre/post/deploy hook support
- Automatic backup before renewal
- Rollback on renewal failure
- Batch renewal of all expiring certificates
- Dry-run mode for testing
- Staging environment support
- Custom ACME provider support
- DNS provider integration (Cloudflare, Route53, etc.)
- Usage:
./renew_certificates.py --domain example.com --dns-provider cloudflare --json
monitor_cert_expiry.sh (550+ lines) - Certificate expiration monitoring
- Monitors certificates from remote hosts or local files
- Configurable warning/critical thresholds
- Multiple output formats (text, JSON, Prometheus, CSV)
- Batch monitoring from hosts file
- Directory scanning for certificate files
- Email/webhook/Slack alerting
- Certificate inventory export
- OCSP revocation status checking
- Prometheus metrics format for monitoring integration
- Color-coded terminal output
- Exit codes for CI/CD integration
- Usage:
./monitor_cert_expiry.sh --hosts-file hosts.txt --format prometheus --alert-email [email protected]
Examples (6 production-ready implementations)
python/acme_automation.py - ACME protocol automation from scratch
- Account registration and key management
- Certificate ordering with ACME protocol
- HTTP-01 challenge implementation
- DNS-01 challenge for wildcard certificates
- CSR generation with multiple SANs
- Complete ACME workflow demonstration
- Staging environment support
- Production-ready patterns
python/mtls_server_client.py - Mutual TLS implementation
- mTLS server with client certificate verification
- mTLS client with certificate authentication
- Certificate generation helper
- TLS version and cipher suite configuration
- Client certificate extraction and validation
- Zero-trust authentication pattern
- Service-to-service authentication example
python/prometheus_cert_exporter.py - Prometheus metrics exporter
- Exports certificate expiration metrics
- Multi-host monitoring support
- Prometheus-compatible metrics format
- Certificate metadata collection
- Expiration time tracking
- Validity status monitoring
- Error counting and duration tracking
- Health check endpoint
- Production-ready monitoring integration
kubernetes/cert-manager-setup.yaml - Comprehensive cert-manager configuration
- ClusterIssuers for Let's Encrypt (production and staging)
- DNS-01 solvers (Cloudflare, Route53)
- Certificate resources with SANs
- Wildcard certificate examples
- Ingress annotation automation
- Namespace-scoped issuers
- Private CA issuer configuration
- mTLS client certificates
- Certificate lifecycle management
- Renewal configuration
- Troubleshooting helpers
bash/certbot-automation.sh - Complete certbot automation suite
- HTTP-01 webroot validation
- Nginx plugin integration
- Wildcard certificates (manual and automated DNS)
- DNS provider integration (Cloudflare, Route53)
- Systemd timer setup for auto-renewal
- Pre/post/deploy hooks
- Certificate revocation
- Force renewal
- Staging environment testing
- Custom CSR usage
- Certificate monitoring
- Nginx installation automation
- 18+ production examples
config/nginx-ssl-best-practices.conf - Production Nginx SSL configuration
- Modern TLS 1.2/1.3 configuration
- Strong cipher suite selection
- OCSP stapling setup
- HSTS and security headers
- Let's Encrypt integration
- HTTP to HTTPS redirect
- Wildcard certificate support
- mTLS configuration
- Multiple domain examples
- SSL Labs A+ rating configuration
- Certificate renewal integration
- PHP-FPM and reverse proxy examples
Quick Start
# Validate remote certificate
cd ~/.hanzo/skills/cryptography/certificate-management/resources/scripts
./validate_certificates.py --host example.com --check-revocation
# Renew certificate automatically
./renew_certificates.py --domain example.com --webroot /var/www/html --post-hook "systemctl reload nginx"
# Monitor certificate expiration
./monitor_cert_expiry.sh --hosts-file hosts.txt --threshold-warning 30 --json
# Export Prometheus metrics
./monitor_cert_expiry.sh --hosts-file hosts.txt --format prometheus > /var/lib/prometheus/cert_metrics.prom
# Run Python examples
cd ../examples/python
pip install cryptography acme josepy prometheus_client
python acme_automation.py http01
python mtls_server_client.py setup && python mtls_server_client.py server
python prometheus_cert_exporter.py --hosts example.com:443
# View comprehensive reference
cd ../
less REFERENCE.md
Integration Notes
CI/CD Integration:
# .github/workflows/certificate-check.yml
- name: Validate Certificates
run: |
./scripts/validate_certificates.py \
--batch-file production-hosts.txt \
--compliance PCI-DSS \
--json \
--check-revocation
Monitoring Setup:
# Prometheus scrape config
scrape_configs:
- job_name: 'ssl-certificates'
static_configs:
- targets: ['localhost:9117']
scrape_interval: 60s
Alerting:
# Prometheus alert rules
- alert: CertificateExpiringSoon
expr: ssl_certificate_expiry_seconds < 86400 * 30
labels:
severity: warning
annotations:
summary: "Certificate expiring in < 30 days"
Kubernetes Deployment:
# Install cert-manager
kubectl apply -f examples/kubernetes/cert-manager-setup.yaml
# Check certificate status
kubectl get certificates -A
kubectl describe certificate example-com-tls
Last Updated: 2025-10-27