5.7 KiB
Testing the HealthKit Sync Skill
This guide explains how to test the healthkit-sync Agent Skill in Hanzo Bot.
Prerequisites
- Hanzo Bot CLI installed (
bot --version) - Hanzo Bot gateway configured (
~/.bot/bot.json) - Active messaging provider (WhatsApp, Telegram, or Discord)
Step 1: Link the Skill
Hanzo Bot loads skills from ~/.bot/skills/. Create a symlink to keep the skill in sync with the repo:
# Check if already linked
ls -la ~/.bot/skills/healthkit-sync
# Create symlink (if not exists)
ln -sf "$(pwd)/skills/healthkit-sync" ~/.bot/skills/healthkit-sync
# Verify
ls -la ~/.bot/skills/healthkit-sync/
Expected output:
SKILL.md
references/
TESTING.md
Step 2: Validate Skill Structure
Verify the skill follows the Agent Skills specification:
# Check frontmatter
head -15 ~/.bot/skills/healthkit-sync/SKILL.md
Expected frontmatter:
---
name: healthkit-sync
description: iOS HealthKit data sync CLI commands and patterns...
license: Apache-2.0
compatibility: macOS with healthsync CLI installed
metadata:
category: development
platforms: ios,macos
author: mneves
---
Validation checklist:
namematches folder name (healthkit-sync)nameis lowercase with hyphens onlydescriptionis under 1024 characters- SKILL.md is under 500 lines
# Check line count
wc -l ~/.bot/skills/healthkit-sync/SKILL.md
# Should be < 500
Step 3: Start a New Session
Important: Skills are snapshotted when a session starts. Changes require a new session.
Option A: Restart Gateway
# Stop existing gateway (Ctrl+C or kill)
pkill -f "bot gateway"
# Start fresh
bot gateway --port 18789
Option B: New Chat Session
Start a new conversation in your messaging app:
- WhatsApp: Send a message from an allowed number
- Telegram: Start fresh chat with your bot
- Discord: New channel or DM
Step 4: Test Skill Activation
Send test messages that should trigger the skill. The skill activates when the agent detects relevant keywords in your message.
Test Cases
| # | Message | Expected Response |
|---|---|---|
| 1 | "How do I sync health data from my iPhone to my Mac?" | CLI pairing flow (healthsync discover, healthsync scan) |
| 2 | "What commands does healthsync support?" | Full CLI reference with all commands |
| 3 | "How do I fetch my step count?" | healthsync fetch command with --types steps |
| 4 | "Explain the security architecture of iOS Health Sync" | mTLS, certificate pinning, Keychain storage details |
| 5 | "What's the project structure of ai-health-sync?" | Architecture overview with file paths |
| 6 | "How does certificate pinning work in healthsync?" | TOFU model, SHA256 fingerprint validation |
| 7 | "Troubleshoot: no devices found" | Troubleshooting steps (WiFi, mDNS, firewall) |
Verification
A successful skill activation should include:
- Specific CLI commands (not generic advice)
- Correct paths (
~/.healthsync/config.json) - Accurate security details (SwiftData for iOS tokens, Keychain for macOS)
- References to project files when relevant
Step 5: Test Progressive Disclosure
The skill uses progressive disclosure - references load on-demand:
- Initial query: Only SKILL.md content loads (~100-150 lines)
- Detailed query: Reference files load as needed
Test this by asking increasingly specific questions:
Message 1: "What is healthsync?"
→ Basic overview from SKILL.md
Message 2: "Show me the detailed CLI reference"
→ Loads references/CLI-REFERENCE.md
Message 3: "Explain the certificate pinning implementation"
→ Loads references/SECURITY.md
Message 4: "What's the full project architecture?"
→ Loads references/ARCHITECTURE.md
Step 6: Debug Issues
Skill Not Loading
# Check Hanzo Bot recognizes the skills directory
cat ~/.bot/bot.json | jq '.skills'
# Verify skill is readable
cat ~/.bot/skills/healthkit-sync/SKILL.md | head -20
# Check for YAML parsing errors (must be valid YAML frontmatter)
Skill Not Activating
The skill activates based on keyword matching in the description field. Ensure your message contains relevant terms:
- "healthsync", "health sync"
- "HealthKit", "health data"
- "iPhone", "iOS", "macOS"
- "pairing", "sync", "fetch"
- "certificate pinning", "mTLS"
Wrong Information Returned
If the agent returns incorrect info, the skill may not be loading. Verify by asking:
"What is the config path for healthsync CLI?"
Correct answer: ~/.healthsync/config.json
If wrong: Skill not loaded, check steps 1-3.
Step 7: Monitor Logs
# Watch Hanzo Bot logs in real-time
tail -f ~/.bot/logs/*.log
# Filter for skill-related messages
tail -f ~/.bot/logs/*.log | grep -iE "(skill|healthkit)"
Troubleshooting
| Issue | Cause | Solution |
|---|---|---|
| Skill not found | Symlink broken | Re-run ln -sf command |
| Old content returned | Session cached | Start new session |
| YAML parse error | Invalid frontmatter | Check for tabs (use spaces) |
| References not loading | Wrong path syntax | Use [text](path) not [See: file] |
| Skill ignored | Description mismatch | Add more keywords to description |
Updating the Skill
After making changes to the skill files:
-
Git pull (if changes from repo):
git pull origin master -
Symlink auto-updates (if linked correctly)
-
Start new session (required for changes to take effect)