bot/docs/providers/groq.md

title, summary, read_when

title	summary	read_when
Groq	Groq setup (auth + model selection)	You want to use Groq with Hanzo Bot You need the API key env var or CLI auth choice

Groq

Groq provides ultra-fast inference on open-source models (Llama, Gemma, Mistral, and more) using custom LPU hardware. Hanzo Bot connects to Groq through its OpenAI-compatible API.

• Provider: groq
• Auth: GROQ_API_KEY
• API: OpenAI-compatible

Quick start

1. Get an API key from console.groq.com/keys.

2. Set the API key:

export GROQ_API_KEY="gsk_..."

3. Set a default model:

{
  agents: {
    defaults: {
      model: { primary: "groq/llama-3.3-70b-versatile" },
    },
  },
}

Config file example

{
  env: { GROQ_API_KEY: "gsk_..." },
  agents: {
    defaults: {
      model: { primary: "groq/llama-3.3-70b-versatile" },
    },
  },
}

Audio transcription

Groq also provides fast Whisper-based audio transcription. When configured as a media-understanding provider, Hanzo Bot uses Groq's whisper-large-v3-turbo model to transcribe voice messages.

{
  media: {
    understanding: {
      audio: {
        models: [{ provider: "groq" }],
      },
    },
  },
}

Environment note

If the Gateway runs as a daemon (launchd/systemd), make sure GROQ_API_KEY is available to that process (for example, in ~/.hanzo/bot/.env or via env.shellEnv).

Available models

Groq's model catalog changes frequently. Run hanzo-bot models list | grep groq to see currently available models, or check console.groq.com/docs/models.

Popular choices include:

• Llama 3.3 70B Versatile - general-purpose, large context
• Llama 3.1 8B Instant - fast, lightweight
• Gemma 2 9B - compact, efficient
• Mixtral 8x7B - MoE architecture, strong reasoning

Links

• Groq Console
• API Documentation
• Model List
• Pricing