Provider: Groq | Respan Docs

This page is for Respan LLM Gateway users.

Use Respan Gateway to call Groq models (llama-3.3-70b-versatile, llama-3.1-8b-instant, mixtral-8x7b-32768, and the rest) while keeping unified observability (logs, cost, latency, reliability) in Respan.

Quick setup

Get a Respan API key

Sign up and create a key on the API keys page.

Add credits (recommended)

Top up credits to pay through Respan. No Groq key required, Respan handles provider auth and billing.

Prefer to route through your own Groq account? See Use your own Groq key.

Send your first request

Pick the integration that matches your stack. The base URL is https://api.respan.ai/api and the only key needed is your RESPAN_API_KEY.

Groq SDK

OpenAI SDK

Respan API

Point the official Groq SDK at the Respan gateway by overriding base_url.

More integrations

Groq models work with every Respan gateway integration:

• OpenAI SDK
• OpenAI Agents SDK
• LangChain
• LlamaIndex
• Pydantic AI
• Respan native (OTel)

Switch models

Change the model parameter to call any supported model through the same client. Use the groq/ prefix to disambiguate when routing across providers. Browse the full list on the Models page.

Use your own Groq key (BYOK)

Credits are the default path. If you’d rather bill Groq directly, attach your own provider key.

Open Providers

Go to the Providers page.

Add Groq

Select Groq and paste your groq.api_key.

Load balancing (Optional)

Add multiple credential sets and use Load balancing weight to distribute traffic across them.

Override credentials per model (Optional)

Use credential_override when one model on a request should use a different Groq key than the default.

Log without proxying (Optional)

Already calling Groq directly? Send logs to Respan asynchronously to track cost, latency, and performance for those external calls.

See the logging guide for the full setup.