Skip to main content
Cerebras uses custom wafer-scale processors to achieve inference speeds that far exceed GPU-based providers. Llama 3.1 70B runs at over 2000 tokens/second - roughly 20x faster than typical cloud GPU inference.

Supported Models

Setup

1

Get API access

Sign up at inference.cerebras.ai. Currently in limited access.
2

Set the environment variable

3

Verify

Environment Variables

string
required
Your Cerebras API key.

Configuration Example

Model Aliases

Usage Examples

Notes

  • API endpoint: https://api.cerebras.ai/v1 (OpenAI-compatible)
  • Status: Experimental - hardware-specific availability, may have capacity constraints.
  • Best use case: real-time streaming, bulk generation tasks, low-latency chat.
  • Cerebras does not support vision or image inputs.