Edge LLM inference,
measured on real hardware.
Real benchmark data from Apple Silicon — tokens/sec, energy efficiency, memory bandwidth. Every byte stays on-device. No cloud. No logs. No surveillance.
peak speed
39.7
tokens / second
first token
255ms
time to first byte
efficiency
2.6
tokens / joule
memory used
1.1GB
of 8 GB
models tested
2
open-weight models
data leaves device
0
privacy guaranteed
Model benchmarks
Mac14,2 Apple M2 · macOS 15.5 · click column headers to sort
Each model ran 1 warmup passes then 5 timed inference passes on a fixed 35-token prompt. Energy estimated from powermetrics.
Performance charts
TOKENS / SECOND · M2
MEMORY BANDWIDTH OVER TIME
EFFICIENCY · TOK / JOULE BY CHIP
Zero data egress
Every inference runs entirely on-chip. Prompts, completions, and intermediate activations never touch a network.
Unified memory advantage
Apple's shared CPU/GPU memory pool eliminates costly data transfers, cutting latency and energy use versus discrete GPU setups.
Neural Engine acceleration
The M3 ANE handles quantized model operations at up to 18 TOPS, offloading the CPU and reducing thermal throttling.
Live inference demo
cloud baseline via groq api · same models as the benchmark · latency comparison shown
// try a prompt
CLOUD VS LOCAL
METHODOLOGY