Meta
Llama 3.3 70B Instruct
70.6 B params · 128000 context · Open weights
Released 2024-11-26
Details
API pricing
No hosted API price verified. Open-weight models can be self-hosted.
Running it locally
Memory needed for 70.554 B parameters at 8,192 tokens of context. Architecture not recorded, so a flat 1.0 GB KV allowance is used. Estimates, not measurements.
| Format | Weights | KV cache | Overhead | Total |
|---|---|---|---|---|
| Q4 K_M | 43.1 GB | 1.0 GB | 2.7 GB | 46.8 GB |
| Q6 K | 57.9 GB | 1.0 GB | 3.4 GB | 62.3 GB |
| Q8 0 | 75.0 GB | 1.0 GB | 4.2 GB | 80.2 GB |
| FP16 | 141.1 GB | 1.0 GB | 7.6 GB | 149.7 GB |
Catalogued hardware that fits at Q4
- GMKtec EVO-X3 (Ryzen AI Max+ 395)comfortable
- Beelink GTR9 Pro (Ryzen AI Max+ 395)comfortable
- OneXPlayer OneXStation i1comfortable
- Beelink GTi15 Ultra (Core Ultra 9 285H)comfortable
- Minisforum MS-02 Ultracomfortable
- Beelink SEi14 (Core Ultra 9 185H)comfortable
- Framework Desktopcomfortable
- Apple Mac mini (M5 Pro)comfortable
Mini PCs reserve 8.0 GB for the operating system. “Comfortable” keeps at least 10% memory spare.
Benchmarks
No benchmarks for this model yet
We only publish results we can evidence, so there is nothing here yet and we have not filled the gap with estimates. Here is how we will test and label results.