Q2_K
25 GBMin VRAM: 27 GB
Recommended VRAM: 32 GB
Min RAM: 32 GB
Context: 8K / 128K
Loading model details...
Fetching variants, compatibility details, and metadata.
Loading model details...
Fetching variants, compatibility details, and metadata.
New to local models? Smaller quantization variants are easier to run, while larger ones can improve quality at the cost of more memory.
Q2_K
25 GBMin VRAM: 27 GB
Recommended VRAM: 32 GB
Min RAM: 32 GB
Context: 8K / 128K
Q3_K_M
33 GBMin VRAM: 35 GB
Recommended VRAM: 40 GB
Min RAM: 40 GB
Context: 8K / 128K
Q4_K_M
40 GBMin VRAM: 42 GB
Recommended VRAM: 48 GB
Min RAM: 48 GB
Context: 8K / 128K
Q5_K_M
48 GBMin VRAM: 50 GB
Recommended VRAM: 56 GB
Min RAM: 56 GB
Context: 8K / 128K
Q8_0
74 GBMin VRAM: 76 GB
Recommended VRAM: 80 GB
Min RAM: 80 GB
Context: 8K / 128K
| Quantization | File Size | Min VRAM | Recommended VRAM | Min RAM | Context |
|---|---|---|---|---|---|
| Q2_K | 25 GB | 27 GB | 32 GB | 32 GB | 8K / 128K |
| Q3_K_M | 33 GB | 35 GB | 40 GB | 40 GB | 8K / 128K |
| Q4_K_M | 40 GB | 42 GB | 48 GB | 48 GB | 8K / 128K |
| Q5_K_M | 48 GB | 50 GB | 56 GB | 56 GB | 8K / 128K |
| Q8_0 | 74 GB | 76 GB | 80 GB | 80 GB | 8K / 128K |