Q: What quantization of Mistral Small 24B should I use on a Intel Arc A770 8GB?

For 8 GB VRAM on the Intel Arc A770 8GB, the Q8_0 variant is the best fit. Estimated ~6 tokens/sec on the Q8_0 quantization.

Q: How fast does Mistral Small 24B run on Intel Arc A770 8GB?

Roughly 6 tokens/sec for Q8_0. Real speed depends on context length, backend (Ollama, llama.cpp, LM Studio), and KV cache size.

Q: What if Intel Arc A770 8GB is not enough for Mistral Small 24B?

Consider upgrading to NVIDIA GeForce RTX 5090 (32 GB VRAM) which fits the recommended 32 GB target. Or pick a smaller quantization to stay on your current card.

Question 1

Can I run Mistral Small 24B on a Intel Arc A770 8GB?

Accepted Answer

Sort of — Intel Arc A770 8GB can run Mistral Small 24B (Q8_0) only by spilling layers to RAM. Generation will be slow. CPU + GPU hybrid — not enough VRAM (8 GB < 27 GB min), but 64 GB RAM is sufficient. Expect significantly slower inference.

Question 2

What quantization of Mistral Small 24B should I use on a Intel Arc A770 8GB?

Accepted Answer

For 8 GB VRAM on the Intel Arc A770 8GB, the Q8_0 variant is the best fit. Estimated ~6 tokens/sec on the Q8_0 quantization.

Question 3

How fast does Mistral Small 24B run on Intel Arc A770 8GB?

Accepted Answer

Roughly 6 tokens/sec for Q8_0. Real speed depends on context length, backend (Ollama, llama.cpp, LM Studio), and KV cache size.

Question 4

What if Intel Arc A770 8GB is not enough for Mistral Small 24B?

Accepted Answer

Consider upgrading to NVIDIA GeForce RTX 5090 (32 GB VRAM) which fits the recommended 32 GB target. Or pick a smaller quantization to stay on your current card.

Quantization	File Size	Min VRAM	Rec VRAM	Context	Verdict	Estimated tok/s
Q4_K_M	14 GB	16 GB	20 GB	8K / 32K	Hybrid CPU+GPU	~10
Q8_0Best fit	25 GB	27 GB	32 GB	8K / 32K	Hybrid CPU+GPU	~6

Can I Run Mistral Small 24B on Intel Arc A770 8GB?

Share this matchup

Every Mistral Small 24B quantization on Intel Arc A770 8GB

Upgrade options that fit Mistral Small 24B better