Q: What quantization of Yi 1.5 9B should I use on a NVIDIA GeForce GTX 1660 Super?

For 6 GB VRAM on the NVIDIA GeForce GTX 1660 Super, the Q4_K_M variant is the best fit. Estimated ~17 tokens/sec on the Q4_K_M quantization.

Q: How fast does Yi 1.5 9B run on NVIDIA GeForce GTX 1660 Super?

Roughly 17 tokens/sec for Q4_K_M. Real speed depends on context length, backend (Ollama, llama.cpp, LM Studio), and KV cache size.

Q: What if NVIDIA GeForce GTX 1660 Super is not enough for Yi 1.5 9B?

Consider upgrading to NVIDIA GeForce RTX 3080 10GB (10 GB VRAM) which fits the recommended 10 GB target. Or pick a smaller quantization to stay on your current card.

Question 1

Can I run Yi 1.5 9B on a NVIDIA GeForce GTX 1660 Super?

Accepted Answer

Sort of — NVIDIA GeForce GTX 1660 Super can run Yi 1.5 9B (Q4_K_M) only by spilling layers to RAM. Generation will be slow. CPU + GPU hybrid — not enough VRAM (6 GB < 6.5 GB min), but 64 GB RAM is sufficient. Expect significantly slower inference.

Question 2

What quantization of Yi 1.5 9B should I use on a NVIDIA GeForce GTX 1660 Super?

Accepted Answer

For 6 GB VRAM on the NVIDIA GeForce GTX 1660 Super, the Q4_K_M variant is the best fit. Estimated ~17 tokens/sec on the Q4_K_M quantization.

Question 3

How fast does Yi 1.5 9B run on NVIDIA GeForce GTX 1660 Super?

Accepted Answer

Roughly 17 tokens/sec for Q4_K_M. Real speed depends on context length, backend (Ollama, llama.cpp, LM Studio), and KV cache size.

Question 4

What if NVIDIA GeForce GTX 1660 Super is not enough for Yi 1.5 9B?

Accepted Answer

Consider upgrading to NVIDIA GeForce RTX 3080 10GB (10 GB VRAM) which fits the recommended 10 GB target. Or pick a smaller quantization to stay on your current card.

Can I Run Yi 1.5 9B on NVIDIA GeForce GTX 1660 Super?

Share this matchup

Every Yi 1.5 9B quantization on NVIDIA GeForce GTX 1660 Super

Upgrade options that fit Yi 1.5 9B better