Q: What quantization of Aya Expanse 32B should I use on a NVIDIA GeForce GTX 1660?

For 6 GB VRAM on the NVIDIA GeForce GTX 1660, the Q4_K_M variant is the best fit. Estimated ~3 tokens/sec on the Q4_K_M quantization.

Q: How fast does Aya Expanse 32B run on NVIDIA GeForce GTX 1660?

Roughly 3 tokens/sec for Q4_K_M. Real speed depends on context length, backend (Ollama, llama.cpp, LM Studio), and KV cache size.

Q: What if NVIDIA GeForce GTX 1660 is not enough for Aya Expanse 32B?

Consider upgrading to NVIDIA GeForce RTX 4090 (24 GB VRAM) which fits the recommended 24 GB target. Or pick a smaller quantization to stay on your current card.

Question 1

Can I run Aya Expanse 32B on a NVIDIA GeForce GTX 1660?

Accepted Answer

Sort of — NVIDIA GeForce GTX 1660 can run Aya Expanse 32B (Q4_K_M) only by spilling layers to RAM. Generation will be slow. CPU + GPU hybrid — not enough VRAM (6 GB < 21 GB min), but 64 GB RAM is sufficient. Expect significantly slower inference.

Question 2

What quantization of Aya Expanse 32B should I use on a NVIDIA GeForce GTX 1660?

Accepted Answer

For 6 GB VRAM on the NVIDIA GeForce GTX 1660, the Q4_K_M variant is the best fit. Estimated ~3 tokens/sec on the Q4_K_M quantization.

Question 3

How fast does Aya Expanse 32B run on NVIDIA GeForce GTX 1660?

Accepted Answer

Roughly 3 tokens/sec for Q4_K_M. Real speed depends on context length, backend (Ollama, llama.cpp, LM Studio), and KV cache size.

Question 4

What if NVIDIA GeForce GTX 1660 is not enough for Aya Expanse 32B?

Accepted Answer

Consider upgrading to NVIDIA GeForce RTX 4090 (24 GB VRAM) which fits the recommended 24 GB target. Or pick a smaller quantization to stay on your current card.

Can I Run Aya Expanse 32B on NVIDIA GeForce GTX 1660?

Share this matchup

Every Aya Expanse 32B quantization on NVIDIA GeForce GTX 1660

Upgrade options that fit Aya Expanse 32B better