Hardware profile

Current generation

NVIDIA GeForce RTX 5090 · 32 GB

32 GB VRAM · 1792 GB/s · verified 2026-11-02

Platform Windows

Available for models 32 GB

Generation Current generation

Release date 2025-01

Backends Ollama, llama.cpp, CUDA

Memory bandwidth 1792 GB/s

Last verified 2026-11-02

The first 32 GB consumer-class target in this catalog. The extra capacity is most useful for larger quantized models or longer contexts.

Recommended quantization by model

This table chooses the highest-quality published quantization that still clears the selected 8K context estimate. It does not compare output quality or claim that every backend has the same kernels.

Model	Recommended quantization	Estimated requirement	Context ceiling	Result
Flint 40B	UD-Q3_K_XL	20.2 GB at 8K	262,144 tokens	Comfortable fit
NVIDIA Nemotron 3.5 Bolt E4B	Q4_K_M	8.0 GB at 8K	65,536 tokens	Comfortable fit
LFM3-1.7B	Q8_0	3.4 GB at 8K	262,144 tokens	Comfortable fit
Bonsai 2	PQ2_0	9.2 GB at 8K	262,144 tokens	Comfortable fit
Qwen3.8 27B	Q4_K_M	19.4 GB at 8K	200,001 tokens	Comfortable fit
Gemma 4 12B Unified	Q8_0	14.7 GB at 8K	262,144 tokens	Comfortable fit
Gemma 4 E4B	Q8_0	11.0 GB at 8K	131,072 tokens	Comfortable fit
gpt-oss-20b	MXFP4	13.8 GB at 8K	131,072 tokens	Comfortable fit
Qwen3.6 35B-A3B	Q4_K_M	24.7 GB at 8K	262,144 tokens	Comfortable fit
Gemma 4 31B	Q4_K_M	22.2 GB at 8K	76,562 tokens	Comfortable fit
Qwen3 8B	Q8_0	11.6 GB at 8K	32,768 tokens	Comfortable fit
DeepSeek R1 8B	Q8_0	11.1 GB at 8K	131,072 tokens	Comfortable fit
Llama 3.2 3B	Q8_0	5.8 GB at 8K	131,072 tokens	Comfortable fit
Phi-4 Mini	Q8_0	6.7 GB at 8K	131,072 tokens	Comfortable fit
Mistral 7B	Q8_0	10.3 GB at 8K	32,768 tokens	Comfortable fit
Gemma 3 4B	Q8_0	7.6 GB at 8K	131,072 tokens	Comfortable fit
gpt-oss-120b	MXFP4	65.2 GB at 8K	0 tokens	Won't fit

Per-quantization fit for recent models

These are the five most relevant non-server models plus the next current model in the catalog. Each row uses the same 8K formula, so changing a quantization or context length produces an arithmetic result rather than a new benchmark claim.

Model	Quantization	Weight files	8K requirement	Result	8K context ceiling	Speed band	Ollama
Flint 40B	UD-IQ1_M	11.35 GB	13.1 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	No verified tag
Flint 40B	UD-IQ2_XXS	11.86 GB	13.6 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	No verified tag
Flint 40B	UD-Q2_K_XL	13.56 GB	15.3 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	No verified tag
Flint 40B	UD-IQ3_XXS	14.54 GB	16.3 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	No verified tag
Flint 40B	UD-Q3_K_XL	18.49 GB	20.2 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	No verified tag
Flint 40B	NVFP4	22.86 GB	24.6 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	`ollama run flint:40b`
Flint 40B	UD-Q4_K_XL	25.05 GB	26.8 GB	Tight fit	189,902 tokens	Estimate · Very fast relative band	No verified tag
Flint 40B	UD-Q5_K_XL	29.92 GB	31.7 GB	Tight fit	20,244 tokens	Estimate · Very fast relative band	No verified tag
Flint 40B	UD-Q6_K_XL	36.32 GB	38.1 GB	Won't fit	0 tokens	Estimate · Very fast relative band	No verified tag
Flint 40B	Q8_0	42.30 GB	44.0 GB	Won't fit	0 tokens	Estimate · Very fast relative band	No verified tag
NVIDIA Nemotron 3.5 Bolt E4B	Q4_K_M	6.18 GB	8.0 GB	Comfortable fit	65,536 tokens	Estimate · Very fast relative band	`ollama run nemotron-3.5-bolt:e4b`
LFM3-1.7B	QAD-Q4_0	1.05 GB	2.7 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	`ollama run lfm3:1.7b-qad-q4_0`
LFM3-1.7B	Q4_K_M	1.10 GB	2.7 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	`ollama run lfm3:1.7b-q4_K_M`
LFM3-1.7B	Q5_K_M	1.26 GB	2.9 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	No verified tag
LFM3-1.7B	Q6_K	1.43 GB	3.0 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	No verified tag
LFM3-1.7B	Q8_0	1.85 GB	3.4 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	`ollama run lfm3:1.7b-q8_0`
Bonsai 2	PTQ1_0	5.95 GB	8.0 GB	Comfortable fit	262,144 tokens	Estimate · Fast relative band	No verified tag
Bonsai 2	PQ2_0	7.21 GB	9.2 GB	Comfortable fit	262,144 tokens	Estimate · Fast relative band	No verified tag
Bonsai 2	Q2_0	7.63 GB	9.7 GB	Comfortable fit	262,144 tokens	Estimate · Fast relative band	`ollama run bonsai-2:27b-q2_0`
Qwen3.8 27B	Q4_K_M	17.39 GB	19.4 GB	Comfortable fit	200,001 tokens	Estimate · Fast relative band	`ollama run qwen3.8:27b-q4_K_M`
Qwen3.8 27B	Q8_0	29.98 GB	32.0 GB	Won't fit	8,010 tokens	Estimate · Fast relative band	`ollama run qwen3.8:27b-q8_0`
Gemma 4 12B Unified	Q4_0	7.15 GB	9.1 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	`ollama run gemma4:12b-it-qat`
Gemma 4 12B Unified	Q4_K_M	7.30 GB	9.2 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	`ollama run gemma4:12b-it-q4_K_M`
Gemma 4 12B Unified	Q8_0	12.85 GB	14.7 GB	Comfortable fit	262,144 tokens	Estimate · Very fast relative band	`ollama run gemma4:12b-it-q8_0`

Speed bands are relative heuristics, not measured tokens per second. Verify the package name, license, backend support, and free memory before relying on a result.

Why this memory result is not a guarantee

NVIDIA GeForce RTX 5090 · 32 GB has 32 GB of VRAM. The formula subtracts weight files, KV cache, and 1.5 GB of runtime overhead, then requires 20% remaining headroom for a comfortable result. Driver reservations, display buffers, other applications, longer prompts, concurrent requests, and backend-specific workspace can all reduce the usable budget.

Low-cost setup upgrades

Useful accessories around NVIDIA GeForce RTX 5090 · 32 GB

These optional setup upgrades address storage, cooling, or power resilience. None increases GPU or unified-memory capacity.

External NVMe SSD for model storage

Keeps large GGUF and MLX files off the boot drive. Fast storage reduces load time, but it does not increase the memory available for inference.

Search current listings on Amazon

Desktop RAM upgrade kit

Useful for MoE CPU offload and keeping the operating system out of VRAM. Match the motherboard's memory type, slot layout, capacity limit, and validated part numbers.

Search current listings on Amazon

GPU support bracket

A lower-cost mechanical upgrade for heavy three-slot cards. Check clearance, mounting height, and whether the bracket blocks fans or case panels.

Search current listings on Amazon

As an Amazon Associate I earn from qualifying purchases. Links are sponsored search links and do not change the price you pay. Affiliate disclosure.

Specific model checks

flint-40b on this machine nemotron-3-5-bolt-e4b on this machine lfm3-1-7b on this machine bonsai-2 on this machine qwen3.8-27b on this machine gemma-4-12b on this machine gemma-4-e4b on this machine gpt-oss-20b on this machine qwen3.6-35b-a3b on this machine gemma-4-31b on this machine gpt-oss-120b on this machine qwen3-8b on this machine deepseek-r1-8b on this machine mistral-7b on this machine

Compare current hardware

The closest current-generation profiles by memory capacity are below. A higher tier can increase the context ceiling or make a larger quantization comfortable, but it does not automatically guarantee faster generation.

AMD Radeon AI PRO R9700 · 32 GB · 32 GB · 640 GB/s NVIDIA RTX PRO 4000 Blackwell · 24 GB · 24 GB · 672 GB/s Intel Arc Pro B60 · 24 GB · 24 GB · 456 GB/s NVIDIA RTX PRO 5000 Blackwell · 48 GB (72GB variant) · 48 GB · 1344 GB/s NVIDIA GeForce RTX 5080 · 16 GB · 16 GB · 960 GB/s

Run the interactive check

Sources and verification

Data reviewed 2026-11-02. Check the linked vendor material for the exact configuration before buying or installing.

https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/

FAQ

Questions about NVIDIA GeForce RTX 5090 · 32 GB.

How much VRAM does NVIDIA GeForce RTX 5090 · 32 GB provide for local AI?

NVIDIA GeForce RTX 5090 · 32 GB is listed with 32 GB of VRAM and 1792 GB/s of memory bandwidth. The operating system, display buffers, and other applications share that capacity, so the model budget is always lower than the bare hardware number.

Which current local models fit comfortably on NVIDIA GeForce RTX 5090 · 32 GB?

Flint 40B, NVIDIA Nemotron 3.5 Bolt E4B, LFM3-1.7B. Comfortable means the 8K estimate leaves at least 20% headroom after weights, KV cache, and runtime overhead. A larger or higher-quality quantization may move an individual result into the tight band.

What context length can Flint 40B at UD-Q3_K_XL use on this machine?

At the selected quantization, the formula reaches 262,144 tokens before the machine's 32 GB capacity is exhausted. The model's native context limit still caps the result, and real workloads may run out earlier because of fragmentation, other applications, or backend workspace.

Is the 1792 GB/s bandwidth figure a measured token speed?

No. Very fast relative band is a relative estimate based on active parameters and memory bandwidth. It is not a benchmark: backend kernels, batch size, thermals, CPU offload, and concurrent work can change real generation speed substantially.

What should I do if a model does not fit on NVIDIA GeForce RTX 5090 · 32 GB?

Try a lower-bit published quantization and a shorter context first. If the minimum requirement still exceeds 32 GB, the arithmetic gap to a 20% comfort target is about 46.2 GB. The next same-family current profile in this catalog is NVIDIA RTX PRO 6000 Blackwell · 96 GB.