Two ways to feed a model
To run a language model, its weights must sit in memory the processor can reach quickly. There are two architectures for that:
Discrete GPU (VRAM): a graphics card has its own onboard memory — fast, but fixed at purchase. An RTX 3090 has 24 GB of VRAM at ~936 GB/s. If the model doesn't fit in VRAM, layers spill to system RAM over a comparatively slow bus, and speed falls off a cliff.
Unified memory: the CPU and GPU share one physical pool. Apple Silicon Macs, NVIDIA's DGX Spark, and AMD's Strix Halo machines work this way. A Mac with 64 GB of unified memory can hand most of it to the GPU — so it can load models that no consumer graphics card can hold.
The tradeoff: capacity vs bandwidth
Unified machines usually have more capacity but less bandwidth than a good discrete GPU:
| Machine | Memory | Bandwidth |
|---|---|---|
| RTX 3090 (discrete) | 24 GB | ~936 GB/s |
| RTX A4000 (discrete) | 16 GB | ~448 GB/s |
| Mac M4 Pro | up to 64 GB | ~273 GB/s |
| NVIDIA DGX Spark | 128 GB | ~273 GB/s |
| AMD Strix Halo | 128 GB | ~256 GB/s |
Capacity decides what you CAN run. Bandwidth decides how FAST it runs. A 128 GB DGX Spark can load a 70B model that a 3090 can't — but a small model that fits both will generate about three times faster on the 3090, because generation speed tracks memory bandwidth almost linearly.
Practical notes
- Unified machines can't give the model everything: the operating system needs a share. Plan on roughly 70–80% of the sticker number being usable.
- Unified memory is soldered. There is no upgrade path — buy the capacity you'll want in three years.
- Spilling a model across VRAM + system RAM (partial offload) works but is dramatically slower than either pure architecture. Treat it as a last resort, not a plan.
Which should you buy?
If the models you care about fit in 16–24 GB (up to ~32B at Q4), a used discrete GPU gives you far more speed per dollar. If you want 70B-class models on one quiet box, unified memory is the only realistic path. Our Can I Run It? calculator has presets for all of the machines above.