The short version: on Linux you generally do not want to carve a large VRAM slice in the BIOS. Our serving node leaves the dedicated slice tiny and lets the GPU reach almost all of system memory through GTT — the kernel’s graphics translation table — and the ceiling that decides how much it can reach is a TTM kernel parameter, not a firmware menu. Below is what that machine reports right now, with two large models resident.
The symptom
You buy a 128 GB Ryzen AI Max+ 395 box specifically to run big models, and then the runtime announces a few gigabytes of usable device memory. Or a model that should fit refuses to load. Or ROCm and Vulkan disagree with each other about how much memory exists. The instinct is to reboot into the BIOS and hand the GPU a bigger slice, which is where a lot of time gets lost: the dedicated slice is not where a Linux box with unified memory keeps its models.
What our node reports
This is a Beelink GTR9 Pro serving llama.cpp, on a 7.0 kernel, read straight from sysfs while two models are loaded:
mem_info_vram_total: 512 MiB # the BIOS-dedicated slice
mem_info_gtt_total: 120266 MiB # what the GPU can actually reach
mem_info_gtt_used: 108704 MiB # two resident models, right nowHalf a gigabyte of “VRAM” on a machine happily holding a hundred gigabytes of weights. The dedicated slice is close to the minimum the firmware offers, and it does not matter, because everything of consequence goes through GTT.
The setting that actually decides the ceiling
GTT lets the GPU map system memory, and how much it may map is bounded by TTM, the kernel’s memory manager for graphics devices. Its page limit is what to check first:
# pages, at 4 KiB each
cat /sys/module/ttm/parameters/pages_limitOn our node the kernel command line carries ttm.pages_limit=30788203,
which is where the ~117 GiB ceiling above comes from. On many distributions
the default is a fraction of system memory, and that default — not the BIOS
— is what stops a large model from loading. Raise it, reboot, and re-read
both sysfs files before believing anything else.
Two things worth knowing while you are in there. First, verify from the
device, not from the tool: mem_info_gtt_total and mem_info_gtt_used
under /sys/class/drm/card*/device/ are the ground truth, and runtimes
frequently report their own idea of “VRAM” that means something else
entirely. Second, leave headroom: on our node, with models resident, the
host had a few gigabytes of ordinary RAM left — GTT allocations come out of
the same physical memory your operating system is using, and a box that
swaps while serving is a box with a latency problem you will misdiagnose as
a model problem.
Why the BIOS slice is the wrong lever
The dedicated slice is carved out of the same physical memory and then becomes unavailable to everything else, including the GTT path the runtime prefers. Handing the GPU a large fixed slice does not add capacity — it converts flexible memory into inflexible memory. Windows behaves differently here, and much of the advice circulating for this hardware silently assumes Windows or an older kernel; check which platform an instruction was written for before rebooting into the firmware menu.
None of this is a claim that a large slice never helps any workload. It is an account of what one serving node, tuned for exactly one job, reports while doing that job.
What to check, in order
cat /sys/module/ttm/parameters/pages_limit— the real ceiling./sys/class/drm/card*/device/mem_info_gtt_total— what the GPU can reach.mem_info_gtt_usedwhile your model is loaded — whether it went where you think it went.- Free system memory under load — headroom, not just capacity.
- Only then, the BIOS.
Scope
One node, one kernel series, one distribution, tuned for llama.cpp serving. Kernel and driver behavior on this hardware has been moving quickly; treat the numbers above as what this machine reports today, not as a specification. What the box does once the memory is sorted out is measured in the 128 GB report, and the protocol for measuring your own is here.