I was thinking in classic Amiga terms, thinking anything above 2Mb or GFX RAM would carry me through the day. I never expected this. :/
Depends on what you do with it. Whenever I'd run a 3D game on my 8MB gfx card, I'd always drop the workbench to 320x240x8-bit first to free up as much of the RAM as possible.
You have to be realistic though. A 1280x1024x32-bit display (a modest resolution for a desktop today) requires 5MB just for the frame buffer. There'll be various allocations going on all the time when windows are opened.
Suppose you wanted to play Quake 3, fullscreen, at 800x600 in 32-bit colour. You're bound to have at least double, if not triple buffering. Then there's the Z-buffer, which may be 16-bits (as it is on the Permedia, actually 15-bit with one bit of stencil) or 24-bit with 8-bits of stencil on many radeon cards. So, with triple buffering and a Z buffer, you might already have 4 800x600x32-bit buffers. There's 1.8MB gone immediately. If your 1280x1024 desktop display is still paged in, you've now got 6.8MB in use before you've loaded a single texture map.
The reason you never really thought of this with your 2MB chip-ram system is because you couldn't actually have resolutions that high. A single 256 HighGfx (1024x768) display, about the best you'll get from AGA, will eat almost half of that in one go.