Reading gfx card memory is really slow. Even writing is in most setups much slower than normal memory, and the memory is cache inhibited (CPU won't cache the memory accesses).
However technically it would go something like this: Allocate a large chunk of the graphics memory permanently (probably only possible with Picasso96). Add the memory to system with exec AddMemList() call.
This only works for setups that have the video memory mapped to amiga address space in linear fashion (this rules out A1200 Mediator + Radeon 9200 as system memory, for example).
