Wow,
I must say that Jethro Tull's "book" ;-) below is just amazing. I agree with him in every point.
There is another thing also that I think is important. We are talking about AGA compatibility using FPGAs. Those are reconfigurable chips. This means we can have AGA if we download the right stuff on then, but if we need something faster, we can write a better graphics display and download it. With FPGAs we are not stuck with physical AGA all the time, you can change it on the fly.
Using a MPC5200 (I just loved this chip.. :-D ) we also get AC97 audio, which means we could use the audio part the FPGA to be part of graphics stuff when reconfigured. I'm pretty sure it would be no match for a top of the line PC Graphics board, but if you really want one of these, all you need is PCI or AGP.
A nice FPGA, a MPC5200, RAM, a codec, and some glue logic would make a very small board at a nice price. It could even be portable if we connect it to a LCD.