With my port of Quake I've discovered that RTG is not really much quicker than AGA, it seems that the C2P/AGA is only slowing it down by about 3% at 320x200 vs RTG direct fame buffer writes.
With 32bit miggies (AGA or Amiga3000) you can write to chipmem up to 7MB/s while ZorroII only allows half of that so in some cases AGA could be even faster. Peter McGavin already stated it in ADoom documentation and I found it interesting, he only recommended ZorroIII or faster bus.
In quake most of the bottleneck comes from the cpu.
IIRC Doom doesn't write to the screen sequentially but in columns so pixels can't be grouped easily when writting to chipram. Quake probably does the same so writting to gfx ram a row of 4 contiguous pixels requires may require redesigning the rasterizer, that could speed up writes notably. Perhaps rendering in 8x8, 16x16 or 32x32 groups speeds up rendering
I think that the engine could be optimized moving to fixed point some parts. AHI usually also slows down everything so perhaps Hyperion's quake2 sources with its paula driver comes handy.
You could install CGX AGA driver and compare the speed of your port with Hyperion's Quake2.