The last time I heard the major problem was speed, graphics.library/HID were designed around a fast processor and vga and not 68000 and ocs/ecs/aga.
I'm hoping that a new driver architecture will be created at some point that solves that.
from Aros Shortlog (Toni Wilen)
"Partially revert r45687, since m68k-amiga blitter support was added, all planar bitmaps have to be in chip ram."