Although OF is a pretty good idea, you're a bit limited in the gfx cards that you can use (ie "Mac" gfx cards), and they are usually more expensive.
Of course you are mistaken. First of all, there is no need to use framebuffer (or even textmode) console in order to boot machine. You may initialize gfx card later on by OS itself.
Secondly many OF intefaced do provide real mode X86 emulation. It is for example true on my PReP machine: Motorola Powerstack.
Surely gfx support (either through build in p-code drivers or through x86 emulator) is nice-to-have feature, as you can use then initialized framebuffer screen and provide generic gfx driver, it's not needed.
BTW. Thanks to OF interface, the same AROS image is able (in theory) to boot from NewWorld CHRP Mac's, IBM PReP machines, Motorola PReP machines and Pegasos. Is then really UBoot better solution for AROS?