If most of planet is PC-based, retarget capability is minor in comparison to what you get with hardware level compatibility:
(1) Faster code
On slower GPUs that take an order of magnitude longer to code for.
(2) Faster response time (worst case analysis) and exactly knowing what is happening (for real-time stuff)
Which works fine right up to the point your "perfect" app ends up in IRQ or I/O conflict with one or more other applications.
(3) No drivers required; they all use same driver (like VGA standard)
lulwut? Of course a driver is going to be required, do you seriously intend to have everyone write their own partial drivers implimenting only the bits they need... Your way would make Windows95 look stable by comparason.
(4) Smaller and efficient code means less resources are used.
etc.
Which you then have to make sure doesn't interact badly with other code also running, meaning having to test against hundreds if not thousands of other applications, each with their own custom code for accessing I/O, memory, GPU, sound etc.
I have yet to see someone here to prove to me that API is required once you make a piece of hardware have more functionality. Making a piece of hardware more complex has no realtionship to whether it uses API-based compatibility or hardware compatibility.
It does when your application has to acces that functionality in terms of the time it takes to build, it is not that you get more functionality, it's that you get that functionality with far far less effort with an API.