Sounds like a lot hassle to me. I wouldn't want to install some emulation environments onto my Amiga to run an Amiga game!
I understand that you want some speed, but I think you should write the game first and optimize then. And please, at least give SDL a chance before crushing it. ;-)
And if you want to write a game for the new Amiga, why wouldn't you do it in a system frienly way? We are still suffering because of lots of old software were written to bang hardware directly.