I'm holding my breath for the comming Blackberry that will have qwerty keyboard AND Android, it will probably be a nice device for handheld Amiga if you need a new phone at the same time
You can get a PSP (phat old ones) super cheep and use CFW on it and use PSPUAE, long time since I tried it but that's handheld and it got a DPad + Buttons you need and you got a ton of other emulators and games.
There are more handheld options that are cheap, stay clear from the chinease crap gaming Android devices if you do not know if it's good or not, a lot of them have crap controls :/ Some works ok tho so do some reaserch first
Oh Almost forgot, the old Sony Xperia Play can be found cheap and have excelent controllers that slide out, but yet again then you get a phone at the same time