I suppose the easiest way to achieve this is to open up your A4000D and remove some fast memory chips

just kidding of course. I am still waiting for my unit to be shipped so I'm curious about how it all works.
But there might be another issue regarding emulating a A4000, the cpu.
According to the information at github (BlitterStudio/amiberry/wiki/ information there are two amiberry versions available, Amiberry and Amyberry-Lite.
Lite is supposed to be used for slower hardware up to Pi4 speed, so I guess this is the version in use for the Orange Pi Zero 3 (A600GS)
From the wiki:
Cycle accurate for 68000 mode only, stripped down CPU emulation for other models. Models higher than 020 (030-040) don't use proper CPU cycles, they are just faked using 020 (they still show up as the selected model under emulation, of course). No 68060 support.
You could probably compile your own full Amiberry version, but I guess it would be slower on this hardware.