I don't agree. You could split the ram in two sections using the MMU. The 68k programs would be put in one memory space and all accesses from 68k would be converted to little-endian and the accesses from x86 software to 68k modules as the 68k modules/programs would be in the "68k memory space" would simply convert the addresses.
Sorry, not possible.
Take, for instance, this case. We have a little-endian ULONG in memory, whose value is, say, 0x01020304, hence it's stored like this (values are in HEX, pipes divide bytes from one another, leftmost digit is the one at the lowest memory address):
Little endian 0x01020304:
|04|03|02|01|
Now say a 68k program wants to access that value, but for some reasons it will do it one byte at time, storing each one of those bytes in consecutive memory locations. Now, since it's doing a byte access to memory, there's presumably no need to do any byte swapping, or rather, it's simply impossible to do any byte swapping. Now, let's see what's the end result of this operation:
Big endian 0x04030201:
|04|03|02|01|
As you can see, the 68k program read the wrong value: it read 0x04030201 whilst it should have read 0x01020304.
OS4 has done that and it's the most portable solution to emulate 68k on x86 IMHO.
AOS4 (why do you people keep forgetting the 'A'?) has not done that at all, it doesn't do any byteswapping, it simply doesn't need to.