Karlos, where can I learn about both trap and patch and trap and emulate? Specifically, I'm interested in the trapping of FPU calls and the potential to improve upon them.
Trap and emulate is one of those things that the 680x0 programmer manuals will tell you about. All you are doing is implementing your own exception handler and then writing some code to deal with the exception (note that this all happens in supervisor state and you need to know the layout of your 680x0 exception stack frame which do vary from CPU to CPU).
You can write a handler to do some specific bit of work and then have it return. Normally, you'd write the handler to implement the unimplemented operation and return from the exception. However, you can go a step further and patch instead. Basically what you do here is modify the opcode that resulted in the exception and have it jump to a location of your choosing. If you are not fairly comfortable poking around in 680x0 supervisor mode this is not trivial to do, you have to be careful how much space there is to insert your jump and also you have to make sure you flush the instruction cache and so on. However, this is the basic gist of how tools like CyberPatcher and OxyPatcher do their magic.
I'm not sure if it will help you much but I played with some CPU exception handling a few years ago on 680x0 albeit for a different purpose:
http://www.amiga.org/forums/showthread.php?t=25181In this case, I was using the CPU to trap illegal operations and have it invoke a language level exception mechanism (a C++ throw in this case). It does demonstrate some of the sneaky shenanigans you can get up to though.
