Well going from 060+onboard ram to a3640 is a good hit on speed, Any of the eyecandy installs will be slow even with mediator. I am guessing os3.9 will be the best bet for what you have there. The cf card is a no brainer, near zero access time compared to a hd makes them snappy.
Some other options if you are handy, the 3640 can be sped up quite a bit, 30-35 mhz with the onboard 25mhz 040 is possible. its noticeable.
Even faster speeds with a 040/40mhz part may be possible, the pals on the 3640 are good to 50mhz,and 40-45mhz may be possible(with a good sink+fan).
Another option is the 060 adapters that work on 3640.
Of course this does not solve the fast ram issue and zorro3 ram is pretty slow,but sometimes its good to have the ram.I doubt you will use up the 128mb on the zoram.
Mapping radeon ram to the amiga might be slightly faster than zoram,but i have not tried a speed comparison, it may not make any noticeable speed difference since z3 is the bottleneck.