4 times faster!? Perhaps in benchmarks, but in real life it feels marginal faster than the 68030/50. Only the FPU seems to be a lot faster. I have used both systems, so i'm talking from my expirience.
One other issue is that the 68040 is much more cache sensitive. If you need to disable the caches for compatibility reasons, it becomes a LOT slower than the 68030 with its caches disabled.