I would agree that going from a Amiga 3640@25MHz to a 060@50MHz with local memory and SCSI Controller makes a HUGE difference.
Would also say, going from a Warp Engine 040@40MHz with local memory and SCSI to a CS MK III is faster. If you compare to a CS MK I or II without SCSI, the Warp Engine it is give and take. For math the 060 is faster, for general usage the WE is faster (Since the WE has faster memory access then the MK II).
I went from a WE/040 to a CS MK2/060 with the SCSI adapter.
I have and ACard IDE to SCSI adapter with a 32GB SDHC card fitted to it so it's nice and silent and the speed is much better than the IDE on my A4000 was. I've got an external SCSI DVD writer attached to the onboard A3000 SCSI too.