Amiga accelerators are affected by slow ram. e.g. I had an A2000 with 68030 and 2MBs of 32-bit ram on the accelerator. Also there was 4MB of 16 bit ram in a zorro slot. The machine would fly on while it had that 2MBs available. If I turned off the CPU slot ram the machine would slow down... a lot. If you turn off all extra memory and it has to share the chip RAM memory there is an even bigger slow down, only slightly faster than a stock unexpanded system.
Starting with an 68040 (8k cache) you start to notice the performance benefit of the cache memory. Processors nowadays wouldn't be any faster if they didn't have very fast and adequately sized level 1 cache memory. Level 2 cache memory larger, but slower only adds about 15% more speed.