Still, at that point you're harnessing a 150MHz CPU to a bus and chipset that still run at 3.5MHz;
You'd put ram on the CPU bus, so you wouldn't need to touch the motherboard until you access the customer chip registers. With write queuing then it wouldn't impact performance much. Reads are more of a problem unless you can execute instructions out of order.
The bus on AGA amigas runs faster than that as well.
Even modern PC's have multiple buses running at different speeds.