Amiga memory nodes are prioritized, MEMF_FAST or MEMF_ANY will always get allocated from the fastest (=highest priority) pool.
So, slower mainboard RAM will not slow you down unless you run out of accelerator RAM - which would throw an error if mainboard RAM isn't available (MEMF_FAST) or would allocate even slower chipmem (MEMF_ANY).