The A4000 always scans its built-in IDE bus. If you don't have anything connected to it, that's where the delay is coming from. Either connect something to it, or install the hardware hack to "fool" it into skipping the bus. I believe the description of the hack is available on Aminet.