I also had problems with the Mediator setup. Sometimes the 3640 card is very picky to get it together working.
I sold my Mediator 4000DI V1 and bought a Mediator 4000DI V3 and solve the problem.
My 3640 adapter has Speedgeek mod to remove the two wait states.
It makes your Amiga 4000 I/O as fast as the 3630.