@TheMagicM
Man, you just missed a deal on FleaBay!! A 4000 mobo with 3640, daughter card for $100 buy it now. I almost got it but I didn't need it. The 4000 MBs typically sell for $150 - $200 on the net, so they are expensive. What you could do to save some cash is get one from Europe. It will work 100% in the states, all you need to do is set the NTSC jumper and you are all set. I built up a 4000 myself like you are, took me a few years to get all the parts! See my photo album for a pic of it.
The 4000 is my favorite Amiga! I have four of them.

I agree the 3640 040 is sllooowww on the 4000. But for the time being if you can get one for cheap go ahead, but you will want a card with local mem on it. I have a GVP-m 4060 and I got it running at 66Mhz, it rocks.
Good luck!