Any accelerator you get should have fast ram, the a3640 is an '040 with no fast ram and I had an '030 50mhz that could beat it. From what I've seen an '040 25mhz with most software is roughly the same as '030 50mhz.
With a 3000 or similar small case '040s can be very hot. '030s are cooler, my '030 50 never had a heat sink or stability problems. On board scsi is a good thing too.
There are some differences in compatibility with the '040 too but I don't think it's much of an issue nowadays.
Personally I would consider case/heat, features, and compatibility over raw speed. Unless you are trying to get amiga quake to run, then you want the fastest '040 or '060 and lots of fast ram.