Only the 2/3/4000 series have a CPU slot - the others have multi-purpose expansion buses. The 128MB limit is in the CPU slot.
To be fair, though, I can't imagine the designers thought people would still be using the latest versions of AmigaOS on their A4000s in 2009! 128MB was a ludicrous amount when the A4000 was designed.