Well, it depends on the kind of games you will play - if it's mainly for older WinUAE titles then don't bother as the stock A4000 is plenty fast for these.
OTOH if you're interested in more modern 3D FPS titles (1995+) then you will definitely notice a difference. The question is whether it's worth it, because these titles are rarely on par with classics as far as gameplay goes + the graphics is still ugly as hell. The fastest, best looking and most fun FPS I've played on my A4000/060 is Doom....