I use my A600 with 2MB ChipRAM and 2MB PCMCIA SRAM for WHDLoad games all the time. Most of them work without a problem on my setup, so I'm assuming with 4MB FastRAM you shouldn't have too many problems, although I'd recommend you get an A603 for that extra meg of ChipRAM and complete the job. If you're not afraid of unscrewing the A600 and opening it up, I'd recommend you get a Kickstart ROM Switcher with 1.3 and 3.1 ROMs on it, then you'll have the best WHDLoad system as well as something that'll have a heck of a lot less problems playing old floppy disk games. I have this in my A600 and it's been really handy for some of those older games that don't have WHDLoad installs yet.
ClassicWB is a good place to start if you're setting up an Amiga as a WHDLoad games machine, all you need to do now is download and extract the games you want onto your hard drive, and make sure you have the ROM images in your Workbench: Devs/Kickstarts/ drawer. Registering WHDLoad will make all your games load up faster too.