The toaster needs at the very minimum 1 meg of chipram to get started. My guess is that you have 1 meg onboard but the agnus is either 512k, or its 1MB but not jumpered correctly. If there isn't about 975k free in chip, toaster won't boot regardless of any amount of 32bit ram on the accel.
To get toaster to boot with only 1meg + 5 other memory, I had to reduce the workbench colors to 2. You can also reinstall and select the option to "boot directly to toaster" or what not, may save you some memory.
