For one thing, the AGA chipset only uses the fastest memory fetching on the highest resolutions. On 3.1 it uses the fastest memory fetching whenever it can thus making the chips run faster. If you're playing games you'll get better performance much of the time with Kickstart 3.1 .
For another thing the sprite handling is broken for using AGA enhanced sprites on Kickstart 3.0 .
That's just 2 bugs in graphics.library alone that make the Kickstart upgrade worthwhile.