Have you tried fullscreen?
The source code I've seen for classic SDL is horribly inefficient for windowed mode.
I tried a port of ScummVM 0.9.1 using the old MorphOS backend updated a bit and modified, but it tends to crash a lot and doesn't work consistently (it may work on my A4000/060 but not under WinUAE and vice versa) so I never really bothered much with it. The problem seems to be related to the horrible quasi-threading for sound/timing.