So, after letting it be for a few days I put a new fuse in, connected it and powered it up without the GVP HD8 connected. It seemd to work just fine, it sat there waiting for a disk to be inserted and I didn't see any artifacts on the screen either.
Turned it off and let it be for a few minutes, turned it on again and I saw some artifacts on the screen before the fuse went POP.
Does this indicate a cap issue in the PSU?
I mearsured the voltages with proper tools and I got about 5.45V on the 5V pin, 12.5V on the 12V pin and exactly -12V on the -12V pin, this when it wasn't connected to the amiga as well. No fluctuations to talk about really.
I did manage to measure a bit when it was on the first time while it "worked" and the 5V was rock solid while the 12V fluctuated betwen 10V and 12.5V. Don't really know where to measure the -12V though so don't know how that fared when connected.