Could all of this be related to a defective Kickstart ROM in any way at all?
I'm asking because the KS ROM is the only thing i don't have 2 of, besides the a604n.
If you have your old KS2.0 chip and if it's 37.300 or 37.350 then yes it wouldn't hurt to try it.
So what happens when you power off at the brick, power back on again with your Workbench 2/3 disk in? Do you get the same results? I'm just wondering if it could be something in your Startup-sequence, like some module or something (what OS are you using BTW?). Or, another possibility is the filesystem stored in the RDB. What filesystem are you using for your CF hard drive?
Also, it could be that KS3.1 doesn't like your CF card/filesystem. Recently on my A600 I had prepared a CF hardrive with PFS AIO which worked fine under KS2.0....but would not boot at all under KS3.1. So I used a real had drive instead with PFS AIO and it booted fine under KS2.0 and KS3.1. I was only using the CF as a test and use real HDD's in all my Amiga's... but it's a matter of personal preference. As you know, some CF cards don't even work properly or at all on the internal IDE of A600/1200. I was using a Transcend 133x 4GB.
There's a tiny master/slave jumper on the CF adapter...maybe it's upset about that? Try a different position.