Some things to try:
Remove all cards,test with a bare bones system.
Yellow = CPU Exception Before Bootstrap Code is Loaded
Just a guess but did you clean and neutralize the battery damage well,or is there a marginal trace going bad between CPU and ?? if it was not neutralized at time of repair,it will keep eating traces. This sounds like a possible bad trace to me,but never know.
Did you install a new Agnus socket or using the old one,they are notorious for problems after 20-30 years now.
Edit: GadgetUK has a video showing ram can cause this also:
https://www.youtube.com/watch?v=qfVxurhovokThe 5721 buster may be problematic for Zorro. some older Toshiba ones were notorious for failure. CSG version usually ok. that being said it could also be the bus buffers.
Move the cable to the other floppy and see if it acts the same. Some Chinon floppies have a electrolytic capacitor on the bottom that can go bad.
I think its just a case of you interrupting loading,but it does sound a bit odd.. try a different workbench disk.
pictures of the battery damage would be great.
I don't think there is a CIA problem but you can swap chips from socket to socket to see if any behavior changes.
Good luck and let us know if you make any progress.