Yeah, I've broken a few Agnus sockets on 500s. I found the sockets at either Jameco or Mouser, and replacement was not a big ordeal.
Now, if the ones on the 4000 are surface mount, that would be a bigger ordeal for me as I have no surface mount experience. But for someone competent, probably not.
None the less, I would recommend a puller as well.