That doesn't follow. Damage to the brain would only have no effect on personality if the brain played absolutely no part in things beyond mechanical coordination, which I don't think most people who believe in a soul are claiming.
I was under the impression that souls were autonomous as they can carry on existing after we are dead. No religion has provided any explanation of that though, so it is possible that your brain and soul could work together and when your brain died your soul goes off in some form of emergency mode.
Again, though, the problem with that idea is that a "soul" that exists within a biologically-deterministic flesh-and-blood creature doesn't fit the general definition of a soul at all, because it's still (theoretically) bound by biochemical determinism.
There is no general definition of soul. I googled definition of soul and it came back with:
1. The spiritual or immaterial part of a human being or animal, regarded as immortal.
2. A person's moral or emotional nature or sense of identity.
The visible effects of a person's "soul" are what everyone can agree to, where that comes from is what is up for debate. Religion's don't own the word soul.
I believe the human "soul" is deterministic, it's just currently too impossibly complex to model it. The idea of the soul being separate came because they couldn't comprehend that anything in the human body could do something that complex, they didn't have digital watches then either though.