Yet I don't get this saintly image that people now have for him. Maybe I am missing something.
You're not. It doesn't make a lick of sense, we're just apparently obligated to venerate the dead unconditionally, irrespective of anything and everything they said and did in life. After all, he was keeping The Name alive, and that's what matters, right?