Speaking, listening, reading and writing are all handled by different parts of the brain. That's why different types of aphasia affect different language modalities. It is perfectly possible for someone to have good reading/writing skills and not so good speaking/listening skills. You probably just need more practice at conversing in German to catch up with all the work you have done on your reading and writing skills.
Also remember that spoken language is fleeting - once it has been said, it's gone and you have to process it mentally (i.e. remember it while you're translating it). And with spoken language you are under pressure to conform to the conventions of conversation - i.e. to speak fluently and not pause to work out what you're going to say next. At least with reading the words remain on the page indefinitely so you can take your time translating them, and with writing you can take as long as you want to work out the next word.
--
moto