Well, assuming you're using a system with secure memory, it also has secure file access permissions.
Also, even if someone else would gain knowlege of the memory addresses used, those adresses are only valid inside the current incarnation of the said application itself (yes, each process has it's own address space). Even if you get the logical address to the memory, you don't get the physical one, so it's impossible to access it from the outside.
So just pointers themselves are pretty much always harmless.
The above on systems such as bsd, linux, windows etc. On Amiga the system is exposed, so I doubt if leaking any pointer does any more harm anyway. Obviously this doesn't mean you should write passwords plaintext, or not wipe out plaintext passwords after reading/decoding them into memory.
