No, you cannot create a file in memory. RAM: or PIPE: are the only possibilities. You can use the task pointer to make the file name unique.
Something like this:
char tempfilename[30];
BPTR tempfile;
sprintf (tempfilename,"t:tempfile%x",FindTask(NULL));
tempfile = Open(tempfilename,MODE_NEWFILE);
Execute ("command",NULL,tempfile);
Close (tempfile);
Open (tempfile,MODE_OLDFILE);
bytes_read = Read (tempfile,buffer,buffersize);
Close (tempfile);
Delete (tempfilename);
Bye,
Thomas