In current releases of Amiga OS 4 the TCP/IP stack is started by an AddNetInterfaces command in Startup-Sequence. Therefore you should check your startup-sequence and user-startup if there is such a command and remove it.
After that (and only then) check the Libs directory if there is bsdsocket.library. If there is, delete it. (If you delete the library before you removed the start command, you might get problems during the boot process).
There might also be usergroup.library. Delete this, too.
