You say you don't play games, but you don't say what you do do! :-o
If you are doing graphics intensive work (web browsing, digital imaging, desktop publishing etc) then it would certainly be worth getting a graphics card, and a tower makes adding bits (ethernet card) easier & cheaper.
Although I have an 060, the graphics speedup was measurable over AGA by seconds!
I used to use a wireless PCMCIA card, but I get better connections by ethernet, so that is a nicer experience.
Plus, seeing a TrueColour Workbench is like an automatic car. Once you used it, you'll never go back :-D