I downloaded Term by Olaf -- for '020/'030/etc and it doesn't run under OS4.1FE.
I'm 100% sure it used to run under 4.0 and (probably) 4.1, can't remember the exact setup required though - it's been to long.
And when I try to run it again it complains that serial.device is busy (I want to use it over ethernet for SSH/Telnet tho)
You need to use a replacement (fake) serial.device that supports TCP, these can be found on Aminet. That's true for any Amiga Terminal program you might want to use though.
In the end I want to SSH/Telnet to a Limux box and have color, ANSI emulation on board.
Unless said Linux box is local, you shouldn't use Telnet, AmTelnet or any Amiga terminal programs due to their lack of or completely outdated encryption technology.
For SSH'ing into a Linux box I simply used Steven Solie's
ssh port. I did have some setup issues with either the cursor keys or the German umlauts (can't remember the details) - but given the upload date of the package now on os4depot, I was probably using an earlier version.
As for how much ANSI the OS4 console supports - no idea. just give it a try.