uIP/lwIP for Amiga OS

[Trev]

I've had a look at the cnet.device source, and I think the problem is a very poor assumption on my part regarding DMA transfers. On a stock A600, the only place for memory to be allocated is in chip memory, and AllocVec should have properly aligned it. cnet.device only supports DMA for writes (sends), which is probably why reads (receives) are still working. I've removed the S2_DMACopyToBuff32 and S2_DMACopyFromBuff32 tags from this build. The debug version of cnet.device (unfortunately, only distributed in binary form for 68020 and higer) would have sent something like "cnet: using DMA write from $nnnnnnnn!" followed by "cnet: txintcode: transmit DMA timeout!" to a debug console (or sashimi).

Updated: http://www.babacar.org/uip.lha

If at any point in time I start talking out of my ass, hopefully folks will continue to chime in and correct me. ;-)

Trev

[Trev]

Also, I received a small and simple patch for MorphOS support for anyone that wants to play with the currently posted source code. Thanks, Stefan!

uip.diff

Code: [Select]


--- m68k-amigaos/sana2dev.c Tue Nov 13 18:53:50 2007
+++ ppc-morphos/sana2dev.c Fri Nov 16 14:37:02 2007
@@ -33,7 +33,12 @@
 #include <SDI_compiler.h>
 #include <clib/alib_protos.h>
 #include <clib/exec_protos.h>
+#ifdef __MORPHOS__
+#include <devices/sana2.h>
+#include <emul/emulregs.h>
+#else
 #include <devices/sana2r4.h>
+#endif
 
 #include <stdlib.h>


Makefile

Code: [Select]


all: uip

CC     = gcc
AR     = ar
APPS   = webserver
CFLAGS = -Wall -O2 -fpack-struct -s -noixemul -I../uip -I.
-include ../uip/Makefile.include

$(OBJECTDIR):
mkdir $(OBJECTDIR)

uip: $(addprefix $(OBJECTDIR)/, main.o sana2dev.o clock-arch.o) apps.a uip.a

clean:
rm -fr *.o *~ *core uip $(OBJECTDIR) *.a


Speaking of ports, the next time I upload source code, I'll make sure vbcc build instructions/scripts/makefiles/whatever are included for at least m68k-amigaos and ppc-morphos.

Trev

[Trev]

I'd like to see DMA working properly, but I think the number of devices (software, not hardware) that actually support anything other than 8-bit transfers is limited. Options are 8-bit, 16-bit, and 32-bit bufferred and 32-bit DMA. Reducing the number of copies would certainly improve performance. Memory tricks in the copy callbacks would be dependent on knowledge of how the SANA-II device allocates memory, and that's why there are multiple tags.

As far as configuration goes, I think supporting all three options--static, DHCP (and maybe BOOTP), and ping-assigned--is a good way to go. uIP is meant to be embedded, so command-line options are probably a better choice than a configuration file. That should let multiple uIP applications coexist nicely, assuming the SANA-II device behaves properly. I'm still liking the idea of uIP in ROM or flash, but we'd need a SANA-II device that fits in with that scheme as well. Devices that completely autoconfigure and don't rely on features not availabe at startup should work. You think?

lwIP also supports link-local auto-configuration, which is a great feature for small networks. Even without auto-configuration, other link-local features, including .local name resolution, are pretty cool and fit in nicely with the spirit of the Amiga.

EDIT:

Jens warned against using the RR-Net on an Amiga, but with enough hacking, I'm sure a clock-port-enabled A500 could be up and running, too. Are there other ethernet options for the A500? ... Looks like there were a few sidecar-style adapters made, but that's no fun. And I guess parallel adapters are out. Serial-to-ethernet is an option, but those devices tend to be microprocessors and firmware that do all the IP work for you and wrap RS-232 or other standards. But with a little creative bridging....

Trev

[Trev]

@Hans_

uIP has IPv6, too, but I haven't tested it.

@all

New version. Back to a simple event loop to aid in troubleshooting applications as opposed to my own bumbling.

Instead of ping-based configuration, this version supports both DHCP (unlike my version of Genesis) and static addresses. It should be possible to keep ping-based configuration around as a backup to DHCP, but I'm not sure that's necessary when static configuration is an option.

I've also added more Amiga-like command-line options via ReadArgs(). Here's a quick summary:

/*
 * Command-line options.
 *
 * Default: DEVICE Devs:Networks/openpci_8139.device UNIT 0 CONFIG DYNAMIC
 *
 * DEVICE
 *   Fully-qualified path to device to open.
 *
 *   Example: DEVICE Devs:Networks/openpci_8139.device
 *
 * UNIT
 *   Unit number to open.
 *
 *   Example: UNIT 0
 *
 * CONFIG
 *   IP address configuration. Valid values are DYNAMIC and STATIC. If
 *   STATIC, the ADDR, MASK, and DEST options are required.
 *
 *   Example: CONFIG DYNAMIC
 *   Example: CONFIG STATIC
 *
 * ADDR
 *   IP address.
 *
 *   Example: ADDR 192.168.0.100
 *
 * MASK
 *   IP subnet mask.
 *
 *   Example: MASK 255.255.255.0
 *
 * DEST
 *   IP default gateway.
 *
 *   Example: DEST 192.168.0.1
 */

So, to configure cnet device 0 using DHCP, you can run any of the following commands:

uip device Devs:Networks/cnet.device unit 0 config dynamic
uip device Devs:Networks/cnet.device unit 0
uip device Devs:Networks/cnet.device

Similarly, for static address assignment, you can run something like the following command:

uip device Devs:Networks/cnet.device config static addr 192.168.0.23 mask 255.255.255.0 dest 192.168.0.1

Note that if you specify 'config static', the addr, mask, and dest options are required.

There's a note in the 3.5 autodocs about ReadArgs() expecting newline characters after strings passed as defaults in the option array. I haven't done that, so if using the defaults fails on EXEC <= V37, try using a complete command-line. Anyone have details on this one?

http://www.babacar.org/uip.lha

So, now we need to come up with some creative uses for uIP--something better than a silly test web server. A general purpose web server would be fun. One that uses the file system instead of pre-compiled files. And performance can be improved. (The port I did for MS-DOS packet drivers performs quite well.) The stack isn't sending full-sized packets, so there's probably a simple buffer issue to overcome.

As I've said before, bsdsocket.library support would be very cool, but I'd rather do that on lwIP.

Trev

[Trev]

Maybe the web server option could also be togglable?!

It could be, but it's the applications that make it useful. I suppose a swiss army knife style bundle of server-based apps would be cool. A stand-alone port of wget sounds interesting, as does a stand-alone SNTP/NTP client. Think WATTCP for Amiga (only not WATTCP, which would have been an interesting port, too).

When you say "ROM" you mean Kickstart?!

That's what I was thinking, for remote boot options.

I'll put some work into optimizing performance first. I could add an option for multiple simultaneous receive requests, but dropped packets don't appear to be a problem yet. (That's dependent in part on how frames are bufferred in the hardware itself.)

Sends are controlled by the event loop, so there isn't really a need for multiple send requests; however, performance could be improved by spawning a request management task that does nothing but wait for IO completion and put requests back into the queue. That would remove the requirement for a WaitIO() call in the send routine.

I'll be replacing the default checksum routines with 680x0 optimized ones as well.

Trev

[Trev]

I'm using a batch file and a custom vc.config file at the moment. The next source release will include gcc and vbcc makefles and a sample vc.config file for the vbcc vc front end. Really, it should compile with the default vbcc target packages. That shouldn't be difficult to do, assuming vbcc is installed on the host system per the vbcc documentation.

I'm no longer using gcc as there's no current support for the m68k-amigaos port. Everyone (including me) bases their packages on years old Geek Gadgets patches, and the results for building cross-compilers tend to vary. It's too bad Hyperion isn't keeping the OS3 compiler stuff up to date as they continue work on OS4. I don't have the compiler kung fu to do it myself.

Trev

[Trev]

@hardlink

Here's a Makefile that works for vbcc:

Code: [Select]


.PHONY: all clean
all: uip

CC     = vc
APPS   = dhcpc webserver
CFLAGS = -I../uip -I. -lamiga

-include ../uip/Makefile.include

uip: $(addprefix $(OBJECTDIR)/, main.o sana2dev.o clock-arch.o $(APP_SOURCES:.c=.o) $(UIP_SOURCES:.c=.o))
$(CC) $(CFLAGS) $^ -o $@

clean:
rm -fr *.o *~ *core uip $(OBJECTDIR) *.a


The default target is determined by your vc.config file. I'll expand the makefile later to allow target selection when building using the default vc config file names, e.g. aos68k. The config files need to be modified appropriately for your host (environment variables, OS include paths, etc.).

You'll also need to modify Makefile.dhcp in uip-1.0/apps/dhcpc to kill the vlink errors:

Code: [Select]


APP_SOURCES += dhcpc.c


timer.c is compiled as part of the main uip source.

Trev

[Trev]

It sounds like your version of make doesn't support the -include option. The missing rules are defined ../uip/Makefile.include, ../app/dhcpc/Makefile.include, and ../app/webserver/Makefile.include. The APPS variable tells ../uip/Makefile.include to include each of the application-specific makefiles. Are you using GNU make? If so, which version? If not, you'll have to sort out the makefile options and commands and build one large makefile. Shouldn't be difficult.

Your includes should be (in addition to the OS stuff in vc.config):

-I../uip -I. -I../uip/apps/webserver

Compile and link the following modules:

main.c sana2dev.c clock-arch.c ../uip/uip.c ../uip/uip_arp.c ../uip/uiplib.c ../uip/psock.c ../uip/timer.c ../uip/uip-neighbor.c ../apps/webserver/httpd.c ../apps/webserver/http-strings.c ../apps/webserver/httpd-fs.c ../apps/webserver/httpd-cgi.c

Link with amiga.lib, e.g. -lamiga. It shouldn't work with the minimum startup code, so you're vc.config will need to use the biggy. That's something that needs to be changed to minimize executable size and memory usage. I just couldn't be bother to do all the library management myself when I got started. :-)

The application doesn't do much without the web server. ;-)

I ditched the __attribute__ stuff and switched to an external (defined in sana2dev.c) uip_buf allocated by AllocVec.

uip_tcp_appstate_t should be typedef'd by all the TCP applications in ../apps. (Changing how that works is one of the challenges of making uIP more general purpose, i.e. to allow more than one application. If a bsdsocket.library emulation layer was the application. . . .)

I'll upload some new source code tonight. It should build correctly assuming your version of make likes the makefiles.

Trev

[Trev]

You can download the Cubic IDE compiler package separately. Pretty sure it's stuck in the same Geek Gadgets quagmire as the rest of the community. And up to date port of make would be nice. Apart from that, Cubic IDE (and GoldED in general) is pretty cool.

There's a copy of make 3.81b1 on http://amiga.sourceforge.net, but I haven't used it.

Try changng '-include' to just 'include'. The '-' ignores errors. Perhaps that feature wasn't available in 3.76.1? I don't know enough about make's history to say.

For the curious, I use Visual Studio as my IDE/editor (currently Visual C++ 2008 Express Edition), a makefile project, and Cygwin (as note before) to host the compiler (and make and anything else I might need). I move stuff back and forth between the PC and the Amiga using floppies. (My house is littered with Catweasels, but I'm using FAT floppies at the moment.) I was using Amiga Explorer, but the latest version has some shell extension-related oddities--and it's a pain opening it up and shutting it down on the Amiga side to test the changes to uIP. Floppies are faster.

I tried AmiDevCpp, but the IDE isn't a well-behaved Windows application. Plus, it wouldn't save the option to override the default makefile. And as we saw earlier, there were issues with either the cross-compiler or the startup code. I want to play with uIP and lwIP and not compilers and IDEs, so I switched to vbcc. :-P

Trev

[Trev]

@mboehmer_e3b

I certainly do seem to have some issues with device configuration and initialization on a number of different interfaces. I've been out of town for a few days but will be fixing broken things and adding new things tonight and tomorrow.

I believe the slowness in ICMP echo replies is caused by the structure of uIP's event loop and uIP's default checksum calculations. I'll be adding M680x0 optimized checksum routines tomorrow.

I haven't compiled in the DNS resolver, as from the current demo's perspective, it wouldn't be very useful. Is that what you meant? On the remote host (not the Amiga running uIP), adding DNS or hosts entries is system-dependent.

@Colani1200

Dropped packets are probably not a good thing. I'll look into it. I'm going to solve the 100% CPU usage issue as well by moving back to signals and waits; however, they'll work properly this time (they were the cause of dropped packets in the past).

I'll also be bringing the maximum packet size back down to 576 bytes as path MTU discovery hasn't been implemented. This will make the stack more interopable, but if/when SLIP (probable) or PPP (not so probable) support is added, I'll need to look at the impact of the MTU size in general.

Trev

[Trev]

Updates:

The first response to any request (ICMP/TCP/UDP) was dropped in leu of an ARP request if the ARP cache did not contain a mapping for the destination host. This has been resolved [1] by enabling uIP to update the ARP cache whenever a request is received.

1. This may not be the desired behavior (the source hardware address for an IP may not be the same as the destination hardware address for the same IP, i.e. routers, hosts with multiple interfaces, hosts that do transmit load-balancing across a virtual interface, etc.), so I should probably make this an option. If disabled, it would be up to the requesting application to retransmit (TCP) or ignore (ICMP echo and UDP) the unacknowledged or dropped packet, respectively. For now, it's enabled.

More to come....

[Trev]

If the A600 is taking longer than 1 second (de facto standard delay between requests) to respond to an ICMP echo request, then yes, something will get dropped. I'll need to look at the counter implementation and see what gets ticked as dropped and what gets discarded, i.e. packets that weren't meant for the active uIP process but made it past the link layer.

[Trev]

Thinking out load, it would be nice to have a checksum off-loading device in the Amiga. I see possibilities. It may be possible to use the blitter on slower systems, but I think the CPU will be faster on even slightly upgraded systems. (Thoughts?) 3D chipsets, e.g. Voodoo 3, and advanced processors, e.g. 68060, might be used for vector and superscalar operations.

Anyhow, an optimized checksum routine should suffice. Benchmarks to come over the next few days.

[Trev]

I do know that with a single receive buffer, the stack doesn't process incoming requests quickly enough. For example, Internet Explorer ACKs the connection and sends the first HTTP GET in less than a microsecond. uIP hasn't yet processed the pending I/O request for the ACK and so never receives the first HTTP GET.

Using more buffers uses more memory, of course, so we're down to a memory/performance trade-off. The number of I/O requests to queue will be an option soon.

Trev

[Trev]

Multiple receive buffers have been implemented and a command-line option (BUFFERS) has been added. The minimum number of buffers is two--one for IP and one for ARP. Currently, the first buffer is reserved for ARP and the rest are used for IP. Each buffer consumes ~600 additional bytes of memory. Three or more buffers provide a decent level of interactivity.

The device I'm using (openpci_8139.device) sends "orphaned" frames to all pending S2_READORPHAN requests. I'm not sure if all devices do that, but I suppose it makes sense. Device driver authors have no way to determine the intent of a task or process when it queues multiple S2_READORPHAN requests. Anyhow, I was receiving multiple copies of frames after implementing mulitple receive requests. That lead to the split between ARP and IP requests. Reads and writes are still raw (the entire frame minus the ethernet trailer is copied to the buffer), but reads will only catch ARP and IP frame types.

EDIT: You know, a device also has no way of knowing how a process or task is going to use multiple ARP or IP requests. As devices are required to service requests for all callers as of S2R3, device driver authors would have to make a conscious decision to send one frame per CMD_READ request once per task or process and not the same frame across multiple CMD_READ requets. Not sure why the same decision wasn't made for S2_READORPHAN requests. More digging to do, I guess.

The PNG image in files.shtml failed to transmit because of a bug in the Perl script that converts content into C. The script was using read(), which wraps stdio. On Windows, CRLF sequences are converted to LF. Not a good thing for binary files. Replaced read() with sysread() to fix that.

Still have checksums to implement. I also have a few ideas about why certain devices aren't initialized properly.

[Trev]

A 680x0 specific checksum routine has been shamelessly stolen from RTEMS (uses a BSD stack) and modified for vbcc. Oh, the joys of open source software. Cycle-wise, it's about 250% faster than uIP's default routine (but only a pinch faster than the BSD optimized C routine). Being BSD-derived, the routines are very similar to AmiTCP's. I'm steering clear of AmiTCP code, however, since it's GPL. I'd rather stick with a BSD-style license that doesn't place any restrictions on use or distribution (short of copyright notices).

Toni Wilen has added a SANA-II device (uaenet.device) to the latest builds of WinUAE. Very cool, but it currently requires TAP-Win32. I'd rather see an implementation that doesn't require a bridged virtual network adapter. Microsoft Virtual PC does it (of course), and so does nmap. Regardless, cheers to Toni for all the hard work.

Still confused as to why some adapters configure and go online and some don't. For those that go online after being configured by another stack, e.g. Miami, the problem is obviously with my initialization code.

If anyone wants to take a stab at using sanautil (available on Aminet) to configure and bring a card online, please let me know the results. I'm particularly intersted in the output of "sanautil device status" before and after the card is configured and before and after the card is brought online.

Here's the compatibility list as it stands today:

ariadne_ii.device,43.12,no
cnet.device,1.9,yes
hydra.device,?,yes
ioblixether.device,37.14,no
norway.device,?,yes*
openpci_8139.device,1.2b4,yes
prism2.device,?,yes
uaenet.device,1.0 (1.4.5b13),yes**

* after configuration by separate stack
** acquires DHCP address; hangs on first ICMP echo reply

Trev