Transcripts from dial-up services?

[CdreCommodore]

I've thought about writing some code to parse these text files to reconstruct  the message threads or file archives from these forums, and making them available on a web site.

I did exactly what you describe here for about 350,000 messages posted to the FidoNet HolySmoke echo between 1993 and 2004.

http://www.tdavis.org/holysmoke/

[CdreCommodore]

So what's under the hood of the FidoNet web?

perl (HTML::Mason) and Postgres.  Search functionality is accomplished with swish-e.  (The same arrangement is being used to drive memphisamigagroup.net and puritytest.net as well.)  

The site is barely 500 lines of Mason, though.  The bulk of the work was by far on the front end, establishing the parent/child relationships among the messages and populating the database.  Only about 40% of the messages had the "REPLY:" pseudo-header that indicates the msgid of the parent message.  The rest had to be figured out by comparing texts, e.g. comparing the first-level quotes in one message to the non-quoted parts of all other messages to find that one that most closely matches.

The look was lovingly styled after^W^W^Wbrazenly ripped off from Google Groups, back when it was still usable for searching Usenet.