Is there an LZX-like archiver for the PC???

[Glaucus]

Greets Amiga fans!

Back in 1995 Jonathan Forbes and Tomi Poutanen released a new Amiga-only archiving tool called LZX. LZX had, at the time, a unique compression feature they called "file merging". Here's how they describe file merging in the official documentation for LZX:

File merging is a feature of LZX which enables compression to be increased, often very significantly, by allowing data from one file to be compressed using the knowledge of previous files in the archive.  This feature, unique to LZX on the Amiga, often improves compression by 300% or more!

This feature is very useful when compressing text files and source files, where there is often a large amount of text common to many files.

There is also a significant advantage to this feature when compressing a large number of small files, since the data overhead of re-starting compression for each file is now removed.  This is particularly true when compressing ".info" files

My question here is, are there file archiving utilities available for the PC (or other platforms) that employ this feature? My understanding is that zip (or WinZip) does not support such a feature, but I could be wrong here. Doing some quick googling around the net I noticed that WinZip 10 supports PPMd compression, but most docs I've found for it are Russian. Unfortunately for me, I can't read Russian. Does anyone know anything about this compression standard?

As for why I'm asking about this? Ummm... Not really sure, but a good buddy of mine has tasked me with figuring out the answer to this question - for reasons not yet clear to me. :-D If anyone could help, it would be greatly appreciated.

  - Mike

[Piru]

a unique compression feature they called "file merging".

Unique... yeah right.

tar cf stuff.tar some files here
your-favorite-compress-app stuff.tar


tar was introduced in UNIX, however this method of grouping files before compression has been in use from at least 60s.

See Wiki: File archiver

[justthatgood]

Maybe Unlzx

http://www.bebits.com/app/901

http://www.mindcreations.com/projects/ports/lzx.php

[Piru]

@justthatgood

Maybe reading the post?

[Glaucus]

I suppose I should have mentioned that their use of the phrase "file merging" is quite different then what seems to be the norm. If you read their definition of "file merging" you'll quickly note that it's quite different from what tar does. In the LZX sense, "file merging" will compress identical files better then lha or zip, by magically merging the data before compressing it.

For example, if you have ten .info file, which are binary identical, and add them all to a zip file, WinZip will compress each file independently, and add them to the archive. LZX would note that the ten files are all the same and use advanced compression techniques to minimize the duplication of compressed data. This makes the compressed file size much smaller.

So I think we're talking about something slightly different then what tar does.

  - Mike

[Piru]

@Glaucus

So I think we're talking about something slightly different then what tar does.

We are not. The only difference is that 'LZX' does the merging inside the program itself, whereas UNIX tar doesn't itself compress the stream, but leaves it to external program.

The only advantage of the LZX method is that the program "knows" how to skip data when it want to decompress single file somewhere in the middle of the stream. Tar + external archiving app doesn't know this, and the whole tar file needs to be decompressed until the specific file is encountered.

tar + external app is great because it's flexible. Once better archiving applications arrive the archiver can be easily replaced (for example tar + gz vs tar + bzip2). LZX is forever tied to the compression algorithm it has built-in.

[Glaucus]

Ah, okay, I see what you're saying now.

Yes, I should have realized this right away. It's been a while since I've used a unix-like system, but I have used tar & gzip together in the past.

But yeah, LZX did have that advantage of allowing you to do things in one step instead of two. That's a good point.

So, are there any other options out there that perform similarly to LZX?

  - Mike

[Piru]

I noticed that WinZip 10 supports PPMd compression, but most docs I've found for it are Russian. Unfortunately for me, I can't read Russian. Does anyone know anything about this compression standard?

http://www.codeproject.com/cpp/ppmd.asp
http://www.winzip.com/ppmd_info.htm
http://datacompression.info/PPM.shtml

Paper here: PPM: one step to practicality (pdf)

[orange]

(win)rar, solid packing, everyone uses it
(win)ace, too, slightly better because it has 'checkpoints' but its rarely used
lzx for pc is cab

[Piru]

@orange

lzx for pc is cab

CAB uses two different algorithms, arithmetic coding and the LZX algorithm (in addition to MSZIP). The actual LZX archive format is not used in CAB (well for obvious reasons).

cabextract documentation has some more details.

[orange]

well he was looking for  'LZX-like'..

anyway, I reccommend using RAR, its VERY common, there are some better packing archivers but too few people are using them

here's a useful link: The Art Of Lossless Data Compression

[Piru]

@orange

Agreed, (win)rar is a good example.

[Glaucus]

Okay, thanks guys!

  - Mike

[Speelgoedmannetje]

Maybe you can use a modern Amiga zipping utility (wich can make windows/dos executables) and run winUAE :-)

[Homer]

Don't quite understand what you are looking for, but I use on the PC "Alzip" (http://www.altools.net) which is free and can do alz, ace, arc, arj, b64, bh, bhx, bin (hex files), bin (CD images), bz2, cab, ear, enc, gz, ha, ice, iso, jar, lcd, lha, lzh, mim, pak, rar, tar, tgz, uue, uu, war, xxe, z, zip, zoo, 7z, and 001files. I think it does even more than that but you'll have to have a look. Just a little something I discovered while working in South Korea :-)