Amiga File Attributes Lost During .lzx & .lha Extraction on Windows?

[BebertBocal-1]

My understanding is that if you extract .lha or .lzx archives on Windows (with WinRAR, unlha.exe, unlzx.exe or some other tool), all the Amiga-specific file attributes that Windows does not understand will be lost. Is this true? For example, as far as I have understood, if you want to install AmigaOS on an Amiga emulated by WinUAE, the installation is unlikely to go properly if you extract the contents of the AmigaOS .lha installation file with your Windows file extraction tools and then proceed with the installation by using the emulated Amiga.

If I'm not mistaken, these file attributes are purely OS-specific in the sense that the Amiga operating system on which you have the files knows how the files are intended to be used. Thus, the attributes are not embedded in the files because if the file attributes were embedded in the very files, you would not lose the file attributes while extracting the Amiga archives on Windows (with Windows tools). Is this true as well? Please do correct me if I'm wrong!

In case all this information is true, I wonder whether there is any way of tackling this problem... And besides, as far as I have understood, it is not only the Amiga-specific file attributes that will be lost but some filenames might get messed up as well (which, I guess, is because Windows simply does not understand all the characters).

If the information above is correct, and the only way of preserving the file attributes while moving the files between different OS:es is to leave the archives intact and extract them only on AmigaOS / Workbench, we have reason to argue that people should never try to preserve their Amiga stuff in any other form than archive files if they intend to move the stuff between different OS:es. With the knowledge that I have, I don't see how anyone could have the time to assign correct attributes to thousands of Amiga files that were originally inside Amiga archives but accidentally extracted on Windows. The way I have understood it, the user should know or find out the attributes of every single file.

[nicholas]

Short answer, yes.

Slightly longer answer, boot WinUAE with the 3.1 Install ADF and a Windows directory mounted as DH0 which contains the lha self extractor and your lha archive that contains your bootable system partition files, open a shell and from RAM: do C:protect DH0:lhasfx.run +RWED, type lhasfx.run, then type lha x dh0:nameoflhaarchive.lha

Eject the ADF and rest winuae et voila.

ps The lha self extractor may not neccessarily be named lhasfx.run, you'll have to check aminet for it.

[ChaosLord]

If the information above is correct, and the only way of preserving the file attributes while moving the files between different OS:es

Yes its a common problem.  I have had to deal with it innumerable times because ppl download my games from Aminet and then extract them to NTFS thus causing them to go bonkers and do berzerk things and break.

Ppl have a misconception that NTFS can do all that the Amiga can do.  This is not true.

But the solution is easy enuff:
1. Never extract Amiga archives to an NTFS partition.
2. Only extract Amiga archives to an Amiga partition.
3. It is dead easy to make an Amiga partition in WinUAE.  You create a .hdf file (hdf = HardDriveFile) of any size you like.  This .hdf file will function as a virtual Amiga hard drive.  Its very fast and preserves all the Amiga file attributes.

You have 3 main choices when creating a hardfile.  You can format it as:
1. PFS3
2. SFS
3. FFS
4. OFS

I personally recommend that you format your hardfiles with PFS3 (with the long filenames option) as it is the fastest and allows filenames of 107 chars.  The maximum parition size of a PFS3 partition is around 104GB.  So I would keep it down to 100GB or less.  A 100GB Amiga hard drive is big enuff for most ppl.

[BebertBocal-1]

Thanks for the prompt replies!

I'll test both approaches once I have received my AmigaOS CD. The .hdf approach might be a good way of preserving all the Amiga stuff -- extracting all the files inside a virtual Amiga hard drive could be the best thing to do to play it safe.

A couple of questions though:

1. In case I do all the file extraction inside a virtual Amiga HD, what file extractors ("unpackers") or file archivers do you recommend either for AmigaOS or for the command-line interface? I have read somewhere that AmigaOS 3.9 has UnArc and nicholas mentioned the lha self extractor.

2. Do all the Amiga extractors/file archivers support the most common archive file formats like .rar, .zip, or .tar? And what about segmented or multi-volume archives? For example, "Data.part01.rar", "Data.part02.rar" and so on. Notice also that some multi-volume archives can take several gigabytes of data, and I'm not sure how well the Amiga can handle them.

I'm asking these questions as I wonder whether I could use the .hdf method all the way without extracting anything with Windows tools.

[ChaosLord]

Thanks for the prompt replies!

I'll test both approaches once I have received my AmigaOS CD. The .hdf approach might be a good way of preserving all the Amiga stuff -- extracting all the files inside a virtual Amiga hard drive could be the best thing to do to play it safe.

Absolutely.

Otherwise you end up with buggy games and apps.

1. In case I do all the file extraction inside a virtual Amiga HD, what file extractors ("unpackers") or file archivers do you recommend either for AmigaOS or for the command-line interface? I have read somewhere that AmigaOS 3.9 has UnArc and nicholas mentioned the lha self extractor.

The only one that works 100% correctly is the lha cli command available from Aminet.

The fancy GUI versions all have bugs and don't unarchive empty dirs correctly.

Other GUI unarchivers mess up the dates.

The CLI version works 100% correctly.

2. Do all the Amiga extractors/file archivers support the most common archive file formats like .rar, .zip, or .tar? And what about segmented or multi-volume archives? For example, "Data.part01.rar", "Data.part02.rar" and so on. Notice also that some multi-volume archives can take several gigabytes of data, and I'm not sure how well the Amiga can handle them.

I'm asking these questions as I wonder whether I could use the .hdf method all the way without extracting anything with Windows tools.

There are probably various GUI unarchivers that do all that you ask.  I am not really familiar with them because of the aforementioned bugs.

When using the original CLI versions:
lha does .lha files
zip (or unzip) does .zip files
etc.

Those are really the only 2 archive types I use on my Amigas.

I think that it might be safe to unzip, unrar, un7zip all those pc formats on the PC to an NTFS partition but I am not 100% sure.

The way I remember it certain characters on the Amiga don't look the same on a Windoze NTFS partition.


I can say for a fact you need to unlha .lha archives "on your Amiga with your Amiga" even if your "Amiga" is a virtual machine running on your W7 pc.  Because I have encountered the problems of not doing so, over and over and over again.

[paul1981]

1. In case I do all the file extraction inside a virtual Amiga HD, what file extractors ("unpackers") or file archivers do you recommend either for AmigaOS or for the command-line interface? I have read somewhere that AmigaOS 3.9 has UnArc and nicholas mentioned the lha self extractor.

My personal favourite is Voodoo-X for archive extraction:

http://aminet.net/package/util/arc/Voodoo-X

I use this on my real Amiga's and WINUAE emulated. Allows virus checking too with xvslibrary. I always virus check everything I've downloaded before actually extracting it. Voodoo-X makes this simpler than ever. Just load the archive, then select the "Check for viruses" from the menu.

Also, with WinUAE, you can create partitions with the "Add Directory" option on your windows drive. This will allow you to perfectly preserve the file attributes/integrity the same as a HDF would do, but only if you use those files under WinUAE. For example, don't copy Amiga files from within Windows. Any copying must be done from your emulated Workbench in WinUAE. The way this works is that extra hidden files are created on your Windows (NTFS for example) drive alongside the real amiga files, and the hidden files are what preserve the amiga file attributes/integrity (the hidden files are created and read by WinUAE).

Or use HDF's, then it saves you the temptation of handling the files in Windows.

[BebertBocal-1]

@ChaosLord & @paul1981

I owe you many thanks!

I think that it might be safe to unzip, unrar, un7zip all those pc formats on the PC to an NTFS partition but I am not 100% sure.

The way I remember it certain characters on the Amiga don't look the same on a Windoze NTFS partition.

Have you tested if these problems are present when Amiga archives are extracted to ext3, ext4 or ext5 partitions that are used for Linux distributions?

Furthermore, if I decide not to extract all the stuff inside a virtual Amiga HD, one workaround could be to find out whether the archives (such as .rar and .zip) that are common for the PC were originally created by using some Windows or Linux tool. For example, my WinRAR is able to find this out -- it can be seen under "Host OS" if you click on the "Info" after opening an archive. All the .lha archives that I have checked with WinRAR seem to have "Host OS: Unknown" and CRC16 instead of CRC32 which suggests that the archives were created on the Amiga. Anyway, I believe I could extract all the archives that were originally created on Windows to my NTFS partition because the file attributes have been lost anyway.

When I just went through some of my Amiga-related files (that I hadn't checked yet) on my NAS, I noticed that many of them were inside several "layers of archives". To give an example of the way they were stored, e.g. "Releases.rar" contained "Releases.part01.rar", "Releases.part02.rar" etc. which, in turn, contained multiple .iso, .lha, and .lzx files. I'm wondering whether it is always safe to extract all the .rar archives (that were, of course, created on Windows) in this case as long as the .iso images and .lha & .lzx archives are kept intact. Notice that all the .rar files that I checked seem to be created on Windows (or at least that's what WinRAR tells me). The only problem is that among these .iso, .lha and .lzx files there were some Amiga executables. However, since the executables are few in number, I could assign attributes to them afterwards...

A couple more important questions though: can any archive files *themselves* contain any Amiga-specific attributes without which the Amiga is not able to successfully extract the archives? I mean the actual archive files, not the files inside them. And how about .iso files?

Moreover, I noticed that nearly all of those .rar archives that I have on my NAS contain lots of different Amiga disk image files. ADF and DMS were the most common but there were ADZ, IPF, HDF, and HDZ as well. At least those archives that merely contain HDF and HDZ files (virtual Amiga HDs) can very well be extracted to an NTFS partition but I'm not entirely sure about the rest of the disk image formats. Of course, I have had gazillions of .adf files on my NTFS partition and at least WinUAE can open them without any problems.

[AAACHIPSET]

i never found that at all  ..i use a program called universal extractor a freeware program ..only  doing lha or lzx ..not configured for rar etc..

[BebertBocal-1]

i never found that at all ..i use a program called universal extractor a freeware program ..only doing lha or lzx ..not configured for rar etc..

Is it the Windows version of Universal Extractor that you use? If yes, then I'm not wholly sure whether you should use that program for extracting .lzx archives. If you look at one of the folders of the program, you should see lots of executables, one of which is "unlzx.exe". I haven't tested the executable with the Universal Extractor GUI but I have used it via Windows Command Prompt (CMD.exe). When I extracted .lzx archives with it by entering "unlzx.exe -x [filename]", I saw several CRC error messages. There's a similar unlzx.exe extractor available on Mind Creations at http://www.mindcreations.com/projects/ports/lzx.php but it seems to produce exactly the same errors. You can test this by trying to extract a file such as "mwi-212.lzx" available on Scene.org. It should give you the following errors:

Extracting "mwi-212/detached/scenes/plane.lobj"... crc bad
Extracting "mwi-212/detached/scenes/fusko/03.lobj"... crc bad
Extracting "mwi-212/detached/scenes/dziwo/01.lobj"... crc bad
Extracting "mwi-212/detached/scenes/chujenka/CHUJENKA.lobj"... crc bad

I also tested the most recent version of the LZX plugin for Total Commander which is an Orthodox File Manager, and it did not produce the same errors. To confirm this, I compared the checksums for the output of both versions of unlzx.exe and the LZX plugin for TotalCMD, and the results were identical except for the same files (as shown above) that gave me the "crc bad" error message:

unlzx.exe from Universal Extractor & unlzx.exe from Mind Creations
3cef9fb9c9f161f799a78fdcd1a8bc881eb787ba ?RIPEMD160*mwi-212\detached\scenes\plane.lobj
111b134583c601ab5dadc8ad3fdd02c7ded1c7e9 ?RIPEMD160*mwi-212\detached\scenes\chujenka\CHUJENKA.lobj
1d0111d0ce8ebdf1d6fa02f7ff1f6b1648f88225 ?RIPEMD160*mwi-212\detached\scenes\dziwo\01.lobj
2a8620f1030d01facfef2349d67d6bd094f7f1c9 ?RIPEMD160*mwi-212\detached\scenes\fusko\03.lobj

TotalCMD LZX Plugin
89b76b23dd91539a02aef2d5b5edc5e1b04c051c ?RIPEMD160*mwi-212\detached\scenes\plane.lobj
4fcfd7be47fb93c03c380ef258e1f0fe862f54fb ?RIPEMD160*mwi-212\detached\scenes\chujenka\CHUJENKA.lobj
d77d8423ed628d34a64c0e7b958ed989ceafeee9 ?RIPEMD160*mwi-212\detached\scenes\dziwo\01.lobj
d702f6aefd9f58f92ed7a57aa2523792c7b7e84f ?RIPEMD160*mwi-212\detached\scenes\fusko\03.lobj

However, I have heard that older versions of the LZX plugin (at least around 2005) may not be reliable. See a discussion about this at http://eab.abime.net/amiga-scene/18695-unarchiving-lha-lzx-files-windows-xp.html for some proof. Besides, Amiga file attributes will be lost with any of these extractors.

[ChaosLord]

Have you tested if these problems are present when Amiga archives are extracted to ext3, ext4 or ext5 partitions that are used for Linux distributions?

Sadly no

I had 2 pcs with linux installed and the HD of course used one of those filesystems.

But I never really got to use it.  As soon as my brother found my pcs with superfancy linux installed he freaked out, formatted the drive and installed WindowsXP.

That is how I came to find endless numbers of bugs related to Windows XP and NTFS and/or WinUAE.

A couple more important questions though: can any archive files *themselves* contain any Amiga-specific attributes without which the Amiga is not able to successfully extract the archives?

Nah.  I don't think so.

But if you get any "ERROR: File is read-protected" then I change my answer because the -R bit did not get set.
The Amiga bits are RWEDHSPA
I don't remember what H does and A shouldn't make any difference for your purposes.
R = file is Readable
W = file is Writable
E = file is Executable
D = file is Deletable
S = file is a Script (I guess Arexx uses this?)
P = file is Pure

If u get any error messsages about that then you know something went wonky.

Moreover, I noticed that nearly all of those .rar archives that I have on my NAS contain lots of different Amiga disk image files. ADF and DMS were the most common but there were ADZ, IPF, HDF, and HDZ as well. At least those archives that merely contain HDF and HDZ files (virtual Amiga HDs) can very well be extracted to an NTFS partition but I'm not entirely sure about the rest of the disk image formats. Of course, I have had gazillions of .adf files on my NTFS partition and at least WinUAE can open them without any problems.

If you have archives within archives then you can extract the outer layers of archiviness on your PC/NTFS and it should work with 99.99% probability.

Its just that final layer when you extract an archive that produces real Amiga files and dirs that you have to take care with.  Do that on your "Amiga" to an Amiga HD.

[LoadWB]

@ChaosLord:

Watch NTFS blow a gasket if you extract a file from an archive with a name of one of Windows' reserved devices, like COM, NUL, etc.  Creating the file isn't usually a problem, but accessing it causes ntfs.sys to crash taking out the whole system with it (BSOD.)

[Matt_H]

@ChaosLord:

Watch NTFS blow a gasket if you extract a file from an archive with a name of one of Windows' reserved devices, like COM, NUL, etc.  Creating the file isn't usually a problem, but accessing it causes ntfs.sys to crash taking out the whole system with it (BSOD.)

AUX, the DOSDriver, is the main culprit in my experience. I think WinUAE has a workaround for it these days, though, in that it adds a prefix to the Windows filesystem filename, but it still presents as AUX on the Amiga side.