Faster directory listing possible?

[matthey]

I do have a CyberSCSI MKII, but SCSI drives are noisy and expensive, that's why I'm using a microdrive currently.

You are showing a CyberVision 64 so check that your sys:prefs/Workbench prefs have the gadget "Images in: Other Memory" selected. That with PeterK's icon.library will greatly accelerate icons. The 68060 version of PFS is much faster than alternative file systems for listing as mentioned.

[matthey]

That will speed things up a lot.  Workbench is coded the slowest way possible so it always stores gfx in the slowest memory possible, which is just the wrong way to do things on AGA or on accelerated Amigas.

Workbench is coded in the most compatible way possible with settings and utilities to speed up and enhance the appearance.

There is a special version of PFS3 for 060?  Awesome!

The original PFS3 package uploaded to Aminet has CPU specific versions. Toni's version does not but his version has a few minor bug fixes. The last regular version of PFS3 has been very stable for me though.

[matthey]

If he opens his window with thousands of icons then Workbench will completely thrash the memory list and there will be thousands of memory fragments and the Amiga will be permanently slower from that point on.  If he opens and closes that window multiple times and then runs random software he is in for a crash.  Its like Workbench doesn't use memory pools at all and just bashes the OS with zillionz of malloc()/free().

I snooped the memory allocation functions and it looks like some of the allocations use memory pools and some don't so TLSFmem should provide some speedup. AllocMem() and AllocVec() are probably faster than TLSFmem until the memory becomes fragmented. Some programs really do thrash memory like you say. TLSFmem uses the BFFFO instruction which isn't exactly fast (and slow relative to other instructions on the 68060). Memory pools are another solution for reducing memory fragmentation but I expect they are slower too. A whole pool can be released at once which is an advantage though. It's nice for memory cleanup. Vbcc compiled programs use memory pools for malloc()/free() making the cleanup easy and guarding against memory that did got get deallocated.

Installing matthey's CopyMem from Aminet will speed up your 060 machine a great deal too.

It doesn't appear that Workbench, the workbench.library or the icon.library are using the exec/CopyMem() functions while listing icons in a workbench window. My commodity.library used CopyMem() with a 16 byte size several times while listing though. Any speedup from my CopyMem patch would probably be minor on a fast processor like the 060 or 040 in this case. Every little bit helps though .

[matthey]

Ooh, I wasn't aware you'd done a commodity.library too.  I'll have to grab myself a copy of that!

I haven't. I meant "my" unmodified commodity.library on "my" Amiga. Sorry for the confusion.

What's your opinion on the PatchFor020 tool on Aminet?  Is it worth running over all the executables on an 3.9 install? I have a CSMK2 060@50MHz.

PatchFor020 works well most of the time but is limited in what it patches. It only patches compiler math functions to faster ones. It will patch with 64 bit integer instructions that are trapped on the 68060 making the code slower for the 68060! I've seen a bug in one of the patches where a signed multiply was used when an unsigned should have been. Most of the time this would not cause a problem but could completely fail in rare cases. Make a backup before you apply the patch and test the speed, output and for MuForce/Enforcer hits. There are a lot more 68020 optimizations that can be done than math routines although these offer some of the biggest gains and are big enough that finding and patching them is reasonably safe. Hand optimizing is slow and error prone but has the potential for a much greater speedup and memory savings. Sometimes 2x the speed and 1/2 the size is possible where the compiler wasn't very good.

[matthey]

Alternative approach.. patch workbench icon-hell?

I do have a working lightly modified (mostly vasm peephole optimizations) AmigaOS 3.9 workbench.library at 191168 bytes where the original was 200856 bytes. I've been testing it for a while for problems and haven't found any yet. I thought PeterK might want a working complete disassembly which is easier than patching the executable for his modified workbench.library but he was worried about it being more error prone which it is . I don't want to release another version without his patches and I didn't think the code was too bad.

I'm currently working on trying to improve vbbc. It's much better to have the compiler do the work better to begin with. I also am looking at possibly optimizing some AROS libraries and trying to incorporate them with AmigaOS 3.9.

[matthey]

Um, my advice, if you want to hear it, would be not to bother with attacking the workbench.library performance problems in this manner. The problems which cause Workbench to perform so poorly are due to poor choice of algorithms, data structures and overall architecture.

No doubt. The original reason for a new workbench.library was to fix a few bugs. A simple cleanup and peephole optimizations that saves 10k is just a bonus and not an attempt to fix the AmigaOS 3.9 bloat :/.

Workbench and its partner in crime icon.library are very closely-coupled and share dependencies which are spread through the entire Workbench code. These dependencies are what make it hard to change either part of the couple. Because Workbench and icon.library have to work under modest memory conditions, and with legacy software, there is little room for making larger changes. So this is why necessary architectural changes have rarely been implemented, if at all.

Looking at Workbench through the lens of 68k assembly language, and the optimizations that are possible at this level, will likely give you only a tiny picture of what makes Workbench tick, and how its chief problems look like (or even how these might be addressed). There is likely nothing you could do that will have a measurable effect on Workbench performance, no matter how much effort you put into it.

Have you tried PeterK's icon.library? It gives a remarkable speedup (including low end systems) without changing the interface. There is a lot possible with assembler but it is a lot of work. It shouldn't be necessary if compilers were doing a better job.

[matthey]

Well just for kicks I ran PatchFor020 over itself and amusingly it patched a few operations.

Don't do that! That's probably the code it's using to detect the slow code. If you patch it, it might not find what it's looking for to patch. There are some other types of critical code that should not be patched too. Patching some of the P96 libraries (and probably CGFX) for example will not work because there are a lot of exec/SetFunction() calls. You don't want to patch the 68060.library or SetPatch for obvious reasons also.

Perhaps there will be a PatchFor060 one day?

From me? Probably not. I've thought about making an automated optimizer but the code really needs to be disassembled correctly first to make a lot of changes. I have worked on a new version of ADis which does a pretty good job but needs more works and I've been involved with other projects recently. It would still never be completely safe to do an automated disassemble, patch and assemble but it could be pretty good and give disassembly warnings. It's better to focus on getting compilers to generate better code.

Would this patched scsi.device help for the OP?

http://eab.abime.net/coders-system/67067-open-source-scsi-device.html

Don Aden's a good guy and knows what he's doing. A faster transfer speed should give a speedup. I have a CSMK3 68060@75MHz with a 15k UltraSCSI hard drive that gives a sustained 30MB/s, PFS, PeterK's icon.library and Voodoo 4 and there really isn't much delay on large directory listings B).

[matthey]

I ran software that is very hard on the Amiga's memory system.  It creates megabajilion of memory fragments.  And TLSFmem was massively better than PoolMem.  It uses a new better algorithm.

Compiler? Listing a drawer full of icons doesn't fragment anything like compiling. Vbcc is about the best test. It doesn't allocate much on the stack at all. I was surprised at the shear volume of memory allocations. A log file of only allocations and deallocations from a few seconds is several megabytes and then I found some illegal deallocations that weren't even in the log :/. I think we are on the right track to getting at least some of the problems fixed. PoolWatch, BDebug (now working as a source debugger) and ThoR's MuTools have been great for catching problems that weren't even noticed on other operating systems. Only Amiga makes it possible .

[matthey]

OK, I booted up my A3000 (and WinUAE) and see that I use "FastCache".  You can set the lines and such from the tooltype info.  I tried this with my A1200 and it makes a huge difference (with the real HD as well as the 8GB CF drive).

Now install PFS3 v5.3:

http://aminet.net/disk/misc/PFS3_53.lha

up the buffers to about 300 and install PeterK's latest icon.library from here:

http://eab.abime.net/coders-system/64079-icon-library-46-4-test-versions.html

And then report back whether "FastCache" is faster .