Faster directory listing possible?

[olsen]

The way I figured it, if Workbench used its own pool and PeterK's icon.library used its own pool then that could double the speed and/or reduce memory fragmentation by 10.  Or something like that

No, Workbench does not use memory pools. It pulls everything from the globally available memory, which leads to fragmentation.

I assume this is why PeterK's icon.library is so much faster (+ all the other optimizations he did.)

I don't know how it works, but there is only so much that can be done, given how Workbench reads icons. What breaks the camel's back is that icons are stored in individual files which must be found first and read quickly. Workbench tries to do that by scanning directory contents and switching back and forth between scanning and reading individual icons, which causes disk accesses to jump around wildly.

If this process could be sped up, it would have to be smarter about when to read the icons. For example, if you could cache the icons to be read from a directory then this would reduce the amount of time which Workbench spends on switching between scanning the directory and reading the icons.

Let's say you had an icon cache. If you could tell Workbench to hold off reading the icons until it has finished scanning the directory, then you could cut down the overall time spent by a significant amount. Of course, once Workbench has read the directory contents, reading the icons must be very quick. I think all of this would be doable.

[JimDrew]

OK, I booted up my A3000 (and WinUAE) and see that I use "FastCache".  You can set the lines and such from the tooltype info.  I tried this with my A1200 and it makes a huge difference (with the real HD as well as the 8GB CF drive).

[matthey]

OK, I booted up my A3000 (and WinUAE) and see that I use "FastCache".  You can set the lines and such from the tooltype info.  I tried this with my A1200 and it makes a huge difference (with the real HD as well as the 8GB CF drive).

Now install PFS3 v5.3:

http://aminet.net/disk/misc/PFS3_53.lha

up the buffers to about 300 and install PeterK's latest icon.library from here:

http://eab.abime.net/coders-system/64079-icon-library-46-4-test-versions.html

And then report back whether "FastCache" is faster .

[ChaosLord]

I always "just assumed" that the reason PFS3 was so fast was because it essentially had a secret built-in copy of FastCache.

Is that not true in some sense?

Everyone has told me that PFS3 uses a large chunk of RAM that FFS does not use.   All that extra ram is used to cache stuff, right?  Or ?

Does anyone know if FastCache works with FFS when I use 4K or 8K or 16K sectors?

Ppl always complain about FFS being slow but if you increase the sector size to 4K or larger you get a giant speedboost.  No caching software needed.  If you then add caching software you should then be around the same speed as SFS or PFS.

[freqmax]

I think PFS is speedy because it simply is smart about things, or rather its author.

[olsen]

I always "just assumed" that the reason PFS3 was so fast was because it essentially had a secret built-in copy of FastCache.

Is that not true in some sense?

Caching is helpful, but it is not essential. A cache will boost performance only if what you need to access next is already stored in the cache. This is the case with data which is accessed repeatedly, but often enough data on the disk is accessed once and then never again.

To provide for a consistent speed improvement you need to organize the file system's data structures in such a way as make accessing the data fast. For example, instead of having to walk through a series of data blocks before you can use the data you are looking for, you could have a tree data structure, which would let you pinpoint within 3-4 steps what would take 15-30 data blocks to walk through in sequence.

The FFS consistently uses data structures which are very simplistic in construction, and in theory ought to be robust. But the robustness is just not there, and the simplicity only succeeds in dropping performance the more data has to be managed.

Ppl always complain about FFS being slow but if you increase the sector size to 4K or larger you get a giant speedboost.  No caching software needed.  If you then add caching software you should then be around the same speed as SFS or PFS.

Changing the block size (a block is made up of several sectors) will have a positive impact on performance only if your disk drive and the controller hardware do not make data accesses noticeably slower if you end up reading 4-8 times the previous amount of data.

You still have the same scalability issues with the FFS, which means that the more data you have to manage, the quicker FFS loses performance. Not all file systems lose performance as quickly as the FFS, as the amount of data increases. It's particularly nasty with the FFS, though.

There is one major drawback to increasing the block size with the FFS: if the file system data structures become corrupted, then the amount of data affected by the corruption will be much higher than with smaller block sizes. Given that the FFS lacks robustness by design, trading greater speed for even smaller robustness may be a bad idea...

[danwood]

I've got my old 3.9 bb2 installation running again. It's on a 8GB microdrive that has multiple partitions.
One of the partitions contains a LOT of folders with WHDLoad games, each folder has a game in it. When I open this partition in workbench it takes a few minutes before the listing shows. In the meantime I can't do anything else.
This partition is either in FFS of SFS, I don't quite remember, but it takes so long. Is there a way to speed up the directory listing?
I remember from older installations there was a .fastdir file that contained the listing? Never seen that on 3.9 though.

Sorry if someone's already mentioned this (not read the full thread) but why not use a WHDLoad front-end rather than launching them all from Workbench?

I just have the excellent, and very simple to use iGame in my dock and load all the games from that in a few seconds:

http://winterland.no-ip.org/igame/

[nicholas]

There is this launcher too.

I've not used either myself though.

http://www.jimneray.com/xbench.html

[mfilos]

X-Bench is really awesome. I use it on my A600 and it rox as iGame is really nice but not the best in speed on plain 030's.

[JimDrew]

Now install PFS3 v5.3:

http://aminet.net/disk/misc/PFS3_53.lha

up the buffers to about 300 and install PeterK's latest icon.library from here:

http://eab.abime.net/coders-system/64079-icon-library-46-4-test-versions.html

And then report back whether "FastCache" is faster .

Ok, what are we using to bench mark this with?  I think FastCache is faster - I am using an A3000 w/40MHz 040 w/80MB of RAM and an A1200 with 50MHz 030 w/256MB of RAM.  The A1200 has the stock hard drive and a 8GB CF drive.

[ChaosLord]

Ok, what are we using to bench mark this with?  I think FastCache is faster - I am using an A3000 w/40MHz 040 w/80MB of RAM and an A1200 with 50MHz 030 w/256MB of RAM.  The A1200 has the stock hard drive and a 8GB CF drive.

All I can say is this reviewer used DiskSpeed 4.2 in July 1993 to race FastCache vs. PowerCache.  So if you use that exact version of that program then you can compare your own results to those of the reviewer.
http://de4.aminet.net/docs/rview/FastCache.txt

In 1997 both FastCache and PowerCache were updated:
FastCache 1.1 1997 http://aminet.net/package/disk/cache/fcache11

And if you really want to figure out the best one there is also HyperCache Professional (commercial) and Dynamicache (commercial).

[djkoelkast]

The new icon.library is in fact *a lot* faster, in stead of minutes it "only" takes 30 seconds now. Only thing I did was change the icon.library, because images to other memory was already set.
It's still a bit of a wait, but not by far as long as before, this is managable.
If only I could connect 2x Microdrive to the thing. I can't

[ChaosLord]

Install FastCache and see if you can get it down to 10 secs.  You'll hafta make sure to set the settings to use FASTram and use a reasonably large cache.

[paul1981]

Of course I used to use PoolMem too!  From the day it first came out I was using PoolMem and it was awesome!

But then TLSFmem came out which is 10x better than PoolMem.

I ran software that is very hard on the Amiga's memory system.  It creates megabajilion of memory fragments.  And TLSFmem was massively better than PoolMem.  It uses a new better algorithm.

You can always comment out PoolMem with a ; and then add in TLSFmem in place of it in your startup-sequence to see what I mean.  If u don't like it you can always switch back.

Yep, I tried TLSFmem a couple of years ago, and I have a newish version on my hard drive which I downloaded earlier this year. I did try it, but I couldn't tell whether it was any better than PoolMem or not. But if everyone on here says it is, then I'll have to take your word for it. The whole world can't be wrong. I may install it for good yet... LOL.

Thanks Chaos...

[stefcep2]

Install FastCache and see if you can get it down to 10 secs.  You'll hafta make sure to set the settings to use FASTram and use a reasonably large cache.

How?

[ChaosLord]

Yep, I tried TLSFmem a couple of years ago, and I have a newish version on my hard drive which I downloaded earlier this year. I did try it, but I couldn't tell whether it was any better than PoolMem or not. But if everyone on here says it is, then I'll have to take your word for it. The whole world can't be wrong. I may install it for good yet... LOL.

Thanks Chaos...

I still have my PoolMem in my startup-sequence too.  But it is commented out.  I have not uncommented it in years.

I am a software developer and there is some sort of hardcore debugging tool that I sometimes have to run (I can't remember which one... its been years since I did something silly like code a bug .  Anyway this debugging tool hacks into the AmigaOS memory list system.  Only TLSFmem doesn't USE that system which is why it is so much faster and less fraggly.  So when running that hardcore debug tool I hafta switch back to PoolMem temporarily.

So like I say, ur not getting married to TLSFmem, or if u r then u can still cheat on the side with PoolMem when u get the urge.