NetSurf OS3.x Issues

[wawrzon]

that doesnt sound like a proper testing. first of all a number of factors need to be eliminated that may be related to other parts of the system, like network stack. how about storing some webpages locally on hd drive and trying to switch between them. does slowdown also occures? because it might be an amitcp bug, or a big in interaction with amitcp, but not particularly netsurf bug.

i still have an impression it is a memory allocation issue, and tha is what i would be obseving, but im a noob.

[wawrzon]

OK I can't test this with 3 different Network stacks. Maybe I could rip some sites, so that I could test without TCP stack. I have already tested this with plain html pages, from CU Amiga cds, no slow down. By the  way, this site doesn't have slow down at all http://www.amigahistory.co.uk/ it is also plain html, no css etc.

simply test with pages saved locally. complex or not.
we ae entering the territory where amiga software has been tested very little in praxis. huge files, frequents allocations, it might be and probably is, kernel libs implementation issue, more general tha just a browser. it might be also an issue when compiling on native 68k..

[wawrzon]

it may well be a fundamental 68k problem, or an exec issue. amiga and atari machines werent built with huge multiple megabytes binaries in mind, frequently allocating and deallocating small bits of memeory. i mean to have observed similar issues when for instance trying to compile something larger with cubic ide. with ixemul the problem likely has been masked by using its own linux memory allocation mechanism. on faster machines or uae the inefficiency might be masked by the pure computing power, or jit, even though os4, aros and morphos exec may differ from the genuine one in this respect.

some general profiling would be neccessary, based on a simple testcase, which may result in need to rewrite parts of the system itself to get rid of this bottleneck.

[wawrzon]

OK, this is more like I would expect - everything running consistently at a slower speed.

I know exactly what is causing the slowdown now.  clib2 uses memory pools with a puddle size and expected allocation size of 4K.  I modified that in the newer build to use normal memory allocations instead.

What is happening, is that early memory allocations are fast and efficiently allocated in 4K chunks.  Then, when bits of memory is de-allocated it leaves holes.  When new memory blocks are allocated it - and this is where I'm not sure of the implementation details in the OS - is trying to fill in the gaps in the already-allocated pools?  With a lot of pools it may be taking some time to search through and find a gap of the correct size, which is similar to how normal memory allocations work when searching through all of RAM (and thus a similar speed).

Quite simply, we are allocating and de-allocating so much memory that we quickly lose any advantage of memory pools.

To fix it... well, that's tricky.  The correct way would be to pool together elements of the same size to avoid fragmentation, but I can't do that in the core and all libraries without re-writing all the memory allocations (which would definitely not be popular).  Note I already do this in the frontend everywhere it is practical (this was one of my earlier OS3 optimisation attempts!)

It may simply be a case of making the memory pools bigger, and I will try that first.

its pretty much as what i imagine to happen. okay, i would expect the system should take care of it, but perhaps its woth to try if it can be tweaked in the application code.

[wawrzon]

I suspect that this may not make much of a difference. The memory pools, which is what the malloc()/alloca()/realloc()/free() functions in clib2 are built upon, were intended to avoid fragmenting main memory. This is accomplished by having all allocations smaller than the preset puddle size draw from a puddle that still has enough room left for it to fit. Fragmentation happens inside that puddle.

The problems begin when the degree of fragmentation inside these puddles becomes so high that the only recourse is to allocate more puddles and allocate memory from that. The number of puddles in use increases over time, and when you try to allocate more memory, the operating system has to first find a puddle that still has room and then try to make the allocation work. Both these operations take more time the more puddles are in play, and the higher the fragmentation within these puddles is. Allocating memory will scale poorly, and what goes for allocations also goes for deallocations.

The other problem is with memory allocations whose length exceeds the puddle size. These allocations will be drawn from main memory rather than from the puddles. This will likely increase main memory fragmentation somewhat, but the same problems that exist with the puddles apply to main memory, too: searching for a chunk to draw the allocation from takes time, and the same goes when deallocating that chunk. There's an additional burden on this procedure because the memory pool has to keep track of that "larger than puddle size" allocation, too.

Because all the memory chunk/puddle, etc. allocations and deallocations use the humble doubly-linked Exec list as its fundamental data structure, the amount of time spent finding the right memory chunk, and putting the fragments back together, scales poorly. Does this sound familiar?

From the clib2 side I'm afraid that the library can only leverage what the operating system provides, and that is not well-suited for applications which have to juggle large number of allocated memory fragments.

Question is what size of memory chunk is common for NetSurf, how many chunks are in play, how large they are. If you have not yet implemented it, you might want to add a memory allocation debugging layer and collect statistics for it over time.

It may be worth investigating how the NetSurf memory allocations could be handled by an application-specific, custom memory allocator that sits on top of what malloc()/alloca()/realloc()/free() can provide and which should offer better scalability.

thanks for confirmation olaf. this is ecxactly what i had in mind. however designing application specific memory allocator oer allocation method is probably not the right way..

[wawrzon]

It doesn't provide much noticeable speed-up (most of the heavy processing happens in the libraries which were already built with optimisations on)

as mentined also, the compiler optimizations usually are a bit overrated, there isnt that much to expect of it, maybe 20-30% at most.

[wawrzon]

@chris:
if reaction/classsact classes is all your frontend needs i can test it under aros. for instance aweb woked there. but im not sure if this is helpful, since we get additional unknown varibles. i dont have a clean 3.5 or 3.9 setup at hand and have no time to set one up, unfortunatelly. however testing without being able to build locally may be not helpful again. and im not sure when i have time to ste up netsurf build system. hmm. maybe ill see if it runs at all.

[wawrzon]

tested with locally saved page?

if not: all kinds of things can happen online at random to make testing like this of no value. such additional factors need to be eliminated

[wawrzon]

Ixemul has better memory allocator but ixemul.library is not safe from application using native Amiga API.

i have never understood whats not safe about it..
howener libnix results are nit that bad in comparison. one simply needs to stay away from clib2.

@bernd
you still here?

[wawrzon]

ixemul programs are in many cases not very responsive to keyboard input. especially on real hardware.

[wawrzon]

seems netsurf expects v44 version of reaction classes. is this a necessary requirement, or could this check be lowered in order to check with v41 class act ones?

[wawrzon]

I'd have to check for sure which features I use, but I suspect not. It needs v44 of other components anyway.

mmm. ok.. simply tried to run it on aros with the freely available classes. but it doesnt load there even with the proper reaction components, which probably expect 3.9 functionality. on my 3.9(+) (i dont have a clean install anymore) it loads properly (have not installed it) and runs for a time being loading simpler sites (images rendering still seems not to work well even with 32 bit) but crashes rather soon. im not able to make any pattern of it at this time, however.

edit: (i gave it a lot of stack of course, which is necessary for it to load from command line anyway, so this is not an issue)

might be a good idea to have a version with full debug compiled in for testing.

edit2: btw. im using the version from aminet, which probably is not the current one.

[wawrzon]

netsurf must be heavily trashing the memory all around. it takes down the whole uae with it when it fails. i might try to recall how ro run muforce or uae enforcer in order to catch these hits. but in order to localize the offending function the binary would have to be compiled with debug symbols anyway. the question is if gccfindhit would work with the utilized compiler, whichever it is. however probably the most sensible would be to enable all debug first. it doesnt make sense to try to speedup and optimize a buggy binary.

[wawrzon]

Really?  I've not seen that here.

have you browsed from site to site or simply loaded a site just once to verify the browser works? a reproducable way to crash netsurf and uae was here to navigate to google.com, enter amiga as search phrase and click a link to amiga wikipedia article, afair.

gccfindhit doesn't work, so I have no way of tracking down Enforcer hits beyond somebody who knows what they are doing disassembling the code.

I'm basically doing what I can with my limited resources, and that doesn't include spending hours tracking down a memory bug I have no hope of finding except by chance.  I suspect it's something like the MinList bug which I would never have found without matthey's help.

maybe its due to my setup, not being clean 3.9, but it isnt crashy otherwise,so i must assume its netsurf this time. what concerns the debug techniques thats unfortunate, since it makes it almost impossible to somehow contribute useful feedback to this project. dont you have at least debug statements spread in the code you could simply turn on in order to find the crash location at least roughly? i know i could look at the sources myself;..
how about others? dont they experience stability issues? because if so, it would be better to go for stability first and then for performance.

[wawrzon]

just tried the last development version. images seem to display all right on rtg 32bit. the crash no matter uae version and cpu setting remains the same. but so far not confirmed by others i assume its my setup fault.

[wawrzon]

NetSurf -v will produce a log.  That's useful for indicating where it crashes but doesn't necessarily pinpoint the exact location.

ah, i recall. i have forgotten that. thats good, might give some hint;)