Amiga.org
Amiga News and Community Announcements => Amiga News and Community Announcements => Amiga Software News => Topic started by: chris on November 20, 2016, 02:47:08 PM
-
NetSurf is a small fast web browser for AmigaOS and other platforms.
NetSurf 3.6 is primarily a bugfix release. It contains several fixes and clean-ups to front end code, as well as supercookie blocking, and certificate chain handling improvements. We recommend all users upgrade to NetSurf 3.6.
Download AmigaOS 4 version (http://download.netsurf-browser.org/netsurf/releases/pre-built/amiga/netsurf-3.6.lha)
There is also a preview/beta of NetSurf 3.6 for AmigaOS 3.5/9. Assistance is requested to bring this up to the standard of the OS4 build.
Download AmigaOS 3.5/9 version (http://aminet.net/package/comm/www/netsurf_os3) (pending upload approval)
-
Congratulations for solving "slow down effect" of OS3.5/9 version :)
-
Congratulations for solving "slow down effect" of OS3.5/9 version :)
Olaf Barthel's work, not mine :)
-
Any change for optimisations to get lost speed back?
-
Any change for optimisations to get lost speed back?
It's the price of eliminating memory fragmentation, I guess.
I believe that's as fast as it can be, any optimisations will be ones in the browser.
-
It's the price of eliminating memory fragmentation, I guess.
I think that this warrants a thorough analysis. This the first time the new memory management system has been used, and we barely know enough about its real-life performance, let alone the constraints under which it operates.
I do not know what causes this slowdown, and I did not expect it either. Allocating and releasing memory should require (roughly) the same amount of time, regardless of the size of the allocation (as long as it can be covered by the allocator's page size). There must be more to that.
I believe that's as fast as it can be, any optimisations will be ones in the browser.
The new memory management (it's called a "slab allocator") is good at keeping fragmentation at bay. But we have barely begun testing it yet, it's hardly even a week old to begin with ;)
There is plenty of room for "tuning" the new memory management. Tuning requires that data is collected on how the memory management is used, for later analysis. To this end there is an interface in the clib2 runtime library which NetSurf uses. It would be nice to allow for memory usage information to be collected through that interface, to be stored in log files.
-
I do not know what causes this slowdown, and I did not expect it either. Allocating and releasing memory should require (roughly) the same amount of time, regardless of the size of the allocation (as long as it can be covered by the allocator's page size). There must be more to that.
@utri007 How much slower is it?
There is plenty of room for "tuning" the new memory management. Tuning requires that data is collected on how the memory management is used, for later analysis. To this end there is an interface in the clib2 runtime library which NetSurf uses. It would be nice to allow for memory usage information to be collected through that interface, to be stored in log files.
You can send SLABSTATS to NetSurf's ARexx port to get it to dump them to the log.
NetSurf -V ram:ns.log
rx "address netsurf 'slabstats'"
I did find with the log enabled NetSurf was crashing, but I don't think that's related to the stats collection. I might see about modifying this so the stats are stored in an ARexx stem variable.
This isn't in the 3.6 build as I added it later, I'll need to upload a new test version but I can't do that today.
I did notice that we get thousands of allocations of 32 bytes, and not much above 2K. I reduced the slab size to 2K (again, after this 3.6 build) and the timings don't look much different. v3.6 has the slab set at 8K.
-
You can send SLABSTATS to NetSurf's ARexx port to get it to dump them to the log.
NetSurf -V ram:ns.log
rx "address netsurf 'slabstats'"
Great! I'd like to see samples of these log files, if possible.
I did find with the log enabled NetSurf was crashing, but I don't think that's related to the stats collection. I might see about modifying this so the stats are stored in an ARexx stem variable.
I think log files should be stored in a format which makes automated analysis easy, e.g. JSON. I'll work something out.
This isn't in the 3.6 build as I added it later, I'll need to upload a new test version but I can't do that today.
I did notice that we get thousands of allocations of 32 bytes, and not much above 2K. I reduced the slab size to 2K (again, after this 3.6 build) and the timings don't look much different. v3.6 has the slab set at 8K.
Thousands of 32 byte allocations translates into hundreds of slabs. Worst case, allocation requires the traversal of the entire slab list to find that a new slab needs to be added. Deallocation likewise would (worst case) require a traversal of the entire list.
I think I ought to beef up the (not terribly sophisticated) slab statistics some more, so that the number of slab list items checked before an allocation/deallocation is performed is counted.
As for the list traversal effort, that says "hash table" to me :)
-
@utri007 How much slower is it?
Amiga.org is now 46-48 seconds first load. Going to forums and back to FrontPage takes 26-28 seconds each time. First pages were previously fastest 20-22 seconds.
Previously there wasn't that big difference when site is cached or not.
I'm happy to help, I can try this.
NetSurf -V ram:ns.log rx "address netsurf 'slabstats'"
-
Great! I'd like to see samples of these log files, if possible.
I'll copy the file I have somewhere useful this evening.
Thousands of 32 byte allocations translates into hundreds of slabs. Worst case, allocation requires the traversal of the entire slab list to find that a new slab needs to be added. Deallocation likewise would (worst case) require a traversal of the entire list.
I was getting something like 800 slabs of 8K in total after visiting amiga.org.
I think I ought to beef up the (not terribly sophisticated) slab statistics some more, so that the number of slab list items checked before an allocation/deallocation is performed is counted.
Some sort of breakdown of size of allocations above the slab size might be useful too.
-
I'll copy the file I have somewhere useful this evening.
Thank you! Please send it to me via e-mail.
I was getting something like 800 slabs of 8K in total after visiting amiga.org.
I'd say that this number is far too high. Which chunk sizes are most frequently-used? Scaling the respective slab size for these might be helpful and might work better in the long term than adding hash tables to speed up allocation/deallocation.
Some sort of breakdown of size of allocations above the slab size might be useful too.
Yes, this information is completely obscure right now. I'll have to change the data structure which tracks these allocations so that it no longer relies upon AllocVec(). The size information should become part of the management data structure.
-
Wau, thank you very much!
Netsurf is the browser I use most on my Sam440.
-
Amiga.org is now 46-48 seconds first load. Going to forums and back to FrontPage takes 26-28 seconds each time. First pages were previously fastest 20-22 seconds.
Interesting, my version is now 2x faster on first load and 6x faster on reload on Amiga.org.
41s first load and 66s reload on old allocator. 22 seconds and 11s on new one.
Amigaworld.net 11 and 15 versus 9 and 8.
-
@olsen
Has the slab allocator from OS4 been back-ported to 3.x? Is that what you are testing?
-
Not bad! Haven't had a chance to play around with this for a while, I forgot what my settings were and just threw it on there, along with a copy of that guigfx.library it looks like it now needs.
- 1st load of amiga.org 35 seconds, reload in 21 seconds
- Wikipedia
- CNN
- Google News
Attached a few screenshots.
I do wish that the minimize gadget worked, lol. ;)
After I took these screenshots I tried playing around with some of the preferences. Nice that they work, I got a few visits from the guru but eventually was able to get it to open up on its own screen instead of of Workbench. Very nice! :banana:
-
@olsen
Has the slab allocator from OS4 been back-ported to 3.x? Is that what you are testing?
No, this is something very different and very much simpler in design. I was suprised that it worked so well after having spent barely a week building and testing it.
Both the AmigaOS4 slab allocator and the one which is part of clib2 (available at https://sourceforge.net/projects/clib2/ and https://github.com/adtools/clib2) are based upon the same 1994/2001 papers written by Jeff Bonwick, who introduced the concept. For reference I looked at the Linux and FreeBSD slab allocators, but then found that the slab allocator used by "memcached" was easiest to understand for me in the context described by the papers.
The papers which I read in addition to the two by Jeff Bonwick strongly supported the idea that using a slab allocator at the application level, e.g. linked against the NetSurf code, made good sense. It was not necessary to have the slab allocator in the operating system to get useful results.
A slab allocator at the operating system level needs to cater for caching requirements, it has to provide for proper data alignment, and it has to watch very carefully how much management overhead it deploys to herd the slabs. A slab allocator at the application level, linked against NetSurf, can ignore some of those constraints, making it simpler in design.
That's what landed in clib2, and I am still working on making it perform better. One week of design and implementation clearly is not sufficient ;)
-
Nice. Thanks for the background information.
Had you considered TLSF? I recall a while ago there was some mud-slinging as to which was "better".
-
Nice. Thanks for the background information.
Had you considered TLSF?
No, I stopped searching for an alternative solution when the slab allocator appeared to do the job.
I recall a while ago there was some mud-slinging as to which was "better".
I'll have another look at TLSF. From what I could learn quickly about TLSF, it is a technically more complex approach than what a slab allocator could get away with. The management overhead (accounting data structures) appears to be much lower, and the smallest usable fragment size is also lower than what my own slab allocator allows for.
However, much of that overhead comes about because clib2 now supports three different back-ends for allocating and managing memory. The management data structures are not strictly needed for the memory pool and slab allocator back-ends, so there is quite some room for improvement in clib2.
I doubt that I will be replacing the slab allocator in clib2 with TLSF any time soon. Instead, I'll work on improving the clib2 memory management system.
-
@gregthecanuck
it seems tlsf is clearly better at keeping the performance up while allocating and reallocating small memory bits, but it isnt perfectly compatible with legacy amiga software. which shouldnt be an issue as long as it is linked statically as part of a standard c library.
@olsen:
according to wikipedia, the behaviour we are experiencing with slab, may be a known handicap, sorry its only on a german page:
https://de.wikipedia.org/wiki/Slab_allocator
down the page:
Ablauf einer Speicheranforderung[Bearbeiten | Quelltext bearbeiten]
Aus dem bisherigen ergibt sich der typische Ablauf einer Speicheranforderung:
1.Es wird versucht, ein Objekt aus dem per-CPU-Cache zu entnehmen.
2.Schlägt dies fehl, wird ein Objekt aus einem bereits bestehenden Slab geliefert.
3.Falls auch keine freien Objekte mehr in den Slabs vorhanden sind, muss ein neuer Slab angelegt werden, mit dem Umweg über das übergeordnete Buddy-System.
Der negative Einfluss auf die Caches und somit auf die Performance steigt dabei von Punkt zu Punkt.
the mentioned buddy allocator is afaik the default one in bernds ixemul library >6x.x, which may explain, why it performs well in comparison. maybe some more considerations should be spent on that issue?
-
I doubt that I will be replacing the slab allocator in clib2 with TLSF any time soon. Instead, I'll work on improving the clib2 memory management system.
TLSF didn't help with Netsurf.
-
according to wikipedia, the behaviour we are experiencing with slab, may be a known handicap, sorry its only on a german page:
https://de.wikipedia.org/wiki/Slab_allocator
The issues mentioned in this context appear to refer to the kernel implementation of the slab allocator. The kernel slab allocator should care about proper alignment of allocations, so as to avoid friction with multiple processors and non-uniform memory access.
The slab allocator in clib2 sidesteps these issues by mostly ignoring them. Unless I made a mistake, allocations are currently aligned to 64 bit word boundaries because of the chunk allocation granularity (this may change, though). No optimizations for multiprocessing or NUMA are needed.
the mentioned buddy allocator is afaik the default one in bernds ixemul library >6x.x, which may explain, why it performs well in comparison. maybe some more considerations should be spent on that issue?
From what I know buddy allocators are much more complex in operation than the slab allocator. For example, the highly configurable dlmalloc allocator, including documentation comments, is more than 5000 lines long.
In a buddy allocator effort is spent on merging chunks, and depending upon the order in which allocations are being made, buddies may not be released in the order which allows them to be merged, slowly increasing fragmentation over time. dlmalloc is designed to make best-fit allocations, as opposed to first-fit, which contributes to the complexity and the effort spent (first-fit is fast at the expense of quickly increasing fragmentation over time).
By comparison, a slab allocator can deliver both best-fit and first-fit performance at the same time without spending any effort on merging chunks. Furthermore, it can deliver this in nearly constant time, i.e. O(1), discounting that it has to obtain the pages which it manages from somewhere ;)
-
TLSF didn't help with Netsurf.
Why did it fail?
-
Why did it fail?
I don't know but it didn't help. There was tlsfmen.lha in aminet, wich is now deleted some reason. It was started from shell anytime according to the guide.
-
I don't know but it didn't help. There was tlsfmen.lha in aminet, wich is now deleted some reason. It was started from shell anytime according to the guide.
I use TLSFMem on all my Amiga's. It was created by Chris Hodges up to version 1.6 then patched to 1.9 by Cosmos, AFAIR. It was pulled when Chris yanked all the rest of his Amiga software, for whatever (@#*$%&(#*$%& reason. :( It is a fantastic program for reducing memory fragmentation, I can't say how it reacts with NetSurf, specifically, but for just general day-to-day Amiga tasks - it's always the very first thing in my Startup-Sequence. PM me your email address if you'd like me to fwd you a copy.
-
I use TLSFMem on all my Amiga's. It was created by Chris Hodges up to version 1.6 then patched to 1.9 by Cosmos, AFAIR. It was pulled when Chris yanked all the rest of his Amiga software, for whatever (@#*$%&(#*$%& reason. :( It is a fantastic program for reducing memory fragmentation, I can't say how it reacts with NetSurf, specifically, but for just general day-to-day Amiga tasks - it's always the very first thing in my Startup-Sequence. PM me your email address if you'd like me to fwd you a copy.
HAH :D Then you can say how it reacted to Netsurf. You noticed "slow down effect" youself and you were running TLSF men all the time?
-
HAH :D Then you can say how it reacted to Netsurf. You noticed "slow down effect" youself and you were running TLSF men all the time?
In that case, yes. I haven't had nearly enough coffee yet, today. What were we talking about again? I just tried the newest version last night after being away from it for a couple months, it seemed to work well. A big improvement. :)
-
In that case, yes. I haven't had nearly enough coffee yet, today. What were we talking about again? I just tried the newest version last night after being away from it for a couple months, it seemed to work well. A big improvement. :)
Slow down effect is gone, it was actual "show stopper". :) Now browser is actually usefull. Now it loads first page longer than it used to, but lets see if it gets faster.
-
I don't know but it didn't help. There was tlsfmen.lha in aminet, wich is now deleted some reason. It was started from shell anytime according to the guide.
Ah, so none of the existing TLSF implementations were linked against NetSurf at any time? I'd say it would not be too difficult to adapt them to work with AmigaOS.
Patching the exec memory management API at runtime would not be quite the same as replacing the memory management for a single application. Once your exec patches have to work for all running programs, it becomes so much harder to diagnose problems and their causes. Your chances of discovery are much better if you can limit the scope.
-
Slow down effect is gone, it was actual "show stopper". :) Now browser is actually usefull. Now it loads first page longer than it used to, but lets see if it gets faster.
I'm working on it, or rather, working on the slab allocator in clib2.
The next set of changes is already checked in. The goal is to speed up allocation and deallocation operations so that the number of "slabs" already in use no longer affects the time spent performing these operations.
Chris sent me a log file which detailed that at times there were more than 800 "slabs" in play. I expect that this number must have had a significant negative effect on overall performance.
Let's see how today's changes fare. Unfortunately, the changes currently result in higher memory usage figures. But I think I have a solution for that problem, too.
-
@olsen
i dont remember very well, as its a longer while ago now, but tlsf by chris hodges and subsequently patched by cosmos are replacements for the genuine system allocator (which may have caused problems with legacy apps) rather than statically linked against an application for internal use. thats pretty different to what you are trying to achieve, so i doubt the mentioned results are relevant, but also im not sure if the other participants understand the difference.
-
Ah, so none of the existing TLSF implementations were linked against NetSurf at any time? I'd say it would not be too difficult to adapt them to work with AmigaOS.
Patching the exec memory management API at runtime would not be quite the same as replacing the memory management for a single application. Once your exec patches have to work for all running programs, it becomes so much harder to diagnose problems and their causes. Your chances of discovery are much better if you can limit the scope.
im afraid the amiga tlsf source isnt openly available at this time. i guess it should, by license requirements? maybe contact chris directly? im convinced, cosmos contribution was asm based..
-
The major problem with all these approaches, including TSLFMem is that it most are not compatible to the (unfortunately) documented exec memory administration structure, which is the simple stupid chunk based format. You'll find it in exec/memory.h, and unfortunately programs may - and some actually do - depend on this particular structure. Replacing the organization of the exec memory system is therefore a bad idea.
You can do two things, however: First, *avoid* fragmentation where it is created by replacing or updating system components that cause them, and improve the allocation strategy of your own application as well. Recycle structures, avoid malloc()/free() (AllocMem()/FreeMem()) if you can re-use the memory yourself. "Recycle, do not through away".
The prime source in the operating system (layers.library) is already gone. The next source are graphics Regions and RegionRectangles, unfortunately this is much harder to fix in a conservative backwards-compatible way.
The second thing you can do - and probably should do - is to link your own programs with their own implementation of a memory manager that only pulls big chunks from exec, and then maintains them itself. Even if these are only the stupid exec memory pools, this might already help keeping the rest of the system happy.
-
Just for the fun of it: PoolMem has a bare-bone concept related to "Slabs", though it is called a little different there - it keeps a "scratch list" of recently released objects up to a maximal size. Hence, if you release an object of < 256 bytes, it will be linked into the scratch list (a very simple pointer adjustment), and if you allocate an object of such size, the scratch list will be visited first (again a simple pointer adjustment). Once in a while, the scratch list is revisited and "compacted".
This can be made compatible to the exec memory pools, so there is no danger of compatibility problems. Of course, programs that handle memory incorrectly (e.g. by writing into memory that has already been released) are likely to cause problems. "scratch list" memory is much more likely to be touched any time sooner, a fact that caused a couple of crashes of software following such bad programming practise.
-
Hats off to everyone in this thread. It is nice to see a civil discussion. :)
-
The next source are graphics Regions and RegionRectangles, unfortunately this is much harder to fix in a conservative backwards-compatible way.
Make graphics.library allocate this stuff in such a way that their addresses are multiple of 2, but not multiple of 4 (ie. address last byte is 0x2, 0x6, 0xA or 0xE). This way it should be possible to detect if it was allocated by itself or by someone evil outside.
-
Of course, programs that handle memory incorrectly (e.g. by writing into memory that has already been released) are likely to cause problems.
For the first 8 bytes of freed memory this is not even save in old memory system, if freed memory region was exactly in front of another allocated memory region, because then it becomes the start of a new "free memory chunk": 8 bytes used as node of free memory list.
-
@ chris
thanks for the update! after years of not using my classics on the net I hope to try this update out on my A4000 soon once I get her all back into her case
-
Post-3.6 version with newly updated clib2: http://cy2.uk/netsurfos3
Let me know if it is faster or not!
-
I made a quick test, when I had my lunchbreak.
Congratulations again to all involved. :) It is much faster, amiga.org downloads now 28-30 seconds with my 68060 / 66mhz. Was 46-48 seconds. Still not that fast than it used to be20-22 seconds, but with memory fragemention problem it wasn't usefull.
I also tested it with my 68040 / 40mhz. It might use little bit more ram? 2-3mb more, when amiga.org is loaded? But still useable with 32mb ram. Amiga.org loads with this system 44 seconds.
-
Chris, is there any *known* issue with CyberGrafX based OS 3.9 systems?
After a quick test yesterday I found that on my Picasso96/Picasso II equipped Amiga Netsurf worked as expected (Picasso screen with 16 Bit/800x600 px) while my CyberGrafX/CV64-3D Amiga always rendered images with 8 Bit - even if the screen hat 16 or 24 Bit depth.
-
I made a quick test, when I had my lunchbreak.
Congratulations again to all involved. :) It is much faster, amiga.org downloads now 28-30 seconds with my 68060 / 66mhz. Was 46-48 seconds. Still not that fast than it used to be20-22 seconds, but with memory fragemention problem it wasn't usefull.
Does the page load time improve if you immediately reload the page (assuming that it is not simply reloaded from the disk cache, but re-fetched from the web site)?
If so, then this may be significant. The "slab allocator" is referred to as a cache in the paper which introduced the concept. For a cache to be helpful it needs to already contain the data that is subsequently drawn from it ;) Otherwise, the cache first needs to be organized and "primed". This process is sometimes called "warming up".
For the "slab allocator" to warm up it needs to allocate memory for the slabs which it will manage, chopping the slabs into (sometimes) hundreds of chunks. If you start this process cold, the "slab allocator" will first spend a lot of effort allocating memory and getting it ready for use.
This process could be sped up by preallocating slabs when NetSurf starts. However, it won't do just to allocate a bunch of slabs, one also needs to know which memory chunk sizes would be most appropriate.
-
Chris, is there any *known* issue with CyberGrafX based OS 3.9 systems?
After a quick test yesterday I found that on my Picasso96/Picasso II equipped Amiga Netsurf worked as expected (Picasso screen with 16 Bit/800x600 px) while my CyberGrafX/CV64-3D Amiga always rendered images with 8 Bit - even if the screen hat 16 or 24 Bit depth.
None that I know of. I'd be surprised if guigfx didn't recognise CGX and work correctly, I have *no* special handling of RTG screens on OS3 - as far as NetSurf is concerned it's just a normal palette-mapped screen.
edit I do get guigfx to stick images in an 8-bit bitmap though, no matter the screen depth, so maybe that's the problem? It should be 8-bits per image though so shouldn't be too noticeable, and this applies to P96 too, so I'd expect to see the same effect on both.
-
Does the page load time improve if you immediately reload the page (assuming that it is not simply reloaded from the disk cache, but re-fetched from the web site)?
To help with this, holding Shift whilst clicking Reload ought to re-fetch it from the network.
-
Does the page load time improve if you immediately reload the page (assuming that it is not simply reloaded from the disk cache, but re-fetched from the web site)?
Sorry, what I did was amiga.org - > forums - > amiga.org - > forums -> amiga.org
At least that way load time didn't improve. Load time did vary, few seconds, but I didn't notice any pattern. But as I said, test was very short.
I can't do some more testing saturday, today I'm going to get drunk. :)
-
Some little tests on my Amiga1200 with 1230/50 accelerator:
Netsurf took - with best configuration - 35,6 seconds to load amiga-news.de using the version before, now takes 31,8 seconds. Jumping to aminet.net (35 seconds) and back to amiga-news.de (31,1 seconds). Holding shift and reload it, Netsurf took 30,8 seconds. So I guess if I reload it ten times I can break the 20 seconds mark...:)
In any case: Thanks for the new version!
By the way: 90 seconds for amiga.org. And it looks quite bloated. The font is much to big.
-
Thanks. Amiga.org is diffult site to render.
Amigaworld.net 17 seconds.
-
Some little tests on my Amiga1200 with 1230/50 accelerator:
Netsurf took - with best configuration - 35,6 seconds to load amiga-news.de using the version before, now takes 31,8 seconds. Jumping to aminet.net (35 seconds) and back to amiga-news.de (31,1 seconds). Holding shift and reload it, Netsurf took 30,8 seconds. So I guess if I reload it ten times I can break the 20 seconds mark...:)
If you keep it at, at some point you'll be able to see web pages before you have decided to visit them, which leads to the possibility of being able to read next week's lottery numbers ;)
But seriously, the longer you use the web browser, the better the new memory management system will adapt. When you switch to a new web page, NetSurf has to break up the memory allocated for all the old page's components, then reuse it for the new page.
Imagine smashing up all your crockery after dinner, then glueing it all back together for the next meal. The fragments will become smaller and smaller over time, making it harder to find those which fit together well enough. Larger fragments need to be broken up into smaller pieces to make them fit with the rest.
That's exactly the problem which NetSurf has. The new memory management system helps here by sorting the fragments into bins from which they can be retrieved more easily without having to smash larger fragments. The longer you use it, the better the new memory management system will know which bin sizes are needed the most, and the more readily-available fragments it will keep at hand.
-
How it will perform "low memory" situations? I was surfing with my A1200, 68040 and 32mb ram system after Netsurf is loaded there is about 11mb free ram. Ram was eated up quite fast, if surfing to other sites ie. amiga.org - > amigaworld.net -> wikipedia.org. There is no problem is just surfing one site, ie. amiga.org -> forums -> thread -> amiga.org
-
How it will perform "low memory" situations? I was surfing with my A1200, 68040 and 32mb ram system after Netsurf is loaded there is about 11mb free ram. Ram was eated up quite fast, if surfing to other sites ie. amiga.org - > amigaworld.net -> wikipedia.org. There is no problem is just surfing one site, ie. amiga.org -> forums -> thread -> amiga.org
How much memory is being consumed needs to be measurable. To this end I just added new code to clib2 which produces machine-readable status information in JSON format. I hope that Chris will add support for it in NetSurf, so that snapshots of this status information could be saved and submitted for analysis.
-
Chris: Would it be easy thing to fix iconify button?
-
@olsen
Added, but I don't think it is working properly: http://www.cy2.uk/tmp/ns-stats.json.gz
@utri007
I can fix it by removing it. I didn't think there was an Iconify button anyway!
-
@utri007
I can fix it by removing it. I didn't think there was an Iconify button anyway!
Aah! Don't do that! ;)
There definitely is a button (you can see it in all my screenshots) and it definitely doesn't work. Having a program open in full screen, on the Workbench screen, covering everything else, with no easy way to make it smaller... Sure would be nice if Iconify worked. ;)
-
Aah! Don't do that! ;)
There definitely is a button (you can see it in all my screenshots) and it definitely doesn't work. Having a program open in full screen, on the Workbench screen, covering everything else, with no easy way to make it smaller... Sure would be nice if Iconify worked. ;)
Resize and Zoom will work. Iconify uses a new feature of OS4, and it's a PITA to change.
-
@olsen
Added, but I don't think it is working properly: http://www.cy2.uk/tmp/ns-stats.json.gz
Something isn't right. The strftime() appears to produce no output whatsoever in the archive, and the vsnprintf() function doesn't even convert %zu output correctly.
The code which prepares the JSON data for output, one line at a time, uses the clib2 vsnprintf() function. There are two tests for it in both the library and the "slab-test.c" program, and they both worked.
Does NetSurf replace vsnprintf() or strftime()? That's my only guess. The output in the archive went completely off the rails...
-
Aah! Don't do that! ;)
There definitely is a button (you can see it in all my screenshots) and it definitely doesn't work. Having a program open in full screen, on the Workbench screen, covering everything else, with no easy way to make it smaller... Sure would be nice if Iconify worked. ;)
Do you mean that you can't do this (sse picture) ;)
Iconify would be nice, but I ques remove button would be OK at least as an temporarily solution.
-
Something isn't right. The strftime() appears to produce no output whatsoever in the archive, and the vsnprintf() function doesn't even convert %zu output correctly.
The code which prepares the JSON data for output, one line at a time, uses the clib2 vsnprintf() function. There are two tests for it in both the library and the "slab-test.c" program, and they both worked.
Does NetSurf replace vsnprintf() or strftime()? That's my only guess. The output in the archive went completely off the rails...
Ah, looks like strftime gets replaced. Before I change this, do you know if this comment is valid?
/* Although these platforms might have strftime or strptime they
* appear not to support the time_t seconds format specifier.
*/
(edit although there's a define to tell NetSurf not to use strftime, it doesn't seem to define strftime anywhere, so I'm not sure this will make a difference anyway)
-
@chris
I have a problem and no idea what is wrong. I downloaded NetSurf and I get the the this (pic attached) every time I launch it. Any ideas what is wrong??
Thanks
-
@chris
I have a problem and no idea what is wrong. I downloaded NetSurf and I get the the this (pic attached) every time I launch it. Any ideas what is wrong??
Thanks
Firstly, you are running the OS4 version on OS4, yes?
Secondly, I'll need the crashlog. Just the "stack trace" section, it's short so post it here.
-
Yes Sir. AmigaOS4.1FE on an X1000. Crash log to big 31.5 kb limit on site is 19.5kb
Thanks
-
I reread your question. Here is the stack trace
Stack trace:
[frontends/amiga/gui.c:5356] ami_gui_splash_open()+0x21c (section 1 @ 0x1773D8)
[frontends/amiga/gui.c:5352] ami_gui_splash_open()+0x214 (section 1 @ 0x1773D0)
native kernel module newlib.library.kmod+0x000020a4
native kernel module newlib.library.kmod+0x00002d54
native kernel module newlib.library.kmod+0x00002ee8
_start()+0x170 (section 1 @ 0x16C)
native kernel module dos.library.kmod+0x00024c18
native kernel module kernel+0x00042654
native kernel module kernel+0x000426d4
Thanks
-
I reread your question. Here is the stack trace
OK, that's a new one. What I think has happened is the splash window has failed to open, and then crashed because I've not checked that it is open.
There's very little reason for that to fail. Lack of memory? (unlikely). Even if it can't open the image (Resources/splash.png - check it in Multiview), the window should still appear.
-
Ah, looks like strftime gets replaced. Before I change this, do you know if this comment is valid?
/* Although these platforms might have strftime or strptime they
* appear not to support the time_t seconds format specifier.
*/
Yes, this is still valid. clib2 does not support the strftime() "%s" conversion specifier. "%s" appears to be a Unix addition, and it is not part of the C99 specs. I could easily add it, though, if it is needed.
Looking at the source code of NetSurf 3.6, it appears that there is a workaround in place which performs the conversion with snprintf(). Makes you wonder why strftime() is used in the first place ;)
As for strptime(), this is not part of the C99 specs either, and quite complex to implement. This seems to be a Unix addition, too. Again, NetSurf has various workarounds for strptime() in place which do the job just fine.
NetSurf neither replaces vsnprintf() nor strftime(), as far as I can tell. There goes my theory as to what interferes with the JSON data generation. I'm really puzzled how this could go so spectacularly awry... Could you check the linker map? It should show where the vsnprintf() and strftime() code comes from, exactly.
That said, it's not difficult to generate the same data using the __get_slab_allocations() and __get_slab_usage() API functions in clib2 if you wanted to implement it yourself. I just thought I'd save you the effort and put it into __get_slab_stats().
-
@olsen
I'm afraid the map file means nothing to me, but it's here if you want to decode it: http://www.cy2.uk/tmp/map.txt.gz
@apsturk
Please try build 3788 or newer from http://ci.netsurf-browser.org/builds/amiga/
It'll probably just crash later on (I have fixed the crash you're seeing), but I'd like you to run it with:
netsurf -v *>ram:ns.log
Then email me the log file.
-
@olsen
I'm afraid the map file means nothing to me, but it's here if you want to decode it: http://www.cy2.uk/tmp/map.txt.gz
Thank you, it's not that cryptic, it's just too long ;)
Here are the "interesting" bits (for given values of "interesting"):
-- 8< --
.text 0x0056a724 0x90 /opt/netsurf/m68k-unknown-amigaos/cross/lib/gcc/m68k-unknown-amigaos/3.4.6/../../../../m68k-unknown-amigaos/lib/libm.a()
0x0056a724 _vsnprintf
.text 0x005886e0 0xe98 /opt/netsurf/m68k-unknown-amigaos/cross/lib/gcc/m68k-unknown-amigaos/3.4.6/../../../../m68k-unknown-amigaos/lib/libc.a(time_strftime.o)
0x00589408 _strftime
-- 8< --
This says that both vsnprintf() and strftime() come from the libm.a and libc.a linker libraries, which are part of clib2, etc. It's odd that the reference to libm.a does not list the source file name (when did you last rebuild all these libraries from scratch?).
Anyhow, these library functions are not getting overridden by NetSurf, libcurl, libssl, libgcc, etc. code.
Which collapses another theory of mine.
How does the code look like which fails to produce the correct JSON data? Have you already checked it in?
-
Do you mean that you can't do this (see picture) ;)
Well sure I can. Just with older versions (and their very slow redraw/refresh rate), clicking & dragging to resize the window took something like two minutes and would often lead to the program crashing entirely.
Just tested, it does work a bit better now. ;)
-
How does the code look like which fails to produce the correct JSON data? Have you already checked it in?
It's checked in, here it is:
static int ami_memory_slab_stats_cb(void *user_data, const char *line, size_t line_length)
{
BPTR fh = (BPTR)user_data;
long err = FPuts(fh, line);
if(err != 0) {
return -1;
} else {
return 0;
}
}
fh = Open(fname, MODE_NEWFILE);
__get_slab_stats(fh, ami_memory_slab_stats_cb);
Close(fh);
-
Well I reinstalled it into a folder and it works. The installer just put stuff everywhere in my work petition and I grabbed everything I thought was with it into a folder but missed some the were not seen in icon view. It works and it is nice :)
Thank you
-
It's checked in, here it is:
static int ami_memory_slab_stats_cb(void *user_data, const char *line, size_t line_length)
{
BPTR fh = (BPTR)user_data;
long err = FPuts(fh, line);
if(err != 0) {
return -1;
} else {
return 0;
}
}
fh = Open(fname, MODE_NEWFILE);
__get_slab_stats(fh, ami_memory_slab_stats_cb);
Close(fh);
Thank you, this looks like it should.
This suggest that the problem may be with the clib2 vsnprintf() code. I only tested the plain libc version (no floating point support), whereas NetSurf uses the libm version (with floating point support). I'll have another go at reproducing the problem.
One last thing: does the behaviour change in any way if you replace Open()/FPuts()/Close() with fopen()/fputs()/fclose(), or store the file on a different volume (i.e. don't save to RAM:, save to a hard disk drive)?
-
One last thing: does the behaviour change in any way if you replace Open()/FPuts()/Close() with fopen()/fputs()/fclose(), or store the file on a different volume (i.e. don't save to RAM:, save to a hard disk drive)?
I just tried saving to RAM: (previously I saved to HD) and the result is the same :(
-
I made some benchmarking an results were weird? First there is no memory issue, or it is very minimal.
Test was made just refreshing amigaworld.net FrontPage. 060 and 040 tells CPU, next numbers are secons. Kb free ram is how much my 040/32mb ram system had free fast ram.
First test was amigaworld.net, no big diffrences.
CPU / SEC 060 17 14,7 15,0 15,3 15,6 14,6 14,8
CPU / SEC 040 21,9 19,9 18,7 18,9 19,5 19,1 19,2
kb free ram 3921 3882 3864 3888
Next test was amiga.org, here I got weird results. After some time my 040 is faster or equal with my 060. So slab allocator gives huge benefit to 68040 cpu, but not for 68060 cpu? After first load rendering time is less than half?
Test was made refreshing amiga.org FrontPage and last two sec values are amiga.org forums and amigaworld. Last two kb values are free ram amiga.org forums and amigaworl.net.
amiga.org
CPU / SEC 060 30,9 24,3 23,8 22,9 25,8 23,5 22,0 27,8 15.1
CPU / SEC 040 65,1 25,7 26,3 22,4 24,5 24,5 24,2 34,5 19,3
kb free ram 945 927 873 907 966 1257
Both Amigas has has Orinoco Silver Wlan card and Genesis TCP/IP stack. No reboot between tests.
-
Reproduced test so that my 4G router/wlan access point was near of Amigas, results were same. There is also a pattern, 5th refresh is slower than previous and 6th faster again.
How can slab allocator be so much faster with 68040 than 68060?
-
I made some benchmarking an results were weird? First there is no memory issue, or it is very minimal.
Test was made just refreshing amigaworld.net FrontPage. 060 and 040 tells CPU, next numbers are secons. Kb free ram is how much my 040/32mb ram system had free fast ram.
First test was amigaworld.net, no big diffrences.
CPU / SEC 060 17 14,7 15,0 15,3 15,6 14,6 14,8
CPU / SEC 040 21,9 19,9 18,7 18,9 19,5 19,1 19,2
kb free ram 3921 3882 3864 3888
Next test was amiga.org, here I got weird results. After some time my 040 is faster or equal with my 060. So slab allocator gives huge benefit to 68040 cpu, but not for 68060 cpu? After first load rendering time is less than half?
Test was made refreshing amiga.org FrontPage and last two sec values are amiga.org forums and amigaworld. Last two kb values are free ram amiga.org forums and amigaworl.net.
amiga.org
CPU / SEC 060 30,9 24,3 23,8 22,9 25,8 23,5 22,0 27,8 15.1
CPU / SEC 040 65,1 25,7 26,3 22,4 24,5 24,5 24,2 34,5 19,3
kb free ram 945 927 873 907 966 1257
Both Amigas has has Orinoco Silver Wlan card and Genesis TCP/IP stack. No reboot between tests.
060 clocked at ? I get 9,2/5,9 for amigaworld and 13,4/7,5 for amiga.org with my vampire, it seems alot faster than a 060 with netsurf
-
060 clocked at ? I get 9,2/5,9 for amigaworld and 13,4/7,5 for amiga.org with my vampire, it seems alot faster than a 060 with netsurf
Vampire is 3.6x faster than my 060
-
060 clocked at ? I get 9,2/5,9 for amigaworld and 13,4/7,5 for amiga.org with my vampire, it seems alot faster than a 060 with netsurf
Oh, is this working on the Vampire now? What was the problem?
-
Oh, is this working on the Vampire now? What was the problem?
is this even the same vesion of netsurf? maybe they are using artis sdl port?
@utri
im not sure if your benchmarking bears any actual meaning. there is so much factors the speed you measure may depend upon, the actual internet rates, the likely varying contents of the pages you are downloading and therefore the possibly varying acces times. above all you dont even use a cable connestion but wlan.. who even cares what stack or interface you use? id test against a locally saved page to at least eliminate few of these unknown variables. but i have said that before..
-
Double checked with wired lan. 68040 benefits more slab allocator than 68060. After twice as slow first load, it halves a download / rendering time, so that is actually same speed than 68060 or even faster. With same logic with 68060 second load of amiga.org should take less than 15 seconds. (now about 25 sec).
id test against a locally saved page to at least eliminate few of these unknown variables. but i have said that before..
My internet connection is 20/20mb. Key is repeating tests, to see that every test gives about same results. I don't try to make science of this.
-
I've updated the build to latest HEAD.
After running NetSurf twice I'm getting a recoverable alert 010000009 - memory freed twice.
Previously everything would just freeze, so that's sort of an improvement.
Need to track this down.
-
Tested quikly and seems that border problems and gui refresh problem is now gone??
Now my 68040 machine "officially" renders amiga.org faster than my 68060 machine.
Amiga.org is rendered 29 seconds with 68040 :), and 31 seconds with 68060. Reloading page and both renders it 23 seconds. Restarted browser and amiga.org was loaded 27 seonds with 68040 and 29 seconds with 68060.
Amigaworld.net is still loaded faster with 68060 15 seconds, and 68040 20seconds.
I haven't had problems restarting browser,except that first version wich has working java.
Restarted browser some 6-7 times, no problems with it.
-
Tested quikly and seems that border problems and gui refresh problem is now gone??
I fixed the SimpleRefresh refreshing. The tree windows are new (rewritten) and now support SimpleRefresh too.
I don't know what you mean by "border problems".
-
I don't know what you mean by "border problems".
When resizing or / and movin Netsurf, window borders used to disapper bottom / left / right.
-
New test build: http://cy2.uk/netsurfos3
This time I'm specifically interested in performance and oddities related to secure websites (https), compared to the previous version (http://www.cy2.uk/tmp/netsurf_os3_oldssl.lha if you need to download to compare)
btw this isn't directly related to the AmiSSL v4 update :) actually it should be faster :)
-
Thanks again :)
-
If I try to go https://google.fi it just says can't fetch.
If I try to answer to this thread wit Netsurf it gives assertion error and can't find utf8
Is it so that it tries to use true type font, maybe speficied true type font used on this site?
I don't notice any difference in performance generally.
-
New test build: http://cy2.uk/netsurfos3
This time I'm specifically interested in performance and oddities related to secure websites (https), compared to the previous version (http://www.cy2.uk/tmp/netsurf_os3_oldssl.lha if you need to download to compare)
Did you used openssl in this version ? :confused:
-
If I try to go https://google.fi it just says can't fetch.
Well I can, but it seems google search isn't redirecting to a secure page (uses http by default).
-
gmail doesn't work right. Can type username, but can't type in password.
-
If I try to go https://google.fi it just says can't fetch.
Works here.
You can try https://www.howsmyssl.com if you want to check SSL is working.
If I try to answer to this thread wit Netsurf it gives assertion error and can't find utf8
Is it so that it tries to use true type font, maybe speficied true type font used on this site?
That's because my bitmap font routines are buggy.
I don't notice any difference in performance generally.
General performance should be the same, only fetching of secure pages will have improved. Wikipedia seems faster here, I know previously that was taking an insane amount of time although newer versions have been better in that regard anyway - but this version appears to be twice as fast on Wikipedia than the previous one.
-
Did you used openssl in this version ? :confused:
I've always been using OpenSSL. The difference is this version contains some previously undiscovered 68k optimisations. ;)
gmail doesn't work right. Can type username, but can't type in password.
That's normal (another bitmap font bug). It is typing, you just can't see that it is typing.
Incidentally GMail is a bit of a faf to get into, you need to log in (it'll complain about the password but will log you in) and then go back to mail.google.com and go to the basic HTML view. I managed to log in here using those steps!
-
Well, another bug. If you unclick the bitmap fonts in the prefs, netsurf goes scanning for fonts. Crashes after a while.
But in the end (when reloading), gmail did work with simple html layout.
-
Works fine so far.
But like previous versions, Netsurf "crashes" after quitting it with 80000004 code (OS3.9, Amiga1200)
-
But like previous versions, Netsurf "crashes" after quitting it with 80000004 code (OS3.9, Amiga1200)
I'd love to fix this. Every time I think I've found the cause it turns out I haven't.
-
I've always been using OpenSSL. The difference is this version contains some previously undiscovered 68k optimisations. ;)
As in already in the OpenSSL source code or some code you've written yourself? Just curious :)
Why not switch to using AmiSSL? I realise this wasn't an option in recent years, but there are many benefits to using AmiSSL instead of OpenSSL directly.
-
As in already in the OpenSSL source code or some code you've written yourself? Just curious :)
None of the above (http://git.netsurf-browser.org/toolchains.git/commit/?h=chris/openssl-68k&id=c71bb2fb9237de8dbba5f2b8bb6ffe56010310f5) ;)
Why not switch to using AmiSSL? I realise this wasn't an option in recent years, but there are many benefits to using AmiSSL instead of OpenSSL directly.
The SSL stuff is in the core and/or in libcurl, so it isn't practical to switch to AmiSSL.
-
I'd love to fix this. Every time I think I've found the cause it turns out I haven't.
Just tried your version from April 16. Started two times, quitted successfully - without an error - one time....
-
I haven't ever had that quit crash.