Welcome, Guest. Please login or register.

Author Topic: Layers.library V45 on the aminet  (Read 128832 times)

Description:

0 Members and 4 Guests are viewing this topic.

Offline olsen

Re: Layers.library V45 on the aminet
« Reply #194 on: September 14, 2014, 05:10:29 PM »
Quote from: modrobert;772976
Isn't the original graphics.library written in C?

All versions (1.x through 3.x) are written in a mix of 'C' and 68k assembly language, strongly leaning towards 68k assembly code. Looking at the API, you'll find that a large part of the operations are about filling in data structures, hooking them up and then eventually telling the hardware what to do with them. Typically, the "clerical operations" are written in 'C', and the part which talks to the hardware is written in assembly language.

graphics.library, like intuition.library, is one of the more complex operating system components, both in how it uses data structures and how many distinct operations it can carry out. Hence, you'll find that 'C' plays a major part in its implementation.

That said, when you need to talk to the hardware in a very specific manner, you couldn't do that quite so elegantly back in 1986 if you didn't use 68k assembly language. There is complex assembly language code in graphics.library, it even uses its own preprocessor (!) in order to simplify writing loops (while .. do) and control structures (if .. then .. else).

Quote from: modrobert;772976
You can't expect a C compiler to match the optimizations made manually by an assembler programmer. I've heard so many stories about how well C compilers optimize these days and how assembler is made redundant, and then when you actually disassemble the code the compiler produced, it's bloated, filled with jump tables, and inefficient code.
If you have specific requirements for carrying out operations, you may get better control through the use of assembly language than any modern 'C' compiler could provide you with. C11 has just gained new control keywords in order to make interfacing to hardware more straightforward, but it will take a while for the language to evolve to give you the kind of control only assembly language can give you.

As for assembly language becoming largely redundant, it's probably unavoidable. 'C' in particular is a more expressive language which encodes in fewer lines what assembly language, by its very nature, requires much more effort to express. The thing is, if you can say the same thing with fewer words, the chances of mucking it up are somewhat reduced. If you have the right language, you can even verify the correctness of the instructions and data structures you used, which is something that eludes assembly language by its very design.

As for inefficient code, performance nowadays does not necessarily come out of implementing an algorithm through the optimum low level language encoding you could choose (that being assembly language). You can't necessarily predict how your code will be executed, and if you can, you might have to run the gauntlet of observing a handful of operating conditions under which it executes, such as regarding optimum pipeline use, prefetch, etc. This can get so ugly that it has to be automated (look up how one programmed the Motorola 56000 DSP in its day, just to get an idea how weird this can get) just to let the programmer worry about implementing his design correctly.

This is how progress looks like: the gains made through building faster processors are used in such a way that writing more complex software, more secure software and more reliable software becomes an easier goal to achieve through the use of tools and code generation which does not necessarily attempt to squeeze the last cycle out of the machine. This is, in the end a trade-off, if not a sacrifice.
« Last Edit: September 14, 2014, 05:30:38 PM by olsen »
 

Offline olsen

Re: Layers.library V45 on the aminet
« Reply #195 on: September 14, 2014, 05:29:30 PM »
Quote from: Cosmos;772972
Have you watching my changes into my code ? Certainly no...

For example, the function R_AndRegionRegion in the graphics.library use two AllocMem (and of course two FreeMem at the end) for two tiny buffers (only $C) : I replaced by six clr.l -(sp) for these buffers on the stack now...

This function is now more than 1000 times faster, so the improvement is real...


You are a professionnal troller, you talk in the void, you have lost all credibility...



:(

Oh dear, the two of you do not even speak of the same thing when you discuss optimizations.

Replacing a memory allocation, which is guaranteed to be more expensive than substituting stack operations, certainly is an optimization. But the actual gains are likely to be entirely absorbed by the respective functions in graphics.library/intuition.library, etc. calling layers.library.

From what I have learned, there is only so much you can achieve at this level of optimization. You are more likely to get exponentially higher improvements if you perform the optimization at the next higher level, through the use of different data structures and algorithms, which is exactly what Thomas did with his layers.library implementation.

For this reason, I remain doubtful if a 1000-fold improvement is possible at this level. As the saying goes, extraordinary claims requite extraordinary explanations and proof to back them up.
 

guest11527

  • Guest
Re: Layers.library V45 on the aminet
« Reply #196 on: September 14, 2014, 05:35:51 PM »
Quote from: modrobert;772976
Isn't the original graphics.library written in C?

It's a mix of both, depending on the level of the function. The low-level stuff is in assembly, the higher and more complex operations are in C. Even then, the assembler stuff depends on some macros to provide at least some orientation on the program flow, i.e. the authors have used macros for conditions, loops and so on. The result is not quite as scary as "raw" assembler.

This specific function, and the region functions in particular, are in assembly. It's part of the low-level slice&dice operations that are commonly used between layers and graphics. Ideally, regions (like other layer-like structures) should be pooled, and hence NewRegion() should first try to take the region memory from a graphics specific pool, and recycle regions whenever possible.  

Unfortunately, it's not what it does. If you want such pooling, PoolMem is there to help (it keeps memory pools for small fragments like struct Region, hence *may* help here).

Even more unfortunately, struct Region is documented, so *in principle* an appplication could create such a structure by AllocMem(), bypassing NewRegion() (bad idea!) or on the stack (also a bad idea!) Even then, such a change should be synchronized between graphics, layers, and code parts that re-implementent parts of layers & graphics (P96, cybergraphics, EGS...). Thus, in general, it is not such a good idea to change this lighthearted. It requires a pretty complete code review to understand the possible implications, and potentially a re-compile of graphics, layers, P96 and cybergraphics (in worst case).

One way or another, this example shows a constructional weakness of the whole AmigaOs: Too many internals have been documented, and the API is not always clear. Ideally, all "primitive objects" should have been left undocumented, or should have been equipped with clear constructor/destructor calls. struct Region is one of the "better" implementations by using such an API (partially), but the situation is much worse with other graphics primitivities, such as struct RastPort, struct Bitmap (big outch!), struct ViewPort (bigger outch!) or struct View. To avoid the many problems, V37 graphics jumps in circles and stores private data on the ViewPort not in the view port, but in the color map (even bigger outch!) loaded in the view port just to be able to extend it, and uses hoops such as GfxAssociate to be able to backwards-extend the gfx engine.

Gfx is a clear case of "how not to design a library". Intuition much better code from a software engineering perspective (there are clear structures and a relatively clean API). I guess this is all because gfx was written in a rush, for a deadline that was too close to allow a good implementation, and pretty much in an ad-hoc way.

Then again, all the region low-level calls should, instead of calling NewRegion() internally through gfx, go through the LVO instead. They don't do that, but probably should. Another outch.
 

Offline biggun

  • Sr. Member
  • ****
  • Join Date: Apr 2006
  • Posts: 397
    • Show only replies by biggun
    • http://www.greyhound-data.com/gunnar/
Re: Layers.library V45 on the aminet
« Reply #197 on: September 14, 2014, 05:44:35 PM »
Please let me give my 2cents to it to.

I think what Thomas does is great,
He improved the algorithm resulting in a big improvement.

I also like what Cosmos does.
Cosmos shows great ASM skills and extrem hard will to understand the ASM codes.
His tuning are certainly interesting for many.
But I personally could also envision other great uses for his shown great skills.

For example:
Lets assume Cosmos would develop the bitplane GFX driver for AROS.
This could be great for the free AMIGA future and make AROS become real fast on 68k.

I think what Cosmos does right now is he looks at the compiler generated ASM output
and then figures that there are 10 instructions - which a smart coder could rewrite doen to 6.
And then he does this.
While this is cool and great - I wonder what would happen if he would not change this one routine -
but change the compiler to in the future see and use such optimizations.
Could this work? And how would the gain be if one would then recompile A-OS or AROS with the tuned compiler?

Offline itix

  • Hero Member
  • *****
  • Join Date: Oct 2002
  • Posts: 2380
    • Show only replies by itix
Re: Layers.library V45 on the aminet
« Reply #198 on: September 14, 2014, 06:06:13 PM »
Quote from: biggun;772986

I think what Cosmos does right now is he looks at the compiler generated ASM output
and then figures that there are 10 instructions - which a smart coder could rewrite doen to 6.
And then he does this.
While this is cool and great - I wonder what would happen if he would not change this one routine -
but change the compiler to in the future see and use such optimizations.
Could this work? And how would the gain be if one would then recompile A-OS or AROS with the tuned compiler?


I dont think we are going to see AOS recompiled ever again. It is irreversibly sucked to a black hole.

And I don't think he has interest to that anyway.

But if it has to be done in 68k asm then it would be better rewrite intuition library in machine language only. Being rewrite rather than being based on disassembled binary you can open source this project.
My Amigas: A500, Mac Mini and PowerBook
 

Offline NorthWay

  • Full Member
  • ***
  • Join Date: Jun 2003
  • Posts: 209
    • Show only replies by NorthWay
Re: Layers.library V45 on the aminet
« Reply #199 on: September 14, 2014, 06:08:06 PM »
Quote from: wawrzon;772969
how are you going to make sure you are buying from the actual owner and that there will not be any claims from third parties in the future?

That is of course the tricky part. The seller should take some legal/contractual responsibility to accept ownership over future disputes.
Which is basically what they have done anyway by selling 3.5/3.9. An attempted buy-out should say something about their confidence in the legal side.

OTOH, internet exposure of source code can turn the reaction knob to 11 and flush out someone who has been biting their tongue or just been sitting idle (or simply been unknowing) since C= died.
 

Offline olsen

Re: Layers.library V45 on the aminet
« Reply #200 on: September 14, 2014, 06:18:26 PM »
Quote from: itix;772988
I dont think we are going to see AOS recompiled ever again. It is irreversibly sucked to a black hole.

Huh? Been there, done that (1999). OS4 is based upon that work.
 

Offline itix

  • Hero Member
  • *****
  • Join Date: Oct 2002
  • Posts: 2380
    • Show only replies by itix
Re: Layers.library V45 on the aminet
« Reply #201 on: September 14, 2014, 06:39:55 PM »
Quote from: olsen;772990
Huh? Been there, done that (1999). OS4 is based upon that work.


That is 15 years ago. I don't think we are going to see new 68k release. Only that matters to people here.
My Amigas: A500, Mac Mini and PowerBook
 

Offline modrobert

  • Newbie
  • *
  • Join Date: Nov 2008
  • Posts: 47
  • Country: th
    • Show only replies by modrobert
Re: Layers.library V45 on the aminet
« Reply #202 on: September 14, 2014, 06:54:46 PM »
Quote from: Thomas Richter;772985
It's a mix of both, depending on the level of the function. The low-level stuff is in assembly, the higher and more complex operations are in C. Even then, the assembler stuff depends on some macros to provide at least some orientation on the program flow, i.e. the authors have used macros for conditions, loops and so on. The result is not quite as scary as "raw" assembler.

This specific function, and the region functions in particular, are in assembly. It's part of the low-level slice&dice operations that are commonly used between layers and graphics. Ideally, regions (like other layer-like structures) should be pooled, and hence NewRegion() should first try to take the region memory from a graphics specific pool, and recycle regions whenever possible.  

Unfortunately, it's not what it does. If you want such pooling, PoolMem is there to help (it keeps memory pools for small fragments like struct Region, hence *may* help here).

Even more unfortunately, struct Region is documented, so *in principle* an appplication could create such a structure by AllocMem(), bypassing NewRegion() (bad idea!) or on the stack (also a bad idea!) Even then, such a change should be synchronized between graphics, layers, and code parts that re-implementent parts of layers & graphics (P96, cybergraphics, EGS...). Thus, in general, it is not such a good idea to change this lighthearted. It requires a pretty complete code review to understand the possible implications, and potentially a re-compile of graphics, layers, P96 and cybergraphics (in worst case).

One way or another, this example shows a constructional weakness of the whole AmigaOs: Too many internals have been documented, and the API is not always clear. Ideally, all "primitive objects" should have been left undocumented, or should have been equipped with clear constructor/destructor calls. struct Region is one of the "better" implementations by using such an API (partially), but the situation is much worse with other graphics primitivities, such as struct RastPort, struct Bitmap (big outch!), struct ViewPort (bigger outch!) or struct View. To avoid the many problems, V37 graphics jumps in circles and stores private data on the ViewPort not in the view port, but in the color map (even bigger outch!) loaded in the view port just to be able to extend it, and uses hoops such as GfxAssociate to be able to backwards-extend the gfx engine.

Gfx is a clear case of "how not to design a library". Intuition much better code from a software engineering perspective (there are clear structures and a relatively clean API). I guess this is all because gfx was written in a rush, for a deadline that was too close to allow a good implementation, and pretty much in an ad-hoc way.

Then again, all the region low-level calls should, instead of calling NewRegion() internally through gfx, go through the LVO instead. They don't do that, but probably should. Another outch.


Thanks for taking the time and explain this.

I'm starting to understand that changing one routine, you can break others unknowingly, and even if you test OK, chances are that hardware compatibility is lost through dependencies of other libraries, also depending on the order of the function calls (which in rare circumstances can fail even without changing anything).

Hmm, I think there is a name for it; "Spaghetti code". Maybe we should be happy this isn't open source (joking)? ;)

So, if I understood correctly, anyone playing with this should at least take time and test these libraries in conjunction; graphics, layers, P96 and cybergraphics.
A1200: 68020 @ 14 MHz (stock), 2MB Chip + 8MB Fast RAM, RTC, 3.1 ROMs, IDE-CF+4GB, WiFi WPA2/AES, AmigaOS 3.1, LCD 23" via composhite - Thanks fitzsteve & PorkLip!
 

Offline vxm

  • Jr. Member
  • **
  • Join Date: Apr 2012
  • Posts: 59
    • Show only replies by vxm
Re: Layers.library V45 on the aminet
« Reply #203 on: September 14, 2014, 07:16:56 PM »
I have a dream. I'd like to see ThoR and Cosmos working intelligently together without killing each other, each in his field of expertise.
 
But unfortunately, since I saw the little mouse to be eaten by the cat, I do not believe in Christmas.
 

guest11527

  • Guest
Re: Layers.library V45 on the aminet
« Reply #204 on: September 14, 2014, 07:57:12 PM »
Quote from: modrobert;772993
Hmm, I think there is a name for it; "Spaghetti code". Maybe we should be happy this isn't open source (joking)?

The problem is not even that individual functions are spaghetti. The problem is that it lacks a proper organization and design. Graphics covers functions that would belong into several separate libaries and resources. It covers low-level blitter functions that should have better gone into a blitter.resource.It covers low-level graphic functionalities like line drawing and filling. These should have gone into a library on top of the blitter.resource, with a clear abstraction layer between them such that the blitter could be exchanged with something else for rtg. Then there is a display-builtup functionality of graphics that handles views and viewports. Again, this should have been separated into a low-level copper.resource, and a high-level display management. Then there is the even higher-level monitor setup and the monitor database. We have simple sprite management (better a sprite.resource) and high-level bobs and vsprite handling (should be a sprite or mobs.library). Then we have all the region-management (this should go into layers). Text handling and rendering (should go into a font.library, and diskfont should become obsolete). There are probably a couple of other functions I've forgotten...

One way or another: Gfx is a mixed collection of functions that are "somehow" related to the display system, on various hierarchical levels, without a clear structure and a clear separation into modules. I believe the main problem is that graphics is "too fat".

Concerning your last comment. Yes, it would be probably best to redesign this from scratch, but it doesn't help because it won't be compatible to the existing AmigaOs. Yes, one should probably rewrite several parts of AmigaOs, but again, it doesn't help because nobody would use this Os. There is already Linux and Windows.
 

Offline modrobert

  • Newbie
  • *
  • Join Date: Nov 2008
  • Posts: 47
  • Country: th
    • Show only replies by modrobert
Re: Layers.library V45 on the aminet
« Reply #205 on: September 14, 2014, 08:16:53 PM »
Quote from: olsen;772983
All versions (1.x through 3.x) are written in a mix of 'C' and 68k assembly language, strongly leaning towards 68k assembly code. Looking at the API, you'll find that a large part of the operations are about filling in data structures, hooking them up and then eventually telling the hardware what to do with them. Typically, the "clerical operations" are written in 'C', and the part which talks to the hardware is written in assembly language.

graphics.library, like intuition.library, is one of the more complex operating system components, both in how it uses data structures and how many distinct operations it can carry out. Hence, you'll find that 'C' plays a major part in its implementation.

Thanks for the explanation.

Quote from: olsen;772983
That said, when you need to talk to the hardware in a very specific manner, you couldn't do that quite so elegantly back in 1986 if you didn't use 68k assembly language. There is complex assembly language code in graphics.library, it even uses its own preprocessor (!) in order to simplify writing loops (while .. do) and control structures (if .. then .. else).

Yes, I assume those are the "macros" Thomas Richter mentioned in his post.


Quote
If you have specific requirements for carrying out operations, you may get better control through the use of assembly language than any modern 'C' compiler could provide you with. C11 has just gained new control keywords in order to make interfacing to hardware more straightforward, but it will take a while for the language to evolve to give you the kind of control only assembly language can give you.

As for assembly language becoming largely redundant, it's probably unavoidable. 'C' in particular is a more expressive language which encodes in fewer lines what assembly language, by its very nature, requires much more effort to express. The thing is, if you can say the same thing with fewer words, the chances of mucking it up are somewhat reduced. If you have the right language, you can even verify the correctness of the instructions and data structures you used, which is something that eludes assembly language by its very design.

As for inefficient code, performance nowadays does not necessarily come out of implementing an algorithm through the optimum low level language encoding you could choose (that being assembly language). You can't necessarily predict how your code will be executed, and if you can, you might have to run the gauntlet of observing a handful of operating conditions under which it executes, such as regarding optimum pipeline use, prefetch, etc. This can get so ugly that it has to be automated (look up how one programmed the Motorola 56000 DSP in its day, just to get an idea how weird this can get) just to let the programmer worry about implementing his design correctly.

This is how progress looks like: the gains made through building faster processors are used in such a way that writing more complex software, more secure software and more reliable software becomes an easier goal to achieve through the use of tools and code generation which does not necessarily attempt to squeeze the last cycle out of the machine. This is, in the end a trade-off, if not a sacrifice.

I agree with this when it comes to modern multi-core CPUs, and especially recent APU architectures. A nightmare to interact with on the low level, way too many opcodes (which look ugly when disassembled...hehe), huge integrated caches to make up for slow external buses, and the die photos looking more like a small city than a CPU.

Thankfully, some of us still have 68k to play with, which is one of the best places to enjoy assembler IMHO.

For me the Amiga is a fun hobby, and since I don't rely on it much for productivity these days, it's all about testing old (and some new) software, trying to code my own programs, playing classic games, and experimenting, even with the hardware building interfaces, coding VHDL and such. The A1200 is my main computer in the electronics lab (that's what I call my little shed with a soldering iron) these days, with proper 25-pin serial interface, parallel port, and mainly TTL signals to play with on the board, it's just great.
« Last Edit: September 14, 2014, 08:32:49 PM by modrobert »
A1200: 68020 @ 14 MHz (stock), 2MB Chip + 8MB Fast RAM, RTC, 3.1 ROMs, IDE-CF+4GB, WiFi WPA2/AES, AmigaOS 3.1, LCD 23" via composhite - Thanks fitzsteve & PorkLip!
 

guest11527

  • Guest
Re: Layers.library V45 on the aminet
« Reply #206 on: September 14, 2014, 08:48:28 PM »
Quote from: modrobert;772999
Yes, I assume those are the "macros" Thomas Richter mentioned in his post.

Yes. Actually, I did not know that this was done by pre-processing. I would do it with macros, but now we're living in different times. Actually, if you'd see my DevPac macro set, then you'd see that it accomplishes pretty much the same without a preprocessor. But DevPac had a very powerful macro processor, and DevPac wasn't surely available when writing graphics.
 

Offline psxphill

Re: Layers.library V45 on the aminet
« Reply #207 on: September 14, 2014, 09:40:22 PM »
Quote from: Thomas Richter;772997
These should have gone into a library on top of the blitter.resource, with a clear abstraction layer between them such that the blitter could be exchanged with something else for rtg.

Abstraction layers slow every thing down and they only had one piece of hardware, so it made sense.
 
 I still wouldn't abstract it straight from graphics.library to blitter.resource (or blitter.device), there should be a device that is in control of allocating a screen, displaying it and drawing into it. graphics.library should just deal with managing those devices.
 

guest11527

  • Guest
Re: Layers.library V45 on the aminet
« Reply #208 on: September 14, 2014, 09:53:54 PM »
Quote from: psxphill;773005
Abstraction layers slow every thing down and they only had one piece of hardware, so it made sense.
Yes and no. There was no reason to stick it all into the same fat library. For example, the whole bunch of BltBitmap() & friends that do not operate on RastPorts but Bitmaps would be simple to lay off into a second library, and it would have cost nothing to make calls into that library for the higher level functions. Unfortunately, this is not what happened.  

If you look for example at all the graphic operations in graphics, they already exist (in a way) in two layers: A core-layer that does the actually graphics hardware banging, and a lot of high-level fuzz that does the clipping of the operation into the available cliprects of the layer of the rast port to draw in. This would be an excellent spot to cut graphics into two. The lower (hardware) resource would get bitmaps and drawing style settings, the higher level would do all the clipping before sending instructions to the lower level. This avoids the kind of mess we see with P96 or cybergraphics where all the clipping operations are duplicated (in P96 *and* graphics) because there is no clean way to tell graphics to leave the hardware access to somebody else.

You would probably be astonished how hard it is to do proper line clipping if you have drawing styles (aka MinTerms), the FirstDot flag and line patterns involved. Graphics gets it *almost* right, cybergraphics messed it completely, and the latest P96 should probably be as fine as the original graphics, hopefully avoiding the one bug I remember in Gfx.
 

Offline wawrzon

Re: Layers.library V45 on the aminet
« Reply #209 from previous page: September 15, 2014, 01:08:27 AM »
Quote from: Thomas Richter;773008
You would probably be astonished how hard it is to do proper line clipping if you have drawing styles (aka MinTerms), the FirstDot flag and line patterns involved. Graphics gets it *almost* right, cybergraphics messed it completely, and the latest P96 should probably be as fine as the original graphics, hopefully avoiding the one bug I remember in Gfx.

seems so. even though actually working now om aros68k there are cases where it fails to work properly, but then again, not enough reliable testing causes issues not to get fixed. it costed two years that my repeated reports were actually taken seriously and interpreted as completely lacking line clipping implementation on planar screens. thats fun, isnt it?

edit:Btw, im not saying something is wrong on developer side, just of more quality reports would happen it would be easier to fix not that obvious problem.
« Last Edit: September 15, 2014, 01:30:53 AM by wawrzon »