Layers.library V45 on the aminet

[modrobert]

Finally I checked this morning the rtg.library branch bug : I changed only one byte ($00 instead of the wrong $04)

Impressive one byte patch!

Thanks for taking the time to actually debug stuff on the assembly level, and giving me another reason to love the Amiga community.

It's not "because they can". Nobody is needlessly destructive here. It is just that software was created under contract, the authors are not necessarily the owners, and the actual owners have no business anymore in the Amiga market, with the responsble persons having already left the company a long time ago, along with insight knowledge on the contract, or possibly even the contract itself.

So what can an author say? If "yes", he's pobably breaching a contract that still exists, even though the owners do not care. Contacting the owners might be hard or even impossible, leave alone they may not even know anything from back then - though that doesn't invalidate the contract. Thus, from a legal perspective, it is of course the safest option not to say anything.

In other words an outsider, not bound by old business contracts like the original authors, can pretty much fix/patch anything under "fair use" considering the age of the Amiga software (and hardware). This could be a way forward, right?

[modrobert]

No, it rather means that an outsider can make a "hack and slash" attack to the software and make the software infrastructure completely unmaintainable becaues nobody knowns for any future applications which version the software had, where the actual bug was, how to fix the bug in first place, and on which version the software depends.

Again, and sorry to sound arrogant again, but this is the difference between software engineering and hacking. Software engineering means to keep the software maintainable, to have a version managenent, a software repository, a revision history and a process how to update software. You may not believe it, but yes, there is a repository for P96.

This makes sense if the software is maintained, but what happens when it is not maintained due to the limitations you stated yourself in previous post?

Also, think of a solution this time, be constructive, try to think out of the box for a moment instead of stating one excuse after another.

[modrobert]

It means that somebody must maintain it if there is a demand. Quite obvious? Isn't it that I said multiple times that I'll try to look for a solution? Hacking up the software to infinity is not a solution because it leaves an unmaintainable junk of garbadge. Now tell me again I won't look for solution.

At this point you are trapped in a circular argument, defunct by design. We were discussing closed source cases where you claimed the legal rights to the code in question were uncertain effectively preventing existing developers to maintain the source code already in their possession, or releasing it to someone else.

The active Amiga community is pretty small these days, so even though I don't know you or Cosmos personally, I know projects both of you are involved in, past and present.

For example, Cosmos ongoing assembler optimization of classic 68k Amiga libs where performance has been improved and size reduced, mainly because they were written in C to begin with.

Also some of your projects, like this one (the thread) and MMULib.

As mentioned previously in this thread, and where most of us seem to agree, is that the Amiga is mainly used by hobbyists these days and a few hardcore individuals who never really stopped using it productively, at least when it comes to classic hardware.

I for one appreciate the "hack and slash" Cosmos is doing, and your projects as well, one does not rule out the other. When old software needs a fix/update in this community, someone will come up with a working solution, even if it means bare metal hacking of the binary or full source code repository access.

[modrobert]

You still don't get it, right? I'm trying to find out what the situation is, understood? How much more explicit do I need to get?

Sorry about that, my mistake, I interpreted your previous replies as if it was intentionally left in limbo for good reason.

Have you made any measurements showing the efficiency of these "improvements", and have you considered that this leaves the libraries in a pretty unmaintainable state?

No, I haven't made my own measurements, was just inspired by the idea to optimize the libraries and acknowledgements by others that Cosmos patches were indeed working. I did check some of the assembler code "before/after change" which he posted over at eab a while back, looked good to me.

My 68k assembler knowledge is still a bit rusty compared to what it used to be, slowly getting back in the game. What I love about 68k is the CISC part and that the Amiga hardware is well documented, even binary patches disassembles to decent source code which can be easily worked on and understood.

I thought about this some more last night, and in cases where no one is maintaining the source code (or where the source code indeed is lost), one solution could be disassembling the binaries (eg. with adis or resource). Make sure the source compiles OK, and then post/push that to github for everyone to enjoy, to either improve further or fork into new projects. At least it would provide some structure compared to binary patches.

Again, what is "working" for you? For me it means that I can also fix something in two years in the future. With the kind of "improvements" I see from Cosmos this is unlikely to be even possible. First of all, it doesn't really improve anything, and second, even if the results were measurable, the improvements were lost or incompatible to any other changes that had to be made.  Anyhow, make your pick. If you don't agree with me, I'm ready to leave and don't touch any of the old stuff anymore.

I hope you will continue, and accept that people have different views on contributions which clearly has good intention.

[modrobert]

Problem is, in none of the cases the source code is lost in limbo. It's all existing, probably not in perfect shape, but in a repository "somewhere". Problem is, it is not so easy to get access to it, and it is a struggle, but maybe it's worth it. I can try. I may fail. That's what I can say. Will it become accessible for everyone? Unlikely, not my decision.

If it is all "existing" I feel confident it will become open source eventually, at least if someone like you is involved and help to push in that direction.

Disassembly doesn't resolve the legal problems. Thus, on this basis, you cannot really place the disassembled sources somewhere and run from there maintainence of obsolete software. It doesn't work, and you put yourself and everyone participating in that in a risky situation. The only *waterproof* way of getting there - if the originals are lost indeed - is really to take the documentation, and re-implement the API as described by the documentation, from scratch, without having access to the sources. You might not like it, but these are the rules. Thus, if you want to maintain intuition, go ahead, contribute to AROS, make it as good as possible, or better than the original, then you have an "intuition" that works for everyone and satisfies your needs. That's perfectly ok for everyone. Yes, it *also* takes long. There is no easy access, I afraid.

My life has always been guided by ethics rather than following specific rules/regulations, so taking a little risk can be worthwhile for the sake of the community. Anyway, I can fully understand that messing with a closed source project which is actively maintained is disrespectful to the original author(s) in the community, and for that reason should be left alone.

Legally I can only see a problem if you are living in USA (or hosting the source code in USA), and limited by the harsh laws regarding reverse engineering there. Perhaps some countries in Europe might be a bad idea to base this as well. There are plenty of ways this can be solved while remaining legal, as long as you don't limit the perspective to one country.

[modrobert]

I have another war story; some guy made a bet on IRC that he could host the Amiga Kickstart ROMs on a web server, free for anyone to download, and he did just that for a year without complications, until the domain hosting ran out.

[modrobert]

Ok, here's an offer. Provided I compile a binary version of P96, will you host it and will you indemnify me from any legal claim? Written contract, signed, by both parties?

I can host the file, but you can forget any contract, because it would be invalid to indemnify you in this case.

Besides, if you really wanted this file hosted somewhere, you could easily have done that without any legal risk by simply remaining anonymous and upload to any free file hosting service out there. This was just some lame attempt to prove a point, sorry, it failed.

[modrobert]

Why, it worked quite well. If you don't want to take the risk, why should I?

Anyhow, I guess you understand the point.

If it was possible to indemnify against a lawsuit every corporation out there would hire some poor drug addict to take the legal responsibility for their actions, it doesn't work this way, fortunately.

EDIT:

Apparently it is possible indemnify against copyright lawsuits regarding software in some countries, my mistake. In a way this proves what a crappy racket the copyright legal system can be ethically, creating misery for any poor bastard who happened to sign a paper, probably a system created by lawyers to earn fees as well. Anyway, in this case it would be meaningless since copyright laws where my company resides in Thailand only applies to domestic software.

[modrobert]

Why would it be invalid? He purely asked you to provide indemnity insurance, which is "a thing".

As I already mentioned in the "EDIT:" part of the post which was written long before your post (check time).

He wouldn't be able to pay up and you would still owe the money.

So, Thomas Richter just have to trust that I'm good for the money then, hehe.

You can indemnify against anything, if someone is willing to take the risk on you.

Any kind of crime...really? No wonder the white collar criminals rarely get caught.

[modrobert]

The only realistic method for achieving this is a H&P bounty for the release of OS3.9 sources.

Great idea. OS3.1 as well. Open source or die (trying)!

BTW: I'm typing this on my A1200, so no more recommendations to use Aros, that's best suited for a PC.


PS:

Have a good weekend everyone, especially those who disagree. High time to turn up the volume in HippoPlayer, goose bumps galore. Nothing really beats Amiga, does it?

[modrobert]

No point in it, it's already been reimplemented 3 times over.

Well, I can *definitely* see the point!

The 3 re-implementations are for "NG Enthusiasts" only, while the fourth one (or really, the first one, since Amiga OS was and is the *original*, the only *real* Amiga)



This one is still is of very much interest to:

1) Retro fans using real Amiga HW, or UAE
2) Retro fans using new HW that mimics the real Amiga, like Minimig
3) Fans (not sure it's right to call them "retro" only) that wants to *improve* the real Amiga HW, I'm of course thinking of NatAmi!

None of these has the *slightest* interest in any of the three re-implementations. It's not difficult to understand. They want to use the real Amiga 68k apps in the true way, the real classical Amiga games, demos, etc, and flawless compatibility is the most important thing. There is no alternative to the real Amiga OS then, running in a 68k environment with all the custom chips in place (either in HW or in UAE). "NG Enthusiasts" has a different view, they want to use an Amiga environment in a way that is as close to 2011 level computing as possible; modern Internet apps, playing modern media files, etc. Amiga compatibility is an appreciated bonus, but that's not the main thing. To them, it's mostly a matter of choosing one or more of the re-implementations.

What the Amiga 68k fans might be interested in however, might be a continuation and improvements of Amiga OS (and by that I'm *not* talking about Hyperions OS4, which is yet another re-implementation, and the most immature one even). Especially NatAmi would benefit from this, since they will need an OS that can make the most use of the new features of their board. That's what I call "Amiga OS+" in the picture above in lack of a better word. An evolved version of the real Amiga OS 68k. The NatAmi crew will probably make their own third party add-ons and replacement if they ever get this far with their HW, possibly using some stuff from AROS, and writing some stuff themselves. It will however be completely unofficial of course.

Amiga OS 68k can only be distributed by Hyperion AFAIK thanks to that crappy "agreement" (cough cough (read: robbery)). But not even they can release it as open source, since Amiga Inc still owns it. You think it's a mess? It truly is!

(Edit: Oh BTW, the time of the first MorphOS release is one month too late in the picture, my bad... )

I agree with takemehomegrandma in this case regarding the "first implementation", quote from this forum thread.

[modrobert]

What exactly is it that you do not understand with the word "measurement"? It means, making an experiment, showing the efficiency of the changes. It does not mean "guesswork". BTW, do you have an idea *why* there is an AllocMem()? (Actually, there isn't. There is a NewRegion() call). Thus, can you ensure that the buffers that are released there are in fact idental to the buffers allocated? Or did you "just made an experiment and it happened to work for you"?

THe software interfaces used there are there for a reason - the regions have luckely (unlike many other crap in gfx) a clear constructor method (NewRegion()), and it's typically a good idea to use that. Unless there is a very clear reason that this is not a good idea, and the only clear inidicator would be that there is a specific bottleneck in AndRegionRegion() in a specific situation. Thus, I ask you again: Which is the bottleneck, and which is the situation within which this bottleneck appears, and what is the speed improvement you get in this specific situation.

Just for the records, here is my specific bottleneck I addressed in layers V45, just for completeness: Open a lot of overlapping windows on a graphics-card driven screen, then re-arrange the windows (depth-arrangement), then measure. You'll get a noticable speedup.

Now, what's your story that requires this change, and how did you ensure that the change is correct?

Software engineering. You know it...

Isn't the original graphics.library written in C?

You can't expect a C compiler to match the optimizations made manually by an assembler programmer. I've heard so many stories about how well C compilers optimize these days and how assembler is made redundant, and then when you actually disassemble the code the compiler produced, it's bloated, filled with jump tables, and inefficient code.

[modrobert]

It's a mix of both, depending on the level of the function. The low-level stuff is in assembly, the higher and more complex operations are in C. Even then, the assembler stuff depends on some macros to provide at least some orientation on the program flow, i.e. the authors have used macros for conditions, loops and so on. The result is not quite as scary as "raw" assembler.

This specific function, and the region functions in particular, are in assembly. It's part of the low-level slice&dice operations that are commonly used between layers and graphics. Ideally, regions (like other layer-like structures) should be pooled, and hence NewRegion() should first try to take the region memory from a graphics specific pool, and recycle regions whenever possible.  

Unfortunately, it's not what it does. If you want such pooling, PoolMem is there to help (it keeps memory pools for small fragments like struct Region, hence *may* help here).

Even more unfortunately, struct Region is documented, so *in principle* an appplication could create such a structure by AllocMem(), bypassing NewRegion() (bad idea!) or on the stack (also a bad idea!) Even then, such a change should be synchronized between graphics, layers, and code parts that re-implementent parts of layers & graphics (P96, cybergraphics, EGS...). Thus, in general, it is not such a good idea to change this lighthearted. It requires a pretty complete code review to understand the possible implications, and potentially a re-compile of graphics, layers, P96 and cybergraphics (in worst case).

One way or another, this example shows a constructional weakness of the whole AmigaOs: Too many internals have been documented, and the API is not always clear. Ideally, all "primitive objects" should have been left undocumented, or should have been equipped with clear constructor/destructor calls. struct Region is one of the "better" implementations by using such an API (partially), but the situation is much worse with other graphics primitivities, such as struct RastPort, struct Bitmap (big outch!), struct ViewPort (bigger outch!) or struct View. To avoid the many problems, V37 graphics jumps in circles and stores private data on the ViewPort not in the view port, but in the color map (even bigger outch!) loaded in the view port just to be able to extend it, and uses hoops such as GfxAssociate to be able to backwards-extend the gfx engine.

Gfx is a clear case of "how not to design a library". Intuition much better code from a software engineering perspective (there are clear structures and a relatively clean API). I guess this is all because gfx was written in a rush, for a deadline that was too close to allow a good implementation, and pretty much in an ad-hoc way.

Then again, all the region low-level calls should, instead of calling NewRegion() internally through gfx, go through the LVO instead. They don't do that, but probably should. Another outch.

Thanks for taking the time and explain this.

I'm starting to understand that changing one routine, you can break others unknowingly, and even if you test OK, chances are that hardware compatibility is lost through dependencies of other libraries, also depending on the order of the function calls (which in rare circumstances can fail even without changing anything).

Hmm, I think there is a name for it; "Spaghetti code". Maybe we should be happy this isn't open source (joking)?

So, if I understood correctly, anyone playing with this should at least take time and test these libraries in conjunction; graphics, layers, P96 and cybergraphics.

[modrobert]

All versions (1.x through 3.x) are written in a mix of 'C' and 68k assembly language, strongly leaning towards 68k assembly code. Looking at the API, you'll find that a large part of the operations are about filling in data structures, hooking them up and then eventually telling the hardware what to do with them. Typically, the "clerical operations" are written in 'C', and the part which talks to the hardware is written in assembly language.

graphics.library, like intuition.library, is one of the more complex operating system components, both in how it uses data structures and how many distinct operations it can carry out. Hence, you'll find that 'C' plays a major part in its implementation.

Thanks for the explanation.

That said, when you need to talk to the hardware in a very specific manner, you couldn't do that quite so elegantly back in 1986 if you didn't use 68k assembly language. There is complex assembly language code in graphics.library, it even uses its own preprocessor (!) in order to simplify writing loops (while .. do) and control structures (if .. then .. else).

Yes, I assume those are the "macros" Thomas Richter mentioned in his post.

If you have specific requirements for carrying out operations, you may get better control through the use of assembly language than any modern 'C' compiler could provide you with. C11 has just gained new control keywords in order to make interfacing to hardware more straightforward, but it will take a while for the language to evolve to give you the kind of control only assembly language can give you.

As for assembly language becoming largely redundant, it's probably unavoidable. 'C' in particular is a more expressive language which encodes in fewer lines what assembly language, by its very nature, requires much more effort to express. The thing is, if you can say the same thing with fewer words, the chances of mucking it up are somewhat reduced. If you have the right language, you can even verify the correctness of the instructions and data structures you used, which is something that eludes assembly language by its very design.

As for inefficient code, performance nowadays does not necessarily come out of implementing an algorithm through the optimum low level language encoding you could choose (that being assembly language). You can't necessarily predict how your code will be executed, and if you can, you might have to run the gauntlet of observing a handful of operating conditions under which it executes, such as regarding optimum pipeline use, prefetch, etc. This can get so ugly that it has to be automated (look up how one programmed the Motorola 56000 DSP in its day, just to get an idea how weird this can get) just to let the programmer worry about implementing his design correctly.

This is how progress looks like: the gains made through building faster processors are used in such a way that writing more complex software, more secure software and more reliable software becomes an easier goal to achieve through the use of tools and code generation which does not necessarily attempt to squeeze the last cycle out of the machine. This is, in the end a trade-off, if not a sacrifice.

I agree with this when it comes to modern multi-core CPUs, and especially recent APU architectures. A nightmare to interact with on the low level, way too many opcodes (which look ugly when disassembled...hehe), huge integrated caches to make up for slow external buses, and the die photos looking more like a small city than a CPU.

Thankfully, some of us still have 68k to play with, which is one of the best places to enjoy assembler IMHO.

For me the Amiga is a fun hobby, and since I don't rely on it much for productivity these days, it's all about testing old (and some new) software, trying to code my own programs, playing classic games, and experimenting, even with the hardware building interfaces, coding VHDL and such. The A1200 is my main computer in the electronics lab (that's what I call my little shed with a soldering iron) these days, with proper 25-pin serial interface, parallel port, and mainly TTL signals to play with on the board, it's just great.

[modrobert]

I understand

it is a pity that the few experienced devs left cannot contribute because of silly contracts from long ago

They can contribute to Aros if a bounty making OS3.1 open source succeeded. This would effectively end the NDA, or did I miss something (again)?

Also, Thomas Richter and olsen have effectively convinced me that binary patching is bad in the current situation, only took like ten posts of explaining to do it (hehe). Still, I can't help liking Cosmos and respect what he does, it goes beyond logical reasoning, so no need to convince me further.