More about why SObjs: isn't a great idea

[Karlos]

To my mind, the biggest problem with shared objects is that they suffer from NIH. However, since we all decided ELF was a good idea, it's not entirely unreasonable for someone to consider the linkable part of the specification.

You didn't even run into the fun scenario: Two incompatible SObjs with the exact same name, but different ABI (this has happened at least once already). There hasn't been proper documentation about SObj creation and this has lead into all sort of trouble.

Yeah, it was a pain back when most people were migrating from gcc 2.95, the ABI changed a great deal. However, one of the reasons it changed so much was to avoid having to do it again for a long, long time. Don't expect to see changes of that magnitude any time soon. Unless C++0x throws a big spanner in the works. Not impossible...

They're worse than amiga shared libraries anyway: SObjs aren't even shared, each opener gets their own copy in memory.

Sure, on AmigaOS, but not for OS that support .so files generally.

As far as I can tell the only reason they exist is that apparently OS4 coders are incapable of creating amiga shared libraries with per caller library base (and the static data in the library base possize). Or they're just too lazy to do it properly, and decided to take the shortcut by introducing the non-shared libraries.

The only reason? I think that's being deliberately short sighted. My guess would be that one really obvious reason would the fact that traditional amigaos shared libraries offer no way to provide the required linker constructs for C++ and other languages that some people, myself included, like to use. The world is not tied to procedural programming.

If someone wrote an improved .library specification and loader that allowed things like C++ symbol export it would surely have problems too and at least we know what .so's particular problems are.

A grave mistake if you ask me.

The only alternative that provides the same support is static linking. Assuming we disregard the increase in disk storage (which is cheap), one disadvantage of static linking is that you don't get any benefits of a new version of a library without re-linking your app. Obviously, any major change to said library, by definition, probably requires a recompile of the application, so no problem there. On the flipside, provided the exported symbol table doesn't change, there shouldn't need to be any such change. Theoretically, I should be able to swap out some generic .so with say an altivec optimised one, for example and all client users of that library get the same benefit.

[Karlos]

I think the rationale was for easy quick-n-dirty porting (which I'm also not fond of), but I'm thoroughly baffled as to why these well-established native programs are affected. OS4 xadmaster dates from December 2006, which I thought was before SObjs: was introduced. And I can't see why DOpus would need libbz2.so on its own.

Agreed, that is pretty weird. A .so having a dependency on a .library is fine, the converse doesn't really seem reasonable when viewed on the "nativeness" scale in which .library is obviously further up the chain than .so.

Compiling these ported libraries into programs statically would be a cleaner solution.

Personally, I dislike static linking, but that's just me. It does make it easier for the end user, but then they always complain about the bloat of statically linked executables and the fact that statically linked application X doesn't support feature X that exists in application Y which happens to have been linked against a newer library

[Karlos]

C++ must die anyway

Get away with ye' , y' love it really :-)

Well, I do, and that's all that matters from this direction

[Karlos]

Personally, I'd love to see some native implementation supported by a decent compiler that would allow traditional shared libraries to provide the required services that .so files provide as far as languages beyond C are concerned.

However, since nobody has done anything like that (and it would not be a trivial undertaking), supporting the existing ELF shared object at least grants the developer some flexibility. Furthermore, it's not as alien a concept as it seems, seeing as we're using ELF already.

The earlier observation that a pair of identically named .so files implementing different ABIs being a uniquely bad problem seems a little contrived. After all, the existing .library mechanism doesn't prevent this kind of lunacy either. I could create a foo.library that is nothing to do with the existing foo.library and as long as the meta data is acceptable, the OS will happily pass it to clients that wanted the original, only to invoke meaningless vectors and crash.

[Karlos]

Also a possibility, but with particularly well-coded programs you can put necessary .library files in PROGDIR: or PROGDIR:Libs instead of SYS:Libs. Granted, that doesn't make it "shared" anymore, but you can have as many different incarnations of foo.library as you have programs that use it. I think MorphOS checks all these places (and more) for every library it attempts to open - Piru can correct me here. This sort of arrangement for .so files would be an improvement.

That doesn't work. If you load application X which requires progdir:foo.library A and then start application Y which requires libs:foo.library B, application Y isn't going to work. The reason being that the request to open foo.library is going to note that it's already open and return the base pointer to it.

Contrast this to progdir:foo.so versus SObjs:foo.so. If the .so files are searched for in the same PROGDIR: > SOBJS: order, then both applications should run fine, each with it's own private copy of the library.

[Karlos]

I know it's sort of off topic, but have you guys played with Objective-C and its runtime object oriented system? I think it's wonderful to have your classes just like the old Amiga shared library

It's the only inherent feature of Objective-C that I actually like. The rest of it is a dog's dinner

[Karlos]

Some issues still outstanding are whether programs can take advantage of updated shared objects if the program in question ties them to a specific version by filename (and if it's possible to not do that), and what to do if PROGDIR: is C: (i.e., for ported command line utilities). Are the latter type of programs stronger candidates for static linking?

You could static link against a library if you only wanted that particular version and no other.

Linux uses a versioned shared object scheme that works, but it does require programmers to observe a few rules. The actual .so file will contain a major and minor version number as part of it's name. Changes to the minor version number are not allowed to change the interface presented by the .so (well, they may be able to add new calls and data but they can't alter or remove any existing ones) , they are for internal changes only (bugfixes etc). Changes to the major version number indicate larger changes to the interface that may not be backwards compatible.

Normally, an application will be linked against a major version. Generally, this is just a symlink to a specific revision to that version. Updates can then come along and add new revisions. If any thing goes wrong with a new revision, rolling back to the previous one is trivial, the symlink is just pointed back at the last good working revision.

If you get a new major version, you still can have your previous one installed and all applications linked against that previous major version will still use it.

Finally, you usually have a simple .so symlink to the most recent major version/revision for applications which aren't supposed to be version sensitive.

It all sounds far more complex than it is, but it does mean you can create schemes with multiple versions/revisions of the same shared library installed that generally satisfies most applications. It used to be a bit of a pain but once a few guidelines were set it all sorted itself out.

I think the problem we have in OS4 right now is more a lack of best-practise when it comes to the creation and distribution of .so files.

[Karlos]

Well, since I now have to use C++ for my microcontroller hobby, I have come tothe conclusion that, for high level App development I much prefer Obj-C... But low-level stuff is without question much better done in C++, objective C really wouldn't make much sense there at all!

Give it time. You will eventually realise, as I did, that it's all, without question, much better done in C++

Seriously though, you're right. C++ was always intended to be a superset of C and not intended to in any way reduce the language's suitability for system level programming, rather it was intended to make it more suitable for large scale application development.

[Karlos]

This is not something inherently connected to .library style libraries but to how the loading/caching is currently implemented in exec.

That's a given, but it is implemented the way it is implemented. You can't have two slightly different versions of the same .library open concurrently, regardless of where you put them on your HD.