More about why SObjs: isn't a great idea

[Matt_H]

I've just finished solving the OS4 equivalent of an episode of DLL hell. An issue in either DOpus 4.18.22 or elf.library 53.12 attempts to open libbz2.so.1.0, which doesn't exist on a 4.1u2 installation with the latest SDK (there's libbz2.so.1.0.4 instead), resulting in 2 highly uninformative error requesters. I was finally able to track down the culprits with the help of a Snoopy log file. xadmaster.library might be involved as well.

This is what happens when version strings are embedded in filenames and then hardcoded into programs. Too many files to keep track of, and they're easily lost during minor upgrades.

Has anyone else experienced this? Or, has anyone ever experienced anything like this with the traditional .library system?

[SamuraiCrow]

It's up to the app to make meaningful error messages if LoadLibrary returns NULL.  If it just exits with no error message, you're back to using snoopy to find which library failed.

[Piru]

You didn't even run into the fun scenario: Two incompatible SObjs with the exact same name, but different ABI (this has happened at least once already). There hasn't been proper documentation about SObj creation and this has lead into all sort of trouble.

They're worse than amiga shared libraries anyway: SObjs aren't even shared, each opener gets their own copy in memory.

As far as I can tell the only reason they exist is that apparently OS4 coders are incapable of creating amiga shared libraries with per caller library base (and the static data in the library base possize). Or they're just too lazy to do it properly, and decided to take the shortcut by introducing the non-shared libraries.

A grave mistake if you ask me.

[wawrzon]

incorporating version numbers into lib names seems to be a bad habit taken over from the systems incapable to provide version information ín another way.

[Matt_H]

As far as I can tell the only reason they exist is that apparently OS4 coders are incapable of creating amiga shared libraries with per caller library base (and the static data in the library base possize). Or they're just too lazy to do it properly, and decided to take the shortcut by introducing the non-shared libraries.

I think the rationale was for easy quick-n-dirty porting (which I'm also not fond of), but I'm thoroughly baffled as to why these well-established native programs are affected. OS4 xadmaster dates from December 2006, which I thought was before SObjs: was introduced. And I can't see why DOpus would need libbz2.so on its own.

incorporating version numbers into lib names seems to be a bad habit taken over from the systems incapable to provide version information ín another way.

Absolutely agreed. That's not a practice we should be emulating.

Compiling these ported libraries into programs statically would be a cleaner solution.

[Karlos]

To my mind, the biggest problem with shared objects is that they suffer from NIH. However, since we all decided ELF was a good idea, it's not entirely unreasonable for someone to consider the linkable part of the specification.

You didn't even run into the fun scenario: Two incompatible SObjs with the exact same name, but different ABI (this has happened at least once already). There hasn't been proper documentation about SObj creation and this has lead into all sort of trouble.

Yeah, it was a pain back when most people were migrating from gcc 2.95, the ABI changed a great deal. However, one of the reasons it changed so much was to avoid having to do it again for a long, long time. Don't expect to see changes of that magnitude any time soon. Unless C++0x throws a big spanner in the works. Not impossible...

They're worse than amiga shared libraries anyway: SObjs aren't even shared, each opener gets their own copy in memory.

Sure, on AmigaOS, but not for OS that support .so files generally.

As far as I can tell the only reason they exist is that apparently OS4 coders are incapable of creating amiga shared libraries with per caller library base (and the static data in the library base possize). Or they're just too lazy to do it properly, and decided to take the shortcut by introducing the non-shared libraries.

The only reason? I think that's being deliberately short sighted. My guess would be that one really obvious reason would the fact that traditional amigaos shared libraries offer no way to provide the required linker constructs for C++ and other languages that some people, myself included, like to use. The world is not tied to procedural programming.

If someone wrote an improved .library specification and loader that allowed things like C++ symbol export it would surely have problems too and at least we know what .so's particular problems are.

A grave mistake if you ask me.

The only alternative that provides the same support is static linking. Assuming we disregard the increase in disk storage (which is cheap), one disadvantage of static linking is that you don't get any benefits of a new version of a library without re-linking your app. Obviously, any major change to said library, by definition, probably requires a recompile of the application, so no problem there. On the flipside, provided the exported symbol table doesn't change, there shouldn't need to be any such change. Theoretically, I should be able to swap out some generic .so with say an altivec optimised one, for example and all client users of that library get the same benefit.

[Karlos]

I think the rationale was for easy quick-n-dirty porting (which I'm also not fond of), but I'm thoroughly baffled as to why these well-established native programs are affected. OS4 xadmaster dates from December 2006, which I thought was before SObjs: was introduced. And I can't see why DOpus would need libbz2.so on its own.

Agreed, that is pretty weird. A .so having a dependency on a .library is fine, the converse doesn't really seem reasonable when viewed on the "nativeness" scale in which .library is obviously further up the chain than .so.

Compiling these ported libraries into programs statically would be a cleaner solution.

Personally, I dislike static linking, but that's just me. It does make it easier for the end user, but then they always complain about the bloat of statically linked executables and the fact that statically linked application X doesn't support feature X that exists in application Y which happens to have been linked against a newer library

[Piru]

Yeah, it was a pain back when most people were migrating from gcc 2.95, the ABI changed a great deal. However, one of the reasons it changed so much was to avoid having to do it again for a long, long time. Don't expect to see changes of that magnitude any time soon. Unless C++0x throws a big spanner in the works. Not impossible...

Well actually what I was trying to say that there were two different libraries with different parameters, but the same name. Clearly this didn't end up well.

Sure, on AmigaOS, but not for OS that support .so files generally.

Obvously I was only talking about the shoddy SObjs implementation of AmigaOS here.

The only reason? I think that's being deliberately short sighted. My guess would be that one really obvious reason would the fact that traditional amigaos shared libraries offer no way to provide the required linker constructs for C++ and other languages that some people, myself included, like to use. The world is not tied to procedural programming.

C++ must die anyway

[Matt_H]

Personally, I dislike static linking, but that's just me. It does make it easier for the end user, but then they always complain about the bloat of statically linked executables and the fact that statically linked application X doesn't support feature X that exists in application Y which happens to have been linked against a newer library

Yeah, in the past I have complained about the bloat of static linking, but as I think about it now, maybe it deserves more consideration. Ease of use for the end user used to be of paramount importance on the Amiga. And a static program is a product of the tools available at the time it was compiled. If the tools improve, recompiling the program to take active advantage of those tools might encourage continued developer involvement.

[Karlos]

C++ must die anyway

Get away with ye' , y' love it really :-)

Well, I do, and that's all that matters from this direction

[Fats]

And a static program is a product of the tools available at the time it was compiled.

This thus also means that if a security flaw in one of the libs that are linked to a lot of programs, they need to be relinked and redistributed to the user.
I know, mentioning security and Amiga in the same context does currently not make much sense.

greets,
Staf.

[Karlos]

Personally, I'd love to see some native implementation supported by a decent compiler that would allow traditional shared libraries to provide the required services that .so files provide as far as languages beyond C are concerned.

However, since nobody has done anything like that (and it would not be a trivial undertaking), supporting the existing ELF shared object at least grants the developer some flexibility. Furthermore, it's not as alien a concept as it seems, seeing as we're using ELF already.

The earlier observation that a pair of identically named .so files implementing different ABIs being a uniquely bad problem seems a little contrived. After all, the existing .library mechanism doesn't prevent this kind of lunacy either. I could create a foo.library that is nothing to do with the existing foo.library and as long as the meta data is acceptable, the OS will happily pass it to clients that wanted the original, only to invoke meaningless vectors and crash.

[Matt_H]

This thus also means that if a security flaw in one of the libs that are linked to a lot of programs, they need to be relinked and redistributed to the user.
I know, mentioning security and Amiga in the same context does currently not make much sense.

greets,
Staf.

True, there is that risk. But I'll argue that we have a) security through obscurity, and b) that TCP/IP through a single system library offers an additional inherent layer of security. For example, look how much trouble we have getting Samba daemons and other servers working properly on home networks, let alone across the web

And if a developer needs to recompile to fix a security hole, maybe they'll also take the time to squash bugs or add new features.

The earlier observation that a pair of identically named .so files implementing different ABIs being a uniquely bad problem seems a little contrived. After all, the existing .library mechanism doesn't prevent this kind of lunacy either. I could create a foo.library that is nothing to do with the existing foo.library and as long as the meta data is acceptable, the OS will happily pass it to clients that wanted the original, only to invoke meaningless vectors and crash.

Also a possibility, but with particularly well-coded programs you can put necessary .library files in PROGDIR: or PROGDIR:Libs instead of SYS:Libs. Granted, that doesn't make it "shared" anymore, but you can have as many different incarnations of foo.library as you have programs that use it. I think MorphOS checks all these places (and more) for every library it attempts to open - Piru can correct me here. This sort of arrangement for .so files would be an improvement.

[Golem!dk]

Also a possibility, but with particularly well-coded programs you can put necessary .library files in PROGDIR: or PROGDIR:Libs instead of SYS:Libs. Granted, that doesn't make it "shared" anymore, but you can have as many different incarnations of foo.library as you have programs that use it.

Mhh... well... if you don't do that multitasking thing, sure. And maybe remember to expunge the library once your program is done with it.

[xeron]

Also a possibility, but with particularly well-coded programs you can put necessary .library files in PROGDIR: or PROGDIR:Libs instead of SYS:Libs.

Sobjs have the same mechanism, IIRC.