Amiga.org

Operating System Specific Discussions => Amiga OS => Amiga OS -- Development => Topic started by: Jose on December 22, 2006, 01:58:08 PM

Title: Counting on AllocVec's size memo..
Post by: Jose on December 22, 2006, 01:58:08 PM
I was thinking on ways to keep track of a structs size and reminded that AllocVec keeps the size of allocated data in the pointer it returns - 1 IIRC. Probably a bit of a hack couting on it or it's not gonna change in the foresenable future, what do you guys think ?
Title: Re: Counting on AllocVec's size memo..
Post by: koaftder on December 22, 2006, 02:12:11 PM
OS4 broke it.

AmigaOS 2.0 to 3.9 and MorphOS have the size at -4.

[EDIT] Oh dear, this really is me, Piru.

Wayne: What the heck is going on? :-) [/EDIT]
Title: Re: Counting on AllocVec's size memo..
Post by: Jose on December 22, 2006, 02:15:31 PM
- 4 LONGs or BYTES ?

Where is the OS4 one ? Does it have some function to get the size (2 maintain future compatibility) ?
Title: Re: Counting on AllocVec's size memo..
Post by: koaftder on December 22, 2006, 02:24:17 PM
Dang it, too bad Piru noticed his session was screwed up, I almost got to look like I know what i'm talking about for once (:
Title: Re: Counting on AllocVec's size memo..
Post by: koaftder on December 22, 2006, 02:26:50 PM
bytes.

(and again this was Piru)
Title: Re: Counting on AllocVec's size memo..
Post by: Jose on December 22, 2006, 02:27:27 PM
@Piru
Screw it, post as Koafder anyway and put a comment in the beggining if you want or something...

Title: Re: Counting on AllocVec's size memo..
Post by: Jose on December 22, 2006, 02:28:03 PM
Just for now of course...
Title: Re: Counting on AllocVec's size memo..
Post by: tboeckel on December 22, 2006, 02:50:11 PM
OS4 didn't break it!!

Please, *NEVER* rely on such "facts" that hold for OS2/3 or MOS just by pure luck.

It was never documented that AllocVec() stores the size of the allocation at a specific position or at all.

OS4's memory system doesn't need to store the allocation size at all, because its internal structures already cover the size information. Hence it is illegal to peek at ptr-4.

If your application *really* needs to know the size of an arbitrary allocation, then write your own wrapper functions to AllocMem()/FreeMem() and store the allocation size yourself. That will save you *very* much trouble in the future.
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on December 22, 2006, 03:13:17 PM
OS4 developers choose to change this with no good justification. There is no urgent need to REMOVE the size from the ptr-4, it could well have been kept there for backwards compatibility alone. OS4 developers did not do this, and as a result some existing applications ceased to work.

I call that breaking.

Naturally applications relying on this are broken aswell, but there was no need to reduce the compatiblity in this way.
Title: Re: Counting on AllocVec's size memo..
Post by: Jose on December 22, 2006, 03:19:32 PM
Hi. It seems that I won't be using this as allocated size is obviously different from object size a whole bunch of times, as pointed by Rogue (I asked if there was some OS4 way of knowing the size in AW.net).

Title: Re: Counting on AllocVec's size memo..
Post by: Karlos on December 22, 2006, 04:02:46 PM
The only thing one should ever depend on is that the memory allocation system is a black box about which you should never assume anything other than what is officially documented.

This is true for every operating system. Making assumptions of this kind is bad development. When you allocate storage for x bytes, you are only guarenteed that you have at least that much; it could be more.
Title: Re: Counting on AllocVec's size memo..
Post by: tboeckel on December 22, 2006, 04:36:46 PM
How can you break something that was never documented? Peeking unallocated memory region was very common practice on 68k system, but will surely crash with OS4. So, does OS4 break this, too?

Ok, the location of AllocVec()'s size storage has become something like a "fact". But that's the point: it never really was a fact.

I think the Frieden brother told more than once, that the size at ptr-4 was removed for various good reasons:
1. it was never documented to be there
2. it was no longer necessary with the new memory system
3. PPC cache lines have different alignment than 68k cache lines and an additional offset of 4 would slow things down
4. so far it did only break one major application (GoldED) and that has been fixed already
5. OS4 tries to enforce certain programming rules and punishes hacking.

The advantages of removing the size information by far outnumbered the disadvantages. And that's why it has been done. Compatibility is a big thing to maintain, but keeping undocumented features, which lead to hacking, is not always very wise.

If MOS could keep up the compatibility to OS3.x's AllocVec() without sacrificing any advantages then its perfect legal to keep this. But relying on undocumented features should never be encouraged.
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on December 22, 2006, 05:03:17 PM
Quote
How can you break something that was never documented?

By changing the implementation for no other reason than to crash broken applications. That is not very smart.

Quote
1. it was never documented to be there

True. But the implementation is widely known, and abused in way more applications than just GoldEd.

Quote
2. it was no longer necessary with the new memory system

This is no reason to remove the compatibility.

Quote
3. PPC cache lines have different alignment than 68k cache lines and an additional offset of 4 would slow things down

Slow down? Yeah right, that must be massive slowdown there. *cough*

Quote
4. so far it did only break one major application (GoldED) and that has been fixed already

That they know about. That is the thing with these changes: you never know when some obscure application breaks, and you have absolutely no way to test everything. And if some application suddenly crashes with new system update, how can you tell why it happens? You can't. Thus, it is a bad idea to do such modifications for no good reasons.

Quote
5. OS4 tries to enforce certain programming rules and punishes hacking.

Now we're getting closer to the truth. The change here makes no sense at all, all the above reasons are not good enough to remove the size at offset -4. IMO changes for the sake of the change itself, or trying to enforce new rules while breaking old applications is a very bad idea.

Similar excersise was "cleaning up" some dos.library return codes (-1 vs 1 for "true"). Good intentions turned out badly, as it broke varions programs checking for "traditional" return values.

Quote
The advantages of removing the size information by far outnumbered the disadvantages.

No it doesn't. Removing the size gives no apparent advantages, but instead makes some applications crash. That is NOT good for the end user.

Quote
And that's why it has been done.

I don't even try to understand why it has been done, there doesn't seem to be any sensible reason for it. If those indeed are the reasons for removing it, I fear OS4 won't run many legacy applications in the end. The compatibility will get worse and worse.

Quote
Compatibility is a big thing to maintain, but keeping undocumented features, which lead to hacking, is not always very wise.

So basically you're saying it's better to make old, existing applications to crash to make sure new programmers don't fall into the old traps? Wouldn't be smarter to provide good programming manuals and encourage good programming practices, rather than making the OS itself crash the programs? Remember, this is not helping the end user. Development tools should indeed make such hacks barf, and this is what tools like Mungwall do.

Quote
But relying on undocumented features should never be encouraged.

Agreed. But deliberately breaking compatibility to force the issue isn't the way either. Provide good development tools, and leave the end user out of the loop.
Title: Re: Counting on AllocVec's size memo..
Post by: tboeckel on December 22, 2006, 05:43:26 PM
I think you perfectly know how beginners start programming. Take a program with source and see how other people solved certain problems. Sometimes you look into the Autodocs, but most of the time you get inspriration from "outside". At least that's the way I started programming many years ago. And if a beginner learns from such hacking it is sometimes very hard to convince this person from the oposite.

Good programming manuals are always welcome, but the biggest programming manual are available source codes, even if they are not perfect.

I don't say "let all the old applications crash and burn". I say "keep (wrong) compatibility as long as it is bareable, but sacrifice it if that would prevent really necessary improvements". And the new memory system really is a big step forward!

Surely it is not ok to sarcifice compatibility just to make old programs crash, but during all the beta testing of OS4 GoldED really was the *only* bad application. I think the Friedens would have decided the other way if more than this single program had been affected. But that was not the case.
Title: Re: Counting on AllocVec's size memo..
Post by: itix on December 22, 2006, 05:44:45 PM
Quote

5. OS4 tries to enforce certain programming rules and punishes hacking.


Still many OS4 programs depend on undocumented side effects and features ;^) It is nature of programming. Coders always did and will use all means to reach goal with minimal effort. If it is possible to exploit an undocumented feature there is always someone doing so.
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on December 22, 2006, 05:49:00 PM
Quote
I don't say "let all the old applications crash and burn". I say "keep (wrong) compatibility as long as it is bareable, but sacrifice it if that would prevent really necessary improvements". And the new memory system really is a big step forward!

While not commenting the qualities of the new memory system, nothing in it required dropping the backwards compatibility here. And this is my point.

Quote
during all the beta testing of OS4 GoldED really was the *only* bad application.

It was? According to who? How broad set of applications were tested? How was it made sure that all memory allocations were tested in each app?

Quote
I think the Friedens would have decided the other way if more than this single program had been affected. But that was not the case.

Well, if they really think they caught the only program abusing this trick, they are gravely wrong.
Title: Re: Counting on AllocVec's size memo..
Post by: itix on December 22, 2006, 06:01:42 PM
Quote

Ok, the location of AllocVec()'s size storage has become something like a "fact". But that's the point: it never really was a fact


This could be compared to how some OS4 programs handle r2 register. AmigaOS 4 uses SysV ABI and according to specification general purpose register r2 is reserved for system. However it is publicly known that r2 is not used by the system at all. Thus using r2 as a spare register in assembler routines is safe and I know OS4 native programs doing so. But this is not stated in development documents, "everyone" just know this feature.

Is it bad? I don't know. In MorphOS, killing r2 is illegal, but os4emu (;-)) occasionally abuses the system and kills r2 against all rules :-)

Title: Re: Counting on AllocVec's size memo..
Post by: weirdami on December 22, 2006, 06:09:43 PM
Quote
Naturally applications relying on this are broken aswell, but there was no need to reduce the compatiblity in this way.


People made the same kind of complaint when AGA came out because lots of games stopped working.
Title: Re: Counting on AllocVec's size memo..
Post by: Karlos on December 22, 2006, 06:19:56 PM
"Killing" any register not known to be defined as volatile on any specification is bad. Do what the hell you want to a register in your own code but be sure to save/restore anything not volatile.

Unless you are a bad coder of the type that makes dangerous assumptions about everything you come across.

I think there's a touch of melodrama in this thread. Some applications may break because they were badly written or had at least one majorly bad assumption.

Any code that makes these kind of assumptions will eventually fall over as the system moves on. There are applications that bail on every major operating system at some stage due to their age. Saying this is always the fault of the OS is a bit unfair when it is down to the actions of the developer.

Would you blame the C/C++ compiler's compatibility here or the developer if we substituted the word "AllocVec" with "malloc" or "operator new[]()" ?


If we look at the issues in context for a moment. There are apps that won't work. If they are very old, the chances are the services they provide have been replaced by newer applications. If they are new apps, the chances are they can  be fixed.

Similar problems with compatibility existed moving from 1.3 to 2.0 and that was a much smaller technical step than the changes from 3.9 to OS4 / MOS.
Title: Re: Counting on AllocVec's size memo..
Post by: itix on December 22, 2006, 06:56:32 PM
Quote

"Killing" any register not known to be defined as volatile on any specification is bad. Do what the hell you want to a register in your own code but be sure to save/restore anything not volatile.


r2 is quite exceptional: it is either used or not. Killing r2 is not very expectional either. It is just about is it good coding style or not - while using r2 for own purposes is maybe deprecated its functionality is known (i.e. unused in OS4).

Quote

Would you blame the C/C++ compiler's compatibility here or the developer if we substituted the word "AllocVec" with "malloc" or "operator new[]()" ?


AllocVec() stayed the same last 15 years. Memory lists have existed last 20 years. I'd say API was rock solid many years...

Quote

Similar problems with compatibility existed moving from 1.3 to 2.0 and that was a much smaller technical step than the changes from 3.9 to OS4 / MOS.


In those times software was developed actively and companies could not let their software to fail under 2.0 especially because productivity users went for 2.0. Today I dont see software companies (what companies?) recompiling their software for MOS and OS4.

Gamers despised 2.0 and A500+ was considered being bad buy because of poor compatibility. A1200 users could not play many games until proper tools (WHDLoad and JST) were invented and games patched.

Of course something must be sacrificed always. I know why Diavolo Backup 68k doesn't work fully in MOS and OS4 and it is only because coder did extremely small mistake in threading code. It could be fixed by small adjust in the DOS side though.
Title: Re: Counting on AllocVec's size memo..
Post by: Karlos on December 22, 2006, 08:24:58 PM
@itix

It appears you missed my point. Regardless of how long AllocVec() behaved the way it did and regarless of how long memory lists are stored it is *not* within the developers remit to assume anything about it.

A given compiler may handle malloc() the same way. If it did, for years and years, would you regard it as acceptable practise to operate on the assumption and use it within your code? I certainly would not, under any circumstances, ever.

If you need to care about the allocation size, perhaps in some system with a lot of dynamic allocation going on, then you wrap the allocator routines yourself or use pools, you sure as hell don't go around poking into negative pointer offsets from the base address, regardless of how long it might have worked historically. You sure as hell wouldn't do it in ANSI-C/C++, there's no reason to do it when using exec either. People only get tempted to do this because most of the structures used by the OS are visible in one header or another and fail to appreciate that they are private to the system regardless.

The point has been raised that there was no need to do this and break compatibility and that for compatibility sake it could have been left alone. Regardless of wether this is true or not, I think criticising the OS in this case is a bit of a strawman attack. The real issue is that developers shouldn't habitually point loaded guns at their own feet and wonder why they might lose some toes one day ;-)
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on December 27, 2006, 09:54:46 PM
So, OS4 "Final Update" with the new memory system is out.

I suppose no important software has suddenly broken with it?
Title: Re: Counting on AllocVec's size memo..
Post by: platon42 on December 27, 2006, 10:21:36 PM
Quote

Piru wrote:
So, OS4 "Final Update" with the new memory system is out.

I suppose no important software has suddenly broken with it?


Uhm, you mean like FinalWriter?

http://amigaworld.net/modules/newbb/viewtopic.php?mode=viewtopic&topic_id=21726&forum=32&viewmode=flat&order=0
Title: Re: Counting on AllocVec's size memo..
Post by: Stevo on December 27, 2006, 10:37:46 PM
Come on, Final Writer '97 isn't that important: I'm sure AmigaWriter2 still works ;-)
Title: Re: Counting on AllocVec's size memo..
Post by: Karlos on December 28, 2006, 07:27:50 AM
I have to say, this is pretty bad :-(

I still stand by the view that the developer was at fault for assuming things about the underlying memory subystem. There's no need for it. Certainly I've never needed to. Feel free to prove me wrong on that count. For a major application, this practise is inexcusable.

Amiga developers need to understand what APIs are and when they must use them. Banging the metal or underlying OS implementation internals may have been acceptable in the 80's but things change.
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on December 28, 2006, 08:41:42 AM
@Karlos

Of course the developer of the app is at fault here, too.

But so are OS4 developers. In insisting on removing the compatibility (even if undocumented!) they've broken important application that worked just fine before. Again, there is no good reason for not to have the compatibility there. Removing it will only break apps.

Reading the local forum, SnoopDOS, AmiDisk, HDRec are all crashing.

QED
Title: Re: Counting on AllocVec's size memo..
Post by: whose on December 28, 2006, 12:24:21 PM
@piru:

>Reading the local forum, SnoopDOS, AmiDisk, HDRec are all
>crashing.

Neither of them do. SnoopDos68K runs in fact "better" than SnoopDosOS4 (you can't open the "Functions" window anymore).

That results from a nice "trick", doing some write accesses to const data sections. Nice, eh? SnoopDos68K doesn't break because 68K programs are still allowed to do this.

AmiDisk works fine, HDRec does so, too (in fact, now you can use certain functions to the extent, because Petunia runs even faster with Final, about 2 times faster).

What breaks now is FinalWriter, WordWorth7 is doing its Job fine here. Even TurboPrint works flawlessy here (with reset proof option disabled).

If one looks at the practice to rely on the memory size notation of AllocMem(), one must say: Why do you do this? It's "forbidden", there is no real problem to do own notation of the alloc'd size, just do it the "right" way.

There is no need to stay compatible here. If older applications break for this, update the application or make an alternative for it.

Greetz
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on December 28, 2006, 12:36:36 PM
Regarding crashing applications: Those apps do crash for the local OS4 user. I have no way to verify those findings.

Quote
If one looks at the practice to rely on the memory size notation of AllocMem(), one must say: Why do you do this? It's "forbidden", there is no real problem to do own notation of the alloc'd size, just do it the "right" way.

It was about AllocVec, but still: I agree fully. Programmers should use the correct way and avoid depending on the size being at -4. But they don't. There are such apps, and not all of them can be replaced.

Quote
There is no need to stay compatible here.

There is no reason to remove compatibility here.

Quote
If older applications break for this, update the application or make an alternative for it.

And if the source code is not available and the application is some important one? You seriously think everything can be replaced? AROS / OS4 / MOS doesn't have enough good developers to write replacement for everything.
Title: Re: Counting on AllocVec's size memo..
Post by: whose on December 28, 2006, 01:30:05 PM
@piru:

Quote
It was about AllocVec, but still: I agree fully.


Yes, my fault. Was in a hurry ;)

Quote

Programmers should use the correct way and avoid depending on the size being at -4. But they don't. There are such apps, and not all of them can be replaced.


Well, they "didn't". Nowadays developers should avoid even to ask for this. Well, they don't, I know. But where's the point to discuss the compatibility question for older programs, when a programmer actually asks for this OS private special? Newer applications should avoid such tricks for any price! There is no excuse to rely on such a "feature" nowadays.

Quote
There is no reason to remove compatibility here.


I bet there is. But I'm not "skilled" enough to judge or explain it. The Frieden brothers can explain it much better, I think.

I don't rely on the size notation of AllocVec() and I don't even ask for it. I simply don't need it. But there are enough "tricks" to achieve the goal in another, OS friendly way (the mentioned wrapper, just notice the size at the beginning of the wanted memory space + (type)1 and give back the address + (type)1. This is the simplest way, there are more clever ways to do this).

Quote

And if the source code is not available and the application is some important one? You seriously think everything can be replaced? AROS / OS4 / MOS doesn't have enough good developers to write replacement for everything.


Well, you don't have to replace everything. There are very few apps that break. Even for SnoopDos there is a replacement now. So, you don't need so much developers to write replacements.

But the will to do so is quite small in most cases. Mostly developers find it easier to e.g. ask for AllocVec() specials to invest lots of work to get such stunts compatible for all known systems. Not very clever, I think.

Quote
Regarding crashing applications: Those apps do crash for the local OS4 user. I have no way to verify those findings.


Yes, I know this. If it would be possible I would invite you here to Germany to see it working yourself.

Some other users had problems which I helped to solve (especially on AW). Much of their problems resulted from not obeying the installation instructions (clean install) or failure of 3rd party programs (especially IBrowse installing old MUI classes, but some older libraries made problems, too, which could have been avoided by using the newest versions, for example expat.library).

For HDRec you could ask the author, he obviously did some speed tests for HDRec with OS4 final and gives the same results I achieved.
Title: Re: Counting on AllocVec's size memo..
Post by: itix on December 28, 2006, 01:37:57 PM
Quote

Neither of them do. SnoopDos68K runs in fact "better" than SnoopDosOS4 (you can't open the "Functions" window anymore).

That results from a nice "trick", doing some write accesses to const data sections. Nice, eh? SnoopDos68K doesn't break because 68K programs are still allowed to do this.


Program overwriting its own const data is not illegal as such and in SnoopDos it happens only because some strings are unexpectedly placed into .rodata section. There is nothing wrong in SnoopDos 68k; bug was created in 68k->PPC transition. Only PPC version of SnoopDos (inluding MorphOS port) writes to const data.

Fixing const data problem does not give you working packet monitor though.

Quote

If one looks at the practice to rely on the memory size notation of AllocMem(), one must say: Why do you do this? It's "forbidden", there is no real problem to do own notation of the alloc'd size, just do it the "right" way.


Even OS4 developers do it. DiskSpeed OS4 assumes that memory returned from AllocVec() is aligned to 8 bytes boundary. It is not documented feature but is "public knowledge". Neither is documented what newlib startup code (one that is statically linked to newlib programs) is doing. Build hello world program and link it against newlib and you are already using undocumented features.
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on December 28, 2006, 01:44:59 PM
@whose
Quote
But where's the point to discuss the compatibility question for older programs, when a programmer actually asks for this OS private special?

Historical context, and implementation details. He did ask if it had changed, too.

Also, in some rare occasions such details might be needed (for example debugging tools).

Quote
I bet there is. But I'm not "skilled" enough to judge or explain it. The Frieden brothers can explain it much better, I think.

Well, nothing in their explanation is good enough reason to justify this change (and I consider myself qualified enough to comment on this). See earlier posts in this thread.
Title: Re: Counting on AllocVec's size memo..
Post by: itix on December 28, 2006, 01:54:26 PM
Quote

The real issue is that developers shouldn't habitually point loaded guns at their own feet and wonder why they might lose some toes one day


Is the real issue that Amiga developers in 80s and 90s were not very good afterall? :-) I mean, when old programs stop working when internals of the OS have changed is it because Amiga software is POS or Amiga operating system is POS? Or both? Or none? Obviously SW developers do mistakes (think about original IBrowse 2.3 issues in OS4) but if mistakes are common practise it is better fix the OS than software. The OS exists for the software.
Title: Re: Counting on AllocVec's size memo..
Post by: whose on December 28, 2006, 02:05:53 PM
@itix:

Quote

Program overwriting its own const data is not illegal as such and in SnoopDos it happens only because some strings are unexpectedly placed into .rodata section.


Well, I would it call illegal, if everybody should know that these sections are write protected (well, in fact they weren't until final, but NDK states it).

Quote
Fixing const data problem does not give you working packet monitor though.


But this is a complete other topic, isn't it? Or does it break because of the new memory system? I would think that this a question of correct OS function use.

Quote

Even OS4 developers do it. DiskSpeed OS4 assumes that memory returned from AllocVec() is aligned to 8 bytes boundary.


I see... hence it is a good practice to encourage such methods by refering to historical errors made and giving them a late "absolution"? I don't think so.

Show them the correct way, if you know about. Don't discuss old "tricks" and the compatibility break of these tricks a new designed OS gives or will give.

Damn easy.

Greetz
Title: Re: Counting on AllocVec's size memo..
Post by: whose on December 28, 2006, 02:13:02 PM
Quote

Historical context, and implementation details. He did ask if it had changed, too.


Ok. Well, but in this case it would have been sufficient to say: "this doesn't work anymore, it is not documented and therefore "forbidden" to use. It will break with OS4, don't use such a detail".

Quote

Also, in some rare occasions such details might be needed (for example debugging tools).


Maybe. But one should ask himself, if such tools would work with OS4 or other OS. If (perhaps or sure) not, search for another, working, way. There are ways to develop such tools in a friendly way, even for OS4.

I really don't understand why it is so important to use such tricks, which are already blamed decades before, sry.

Greetz
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on December 28, 2006, 02:30:32 PM
@whose
Quote
I really don't understand why it is so important to use such tricks, which are already blamed decades before, sry.

I am not promoting use of such tricks.
Title: Re: Counting on AllocVec's size memo..
Post by: whose on December 28, 2006, 02:40:35 PM
@piru:

Not directly, that is true. But you do it indirectly by saying, that e.g. MOS supports such a practice and hence is "more compatible" (or better). See the difference?

OS4 discourages such practices active and I can't believe that they (the OS4 developers) did it to just say "hurray, we broke more applications than any other amiga-like OS".

It's now time to say "good bye" to tricks like this one (and some other).

Greetz
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on December 28, 2006, 02:45:56 PM
@whose
Quote
But you do it indirectly by saying, that e.g. MOS supports such a practice and hence is "more compatible" (or better). See the difference?

No. I know AmigaOS 2.x, 3.x and MorphOS works this way. If I knew how AROS works, I would have included it aswell. I see you're reading more to my comments, though.

Quote
OS4 discourages such practices active and I can't believe that they (the OS4 developers) did it to just say "hurray, we broke more applications than any other amiga-like OS".

That's fine. MorphOS does the same, really, but it tries to avoid breaking things if only possible. In this case the size could have been left to ptr - 4 without any ill effects.

Quote
It's now time to say "good bye" to tricks like this one (and some other).

Yes, was that in question anyway? The fact is that old, existing applications use such tricks, and the OS should not be broken here (even though such tricks are "illegal" and "undocumented").
Title: Re: Counting on AllocVec's size memo..
Post by: whose on December 28, 2006, 03:00:53 PM
@piru:
Quote
No. I know AmigaOS 2.x, 3.x and MorphOS works this way. If I knew how AROS works, I would have included it aswell. I see you're reading more to my comments, though.


First: It was and is not meant as a personal offence, really.

Well, if I read more to your comments, why shouldn't (unexperienced) developers do? They will do it, and that's the problem. I spoke of "late absolution" given to such practices, this is an example.

Quote
Yes, was that in question anyway?


More or less.

Quote

The fact is that old, existing applications use such tricks, and the OS should not be broken here (even though such tricks are "illegal" and "undocumented").


Shouldn't it? Why? To support this tricks several decades more? To gain compatibility to broken software and discourage new developments in this direction?

I think that time will tell what the better way was. I don't see any real disadvantage for OS4 regarding the AllocVec() question.

Greetz
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on December 28, 2006, 03:11:11 PM
@whose

I've explained it already, but lets repeat once more:

Quote
Shouldn't it?

It shouldn't.

Quote
Why?

To avoid crashing old applications.

Quote
To support this tricks several decades more?

The only reason to have it here is to avoid crashing the apps that read the size from the ptr - 4. Nothing else.

Quote
To gain compatibility to broken software and discourage new developments in this direction?

Compatibility with old existing software. Debugging tools for the OS (or the debugging mode of the OS itself) should complain loudly if someone uses such tricks.

In fact, OS4 could have easily retained the "size at ptr - 4" for m68k apps (as far as I know). However, this was changed for both new and old apps. Why?
Title: Re: Counting on AllocVec's size memo..
Post by: Karlos on December 28, 2006, 04:07:10 PM
I wonder just how difficult it would be to add this value back to ptr-4 with the new system.

Ok it isn't clean, but it shouldn't be that hard to do. Just clone the value from wherever else it is stored.
Title: Re: Counting on AllocVec's size memo..
Post by: Doobrey on December 29, 2006, 03:13:16 PM
Quote

Karlos wrote:
I wonder just how difficult it would be to add this value back to ptr-4 with the new system.


I was thinking the same earlier, but not I'm not 100% sure how OS4 handles the 68k->PPC library calls so this is all a wild guess.

I assume ExecSG must have a 68k jump table for compatibility with old apps( which jumps to a wrapper that does the 68k->PPC register handling and calls the PPC native function etc)
So is it just a simple matter of loading new AllocVec and FreeVec routines to memory and using SetFunction to insert them into the jump table?

Then again there's probably a major wrong assumption there somewhere, and  I could be talking outta my backside
Title: Re: Counting on AllocVec's size memo..
Post by: Gilloo on January 05, 2007, 03:18:37 PM
Quote

Jose wrote:
I was thinking on ways to keep track of a structs size and reminded that AllocVec keeps the size of allocated data in the pointer it returns - 1 IIRC.


Yes.
Look at this piece of ugly code: I use it in V34 routines...
APTR MAllocVec( unsigned long byteSize, unsigned long requirements )
{
  if (SysBase->LibNode.lib_Version >= 36)
  {
    return AllocVec(byteSize, requirements) ;
  }
  else
  {
    unsigned long *data = NULL ;

    byteSize += sizeof(unsigned long) ;
    data = (unsigned long *)AllocMem(byteSize, requirements) ;
 
    *data = byteSize ;
    data++ ;
    return data ;
  }
}

void MFreeVec( APTR memoryBlock )
{
  if (SysBase->LibNode.lib_Version >= 36)
  {
    FreeVec(memoryBlock) ;
  }
  else
  {
    unsigned long byteSize ;
    unsigned long *data ;

    data = (unsigned long*)memoryBlock ;
    data-- ;
    byteSize = *data ;
    FreeMem(data, byteSize) ;
  }
}

Quote

Jose wrote:
Probably a bit of a hack couting on it or it's not gonna change in the foresenable future, what do you guys think ?

AllocVec/FreeVec are shorthand implementations to AllocMem/FreeMem functions. Internaly, AllocVec uses AllocMem, and FreeVec... FreeMem.
AllocMem/FreeMem uses Allocate/Deallocate and so on...
You can modify AllocVec/FreeVec (with SetFunction) as you want, and add what you want before the data pointer returned by AllocVec (Process name, hunk, Programm name...) to track memory allocations. This story of size stored before pointer returned by AllocVec relies only on ONE implementation!!
Title: Re: Counting on AllocVec's size memo..
Post by: n-ary on January 09, 2007, 08:40:26 PM
Quote

whose wrote:
Shouldn't it? Why? To support this tricks several decades more? To gain compatibility to broken software and discourage new developments in this direction?

Very good, as for a developer view.
But at the end of the day, isn't it how the end user experiences and appreciates the result what really matters?
The purpose of the whole endeavour?

So what does the end user get?

1. a working application + dirty code
 or
2. a crashing application + clean code + possibly some unperceivable performance gain?

Frankly, I see no way the option 2. could win, unless this is strictly a development system and "end users" are actually developers.
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on January 09, 2007, 09:00:24 PM
dupe (can be deleted if some mod runs into this)
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on January 09, 2007, 09:02:13 PM
MAllocVec will write zeropage if AllocMem fails, fixed version:
Code: [Select]
APTR MAllocVec( unsigned long byteSize, unsigned long requirements )
{
    if (SysBase->LibNode.lib_Version >= 36)
    {
        return AllocVec(byteSize, requirements);
    }
    else
    {
        unsigned long *data;

        byteSize += sizeof(unsigned long);
        data = (unsigned long *)AllocMem(byteSize, requirements);
        if (data)
        {
            *data = byteSize;
            data++;
        }
        return data;
    }
}


Small fix to MFreeVec aswell, in case this routine is to be used as drop-in replacement for FreeVec:
Code: [Select]

void MFreeVec( APTR memoryBlock )
{
    if (SysBase->LibNode.lib_Version >= 36)
    {
        FreeVec(memoryBlock);
    }
    else if (MemoryBlock)
    {
        unsigned long byteSize;
        unsigned long *data;

        data = (unsigned long*)memoryBlock;
        data--;
        byteSize = *data;
        FreeMem(data, byteSize);
    }
}


Excuse me, I just can't help myself. ;-)
Title: Re: Counting on AllocVec's size memo..
Post by: koaftder on January 09, 2007, 10:32:02 PM
I still can't get used to that leading '{' on its own line. Am I defective? Why do people do this?
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on January 09, 2007, 10:47:02 PM
Wikipedia: Indent Style#BSD/Allman style (http://en.wikipedia.org/wiki/Indent_style#BSD.2FAllman_style)

Basically it's easier to read: More "spacious" and it's easier to see where blocks start and end (just find  visually matching brackets).

This is not the universally superior indent style, but it is fairly common. However, the most important thing is to stay consistent: When you choose a style, stick to it. Don't change styles (at least not within the same project).

See more at Wikipedia: Programming style (http://en.wikipedia.org/wiki/Programming_style)
Title: Re: Counting on AllocVec's size memo..
Post by: koaftder on January 09, 2007, 11:09:34 PM
Quote

Piru wrote:
Wikipedia: Indent Style#BSD/Allman style (http://en.wikipedia.org/wiki/Indent_style#BSD.2FAllman_style)

Basically it's easier to read: More "spacious" and it's easier to see where blocks start and end (just find  visually matching brackets).

This is not the universally superior indent style, but it is fairly common. However, the most important thing is to stay consistent: When you choose a style, stick to it. Don't change styles (at least not within the same project).

See more at Wikipedia: Programming style (http://en.wikipedia.org/wiki/Programming_style)


I guess I'm in the "BSD/KNF" camp. I guess I need to get used to the Allman style. Allman isn't easier to read to me.... I'm doing a lot of windows stuff right now and its all some perverted allman style. Driving me nuts. Especially allan style with win32 calls with 1000 parameters blocked up on multiple lines.
Title: Re: Counting on AllocVec's size memo..
Post by: CannonFodder on January 09, 2007, 11:10:58 PM
Quote

koaftder wrote:
I still can't get used to that leading '{' on its own line. Am I defective? Why do people do this?


Hallelujah I am not alone in this world!

I cannot stand that.  It's ridiculous.

BSD/KNF style (http://en.wikipedia.org/wiki/Indent_style#BSD.2FKNF_style)

Much better.
Title: Re: Counting on AllocVec's size memo..
Post by: koaftder on January 09, 2007, 11:18:03 PM
Quote

CannonFodder wrote:

Hallelujah I am not alone in this world!

Much better.


you definately are not alone. I hate the single line entry '{' what a waste of space. The indent make it obvious what scope one is in. The trailing '}' is ok, it denotes down scope. Thats all you need.
Title: Re: Counting on AllocVec's size memo..
Post by: CannonFodder on January 09, 2007, 11:18:46 PM
Code: [Select]

void MFreeVec( APTR memoryBlock ) {
    if (SysBase->LibNode.lib_Version >= 36) {
        FreeVec(memoryBlock);
    }

    else if (MemoryBlock) {
        unsigned long byteSize;
        unsigned long *data;

        data = (unsigned long*)memoryBlock;
        data--;
        byteSize = *data;
        FreeMem(data, byteSize);
    }
}


Much nicer. ;-)
Title: Re: Counting on AllocVec's size memo..
Post by: koaftder on January 09, 2007, 11:19:49 PM
Quote

CannonFodder wrote:
Code: [Select]

void MFreeVec( APTR memoryBlock ) {
    if (SysBase->LibNode.lib_Version >= 36) {
        FreeVec(memoryBlock);
    }

    else if (MemoryBlock) {
        unsigned long byteSize;
        unsigned long *data;

        data = (unsigned long*)memoryBlock;
        data--;
        byteSize = *data;
        FreeMem(data, byteSize);
    }
}


Much nicer. ;-)


wow, I can actually read that.
Title: Re: Counting on AllocVec's size memo..
Post by: koaftder on January 09, 2007, 11:21:27 PM
Quote

CannonFodder wrote:
Code: [Select]

void MFreeVec( APTR memoryBlock ) {
    if (SysBase->LibNode.lib_Version >= 36)
        FreeVec(memoryBlock);
    else if (MemoryBlock) {
        unsigned long byteSize;
        unsigned long *data;

        data = (unsigned long*)memoryBlock;
        data--;
        byteSize = *data;
        FreeMem(data, byteSize);
    }
}



sorry, had to do that.
Title: Re: Counting on AllocVec's size memo..
Post by: CannonFodder on January 09, 2007, 11:43:54 PM
Quote

koaftder wrote:
Quote

CannonFodder wrote:
Code: [Select]

void MFreeVec( APTR memoryBlock ) {
    if (SysBase->LibNode.lib_Version >= 36)
        FreeVec(memoryBlock);
    else if (MemoryBlock) {
        unsigned long byteSize;
        unsigned long *data;

        data = (unsigned long*)memoryBlock;
        data--;
        byteSize = *data;
        FreeMem(data, byteSize);
    }
}



sorry, had to do that.


:lol:
Title: Re: Counting on AllocVec's size memo..
Post by: Piru on January 09, 2007, 11:52:28 PM
Oh no, yet another holy war... But of course it's a matter of taste....






Except that BSD/Allman is teh bestest!!!1






:-D
Title: Re: Counting on AllocVec's size memo..
Post by: koaftder on January 10, 2007, 12:14:40 AM
Quote

Piru wrote:
Oh no, yet another holy war... But of course it's a matter of taste....
Except that BSD/Allman is teh bestest!!!1
:-D


Might as well be, everybody is doing it. I'll have to learn to cope.
Title: Re: Counting on AllocVec's size memo..
Post by: CannonFodder on January 10, 2007, 01:02:21 AM
Quote

Piru wrote:
Oh no, yet another holy war... But of course it's a matter of taste....


Except that BSD/Allman is teh bestest!!!1

:-D


Infidel!!! ;-)