Weird, C string. Crashes on m68k, works on MOS

[Georg]

before you consider switching to a different memory management scheme.

Btw, it sometimes does help to hack memory management functions ~"down" to lowest level, ie. AllocMem, FreeMem. For example AllocPooled -> AllocMem, FreePooled -> FreeMem (ignoring for a moment side effect like DeletePool no longer freeing all allocations still in the pool), but also malloc -> AllocMem or free -> FreeMem.

In case debugging tool (like original Mungwall?) otherwise does not see and monitor all the individual single allocations from the higher level, but only bigger chunks at lower level with multiple higher end allocations embedded in chunks.

[olsen]

Btw, it sometimes does help to hack memory management functions ~"down" to lowest level, ie. AllocMem, FreeMem. For example AllocPooled -> AllocMem, FreePooled -> FreeMem (ignoring for a moment side effect like DeletePool no longer freeing all allocations still in the pool), but also malloc -> AllocMem or free -> FreeMem.

In case debugging tool (like original Mungwall?) otherwise does not see and monitor all the individual single allocations from the higher level, but only bigger chunks at lower level with multiple higher end allocations embedded in chunks.

You may be able to obtain more information about the problem in your program by replacing the memory allocation functions it uses. Every memory allocation and deallocation must be followed up by a consistency check, but that won't necessarily catch misbehaving code while it's trashing memory beyond an allocation boundary. You can get lucky, though. Figuring out what went wrong becomes a little bit easier, but not that much easier.

You can raise the level of this consistency and boundary checking to the operating system level through MungWall of old or Wipeout, but you will invariably get a lot of noise mixed in with the information you are after. The system will also become a lot slower due to the consistency checking and "wipe after allocate" and "wipe after free" processing that is being done, now that it's happening on the global level and not just in your own program.

One reason why MungWall was written was to catch cases in which memory was used after it was freed, or the contents of memory just allocated were used under the assumption that its contents were well-known. Discovering that memory contents were trashed was a side-effect of this.

Your compiler runtime library may feature built-in memory debugging tools. I specifically built those into my own clib2 (https://github.com/adtools/clib2). True story: not even these tools helped me to track down the one big/tiny bug which kept Samba 2.0.7 (which was the reason why clib2 was created in the first place) from working properly. The bug was in strtok(), and I only managed to fix it back in October 2004. It had been in there for more than two years, and the effects of it misbehaving were not getting caught by the memory debugging tools built into clib2.

[psxphill]

I could partly narrow it down to allocating dynamic memory instead of using stack data.

It's likely that you've got a buffer overrun and you've changed from corrupting the stack to corrupting the heap & it just happens that it can tolerate the heap becoming corrupt.

You need to put it back so that it crashes and then dismantle the program so that it does less until it stops crashing. There are times when I've not been able to spot my dumb mistakes, so I just throw away the code and start again.

In the 90's I knew someone who went days trying to figure out why he was getting corrupt memory and in the end he had mixed up some strcmp and strcpy calls.

[olsen]

It's likely that you've got a buffer overrun and you've changed from corrupting the stack to corrupting the heap & it just happens that it can tolerate the heap becoming corrupt.

Some explanation: "heap" is a Unix term for the memory your program can manage through functions such as malloc() and free(). By comparison, "stack" is storage space available to your program which does not need to be managed by malloc() and free(). You define local variables in your program's functions, and those come from the "stack".

You need to put it back so that it crashes and then dismantle the program so that it does less until it stops crashing. There are times when I've not been able to spot my dumb mistakes, so I just throw away the code and start again.

Easier said than done  It's possible that you might just repeat the same mistake again.

There is some value in trying to understand what the problem actually is, although at the time it will feel like some the worst spent minutes or hours of your life

In the 90's I knew someone who went days trying to figure out why he was getting corrupt memory and in the end he had mixed up some strcmp and strcpy calls.

Tell me about it... The 'C' runtime library design is as much a double-edged sword as the language itself. Scores of books have been written about how not to fall into the traps you unknowingly set for yourself by using 'C' (one of the first books being "'C' traps and pitfalls" by Andrew Koenig).

From my experience, the books do have value (e.g. Andrew Koenig's book details defensive programming measures which help you to cope with the side-effects of the language design which are unavoidable), but in the end remembering all the likely and some of the possible ways you should avoid is a burden.

What helps?

Learn how the 'C' language design may trip you up, because while you may be able to build a better set of functions and data structures for managing strings (for example), you cannot do the same with the 'C' language.

The 'C' programming examples and literature you are using to learn 'C' programming might be referencing programming practices which are long obsolete. It makes for more compact example code which is easier to understand, but that comes with a cost. To use functions such as sprintf(), strcpy(), strcat() or memcpy() in your program may seem the obvious choice, but they are not. All of these functions are unsafe to use because you have to be acutely aware of how much data they write to the destination: they don't know when to stop, or why to stop.

I rewrote my "term" application (http://aminet.net/comm/term/term-main.lha) in around 1995 to use snprintf(), strlcpy(), strlcat(), memmove(), etc. It was quite the humbling experience to learn how these changes improved the overall stability of the program. Bugs came to light which were impossible to spot, because they were caused indirectly by buffer overruns and memory corruption.

Finally, it may make sense to write your own little library of functions which do in about what the 'C' runtime library takes care of, but which gives you more control and insight into how they are being used. For example, the 'C' runtime library performs little to no sanity checking on function parameters. You could write your own functions which does. The 'C' runtime library contains many functions whose parameter order is inconsistent (e.g. first parameter may be the input, or it may be the last parameter, or the second). You could write your own functions which are more consistent.

There are reasons for sticking with the 'C' runtime library, such as that it is likely to be very well-tested and almost free of bugs. But design decisions made in the 1970'ies, leading to lack of consistency and lack of sanity checking, are not the kind of "bugs" which a well-tested runtime library will resolve. You might be better off making your own choices.

[asrael22]

So, thanks for your answers.
I'm now trying to add more fine grained test options, like allowing each .c file execute the main() method with test options.
Hope to have some more time on the WE.

Manfred

[asrael22]

So I've narrowed it down to this one, which is weird:

Code: [Select]

char* ConvertString2HexString(char *string, long len, char *outHex) {
printf("string len: %ld\n", len);
return string;
}

int main(void) {
char foo[] = "This is my way to say hello!";
long len = strlen(foo);
printf("len of foo: %ld\n", len);
char hexOut[len*2+1];

ConvertString2HexString(foo, len, hexOut);

return 0;
}


This code when compiled on Amiga with VBCC 0.9f gives me this output:
len of foo: 28
string len: 7168


Compiling this on macOS (on some C compiler which is used by Xcode) I get:
len of foo: 28
string len: 28


So, what the heck?

Obviously when using this string len in real it goes far beyond the real string length and overrides some other memory.


Manfred

[asrael22]

Compiling this with GCC 2.95 (which doesn't support C99, so I've swapped the printf line later in the source).
I'm, also getting:
len of foo: 28
string len: 28


So WTF, is there something wrong with VBCC?

[guest11527]

Code: [Select]

   char foo[] = "This is my way to say hello!";


While correct, it is probably worth noting what it does: a) it creates an array of characters, and b) it copies the string from the right into the array just created. While this might be just what you want, let me note that this is different from

Code: [Select]

const char *foo = "This is my way to say hello!";
which omits the copy and just keeps the pointer to the string. Note that the type of the right hand side is "const char[]", not "char []".

Code: [Select]

   char hexOut[len*2+1];


This creates a VLA, which is a language element that came in at C99. Not all compilers support VLAs, most notably SAS/C does not - it only supports C90. You also make a definition in the middle of a statement block, which is another extension.

If you want to be compatible to C90, the only chance is to allocate the object dynamically:

Code: [Select]

hexOut = malloc(len * 2 + 1);
and declare "hexOut" as "char *hexOut" along with "foo" above.

 

So, what the heck?

Well, as you just found out, VBCC does not support VLAs correctly, even less so if they are defined in the middle of a function block. Yes, it's a violation of the C99 standard, so a compiler defect. However, as stated above, I would avoid C99 features in first place in Amiga programs as only few compilers support them, and even less support them correctly, as you have just found out. (-:

What most likely happened is that VBCC got completely lost in its stack frame, i.e. the stack offset of "len" changed in the middle of the function, apparently an effect the code generator did not properly take care of.
%&$#?@!%&$#?@!%&$#?@!%&$#?@! happens if you depend on exotic language features.

[asrael22]

While correct, it is probably worth noting what it does: a) it creates an array of characters, and b) it copies the string from the right into the array just created. While this might be just what you want, let me note that this is different from

Code: [Select]


const char *foo = "This is my way to say hello!";

which omits the copy and just keeps the pointer to the string. Note that the type of the right hand side is "const char[]", not "char []".

I just want a string. Then I guess the pointer version is better suited.
This was just to create a snipped to reproduce the error.
In the real version that string comes from the serial device.

My pure C programming is more than 10 years ago. Since then I developed in higher abstracted languages.
So I have to adapt a bit to the C details.

Well, as you just found out, VBCC does not support VLAs correctly, even less so if they are defined in the middle of a function block. Yes, it's a violation of the C99 standard, so a compiler defect. However, as stated above, I would avoid C99 features in first place in Amiga programs as only few compilers support them, and even less support them correctly, as you have just found out. (-:

What most likely happened is that VBCC got completely lost in its stack frame, i.e. the stack offset of "len" changed in the middle of the function, apparently an effect the code generator did not properly take care of.
%&$#?@!%&$#?@!%&$#?@!%&$#?@! happens if you depend on exotic language features.

I guess I'll report that to Frank Wille and/or Volker Barthelmann.
VBCC only supports a subset of C99, maybe I ran into something that isn't supported.

Since I'm now basically back porting this program from MorphOS where you have GCC 4 and 5 it might be a good idea to stick to C90 to have as much code combatibility as possible between the Amiga OS flavours.


Manfred

[guest11527]

I just want a string. Then I guess the pointer version is better suited.

Both are "strings" (C doesn't really have strings), though the first is a "char []", hence can be modified, whereas the second is a "const char *", and hence cannot be modified (note the "const"). Or rather, "should not be modified", and if you try, strange and wonderful things may happen. On the Amiga, most likely nothing will happen, though, and you probably will not notice the difference. It's more a "Mr. Language Lawer" argument on this particular machine. On Linux, you'll get a core dump, though, as soon as you'd try to write to the const array (aka "string").

I guess I'll report that to Frank Wille and/or Volker Barthelmann.

Please do. In this particular case, VLAs are probably a hassle to support since the stack offsets of the objects with function scope are no longer constant, so it's easy to understand why that fails.

[psxphill]

Code: [Select]

char hexOut[len*2+1];

It looks like a compiler bug. One of the compilers I have to support didn't work with variable length arrays until very recently, I still avoid them.

I never used vbcc, I always stuck to gcc 2.95 and sas/c. But vbcc is still supported, so you may be able to get it fixed. It's a pity there isn't a modern gcc or clang build targeting 68k.

"Please support vbcc by contacting the authors if you find any bugs or problems. Supporting eight different architectures makes testing extremely time consuming, so this release is probably not free of bugs.

For problems with the compiler core contact Dr. Volker Barthelmann (vbemail), and for Amiga/Atari-specific problems, including assembler, linker, startup-codes and linker-libraries, contact Frank Wille (fwemail)."

[Tygre]

Thanks, I'll try those tools.
Manfred

Hi all!

Me too
Also, I had compiled a list of other tools for debugging/preventing memory problems in C here. In particular, I love Fortify and use it in all my programs now, it is really excellent at catching (some) problems!