Weird, C string. Crashes on m68k, works on MOS

[asrael22]

Hi.

I've written a small cmd line utility on MorphOS.
It deals quite a lot with strings, strcpy, strcat, sprintf, etc.
I compile it with GCC on MorphOS. It runs fine there. Not only on quick tests, but running it for days.

On AmigaOS 3.9 however, the same sources, compiled with VBCC crash on a couple of places.
I could partly narrow it down to allocating dynamic memory instead of using stack data. But I'm not sure if it's that.

I've tried to create isolated tests for specific units. But I can't really figure out what's wrong.

Are there any debugging tools on AmigaOs I could give a try?
Are there any best practices for using stack memory vs. dynamic allocations (say for size of 1 to 32 kB)?
What about sprintf et al., it it save to use?



Manfred

[guest11527]

Are there any debugging tools on AmigaOs I could give a try?

MuForce for NULL-pointer references. MuGuardianAngel for out-of-bounds buffer accesses, PatchWork to detect illegal system calls, SegTracker to hunt down where the crashes are. And a serial terminal connected with a null-modem cable, configured at 9600-8-N-1. Or Sashimi to get the "hits" on the terminal.

Are there any best practices for using stack memory vs. dynamic allocations (say for size of 1 to 32 kB)?

An AmigaOs binary typically has only 4K of stack. Thus, whatever object requires a substantial amount of stack, put it on the heap. As a rule of thumb: If it's longer than 256 bytes, allocate it.

What about sprintf et al., it it save to use?

Yes, why shouldn't it? If it is C, I would suggest snprintf, however, as it avoids buffer overruns.

[pVC]

I compile it with GCC on MorphOS. It runs fine there. Not only on quick tests, but running it for days.

Are you sure it still doesn't do something illegal, even though it doesn't crash? I'd check it with the Wipeout program to see if it trashes memory or so.

I'd check it like this:
1) Launch SDK:Tools/logtool
2) Run SDK:Tools/Wipeout
3) Test your program and see if you get anything suspicious to the logtool window

[asrael22]

An AmigaOs binary typically has only 4K of stack. Thus, whatever object requires a substantial amount of stack, put it on the heap. As a rule of thumb: If it's longer than 256 bytes, allocate it.

OK. What happens if there is not enough stack space available?

I was reading that memory management on AmigaOS doesn't defragment memory and thought that allocations, if there are many over time, will heavily fragment memory. But for small allocations, a few kB there should always be continuous memory.


Manfred

[asrael22]

Are you sure it still doesn't do something illegal, even though it doesn't crash? I'd check it with the Wipeout program to see if it trashes memory or so.

I'd check it like this:
1) Launch SDK:Tools/logtool
2) Run SDK:Tools/Wipeout
3) Test your program and see if you get anything suspicious to the logtool window

Thanks, I'll try those tools.


Manfred

[guest11527]

OK. What happens if there is not enough stack space available?

The stack fill float into some other object of some other program. That is, everything is possible, from silent data corruption to immediate crashes.

I was reading that memory management on AmigaOS doesn't defragment memory and thought that allocations, if there are many over time, will heavily fragment memory.

Yes. However, for generic C programs, the malloc() implementation of the standard C library will typically use some sort of memory pools to minimize the effect. If you do not want to use malloc(), consider using the memory pools exec provides.

[asrael22]

Yes. However, for generic C programs, the malloc() implementation of the standard C library will typically use some sort of memory pools to minimize the effect. If you do not want to use malloc(), consider using the memory pools exec provides.

OK. I'm using AllocVec/FreeVec. So I should be using either AllocPooled or malloc (because it does it already).


Thanks,
Manfred

[olsen]

OK. What happens if there is not enough stack space available?

Neither Workbench nor the shell will warn you if the program/shell command which you created would use too much stack space. Anything can happen.

Overrunning the alloted stack space produces errors which are extremely hard to diagnose because there may be no directly visible effects. Your program may seem to work correctly, in spite of having had too little stack space to work with.

The side-effects of overwriting the contents of memory which your program did not allocate can be baffling, and you wouldn't know from the effect what exactly the cause was. The next time you start the compiler, it might crash (huh?). Your text editor might hang whilst trying to save a file to disk. The pull-down menus of your text editor may no longer show up. Your system might just crash and hang.

Some of the worst side-effects to diagnose result from trashing the local stack of a function, which includes local variables and the return address of the function. The local variables contents may be damaged, which can result in pointers referencing illegal addresses, for example. If the stack is trashed so far that the function's return address is overwritten, then that function will crash upon returning.

[olsen]

OK. I'm using AllocVec/FreeVec. So I should be using either AllocPooled or malloc (because it does it already).

The memory debugging tools (Wipeout, etc.) that have been mentioned before will directly hook into the exec.library management functions. You might get a clearer indication of where the memory was trashed than if you were using the 'C' runtime library malloc(), etc. functions which add some overhead and thereby obscure the problem. Note that Wipeout can (if you are lucky) only tell you which memory was overwritten, but not necessarily which piece of code caused it to be overwritten.

My advice would be not to put too much hope into tools such as Wipeout. Memory trashing bugs are extremely difficult to diagnose, not just under AmigaOS.

I my experience the only approach which works is by plastering the suspicious code with debug output and tests and stepping through it over and over again until you are confident that it works correctly without misbehaving. That's a lot of work waiting to be done, I can tell you

In a situation such as this I would begin by writing new debug code which displays which memory range is affected by the operations to be performed. What's the first byte that will be copied, what's the last one. Next, add code which verifies that this range is correct and only changes memory which your program is permitted to use (e.g. by having allocated it).

If you want to be safe, print that information and add a call to getchar() so that you will have to hit the Return key for the code execution to continue. That way, you will at least get to see the output before the system crashes.

Make sure that you can trust your code to do what you assume it does. When you are copying NUL-terminated strings, you need to be confident that only as much data is being copied as the memory it is copied to will hold. If your code works well with short strings, but not so much with long strings, you'll have to work on your boundary checking.

[Georg]

Note that Wipeout can (if you are lucky) only tell you which memory was overwritten, but not necessarily which piece of code caused it to be overwritten.

In a certain other AOs clone where it's easy to modify OS source on demand for debugging purposes you can use some tricks:

- allow triggering of memory wall checks of all allocations at any time when app calls some easily reachable function (like hack this into AvailMEM(MEMF_CLEAR)). Whenever anyone calls AvailMem(MEMF_CLEAR) all memory walls of all allocations are checked for mem trashes.

- use this all over the place in the code to debug.

- for tricky cases modify exec mem alloc functions to be be disable/enable protected instead of forbid/permit, and then in the Exec task switching routines trigger the mem trash check. To find out under which task mem trashes happen.

- for other tricky cases use gdb (the OS as a whole runs under gdb, not single tasks or apps) hardware watch points (no slowdown) if you know what is trashed, but not when and by whom. With gdb it is possible to install/uninstall watch points dynamically. For example have a normal breakpoint somewhere at the start of a function and one at the end of a function. Then when hitting first breakpoint have gdb automatically install a hardware watch point on a certain local address and immediately continue running again. When hitting the second breakpoint, have gdb automatically remove hardware watch point, and immediately continue running again. So the thing keeps running and running and only when the hardware watch point triggers the debugger stops.

For AOS itself some of this stuff may be possible to do more easily with an emulator (maybe after enhancing debugging features of emulator some more).

[guest11527]

- allow triggering of memory wall checks of all allocations at any time when app calls some easily reachable function (like hack this into AvailMEM(MEMF_CLEAR)). Whenever anyone calls AvailMem(MEMF_CLEAR) all memory walls of all allocations are checked for mem trashes.

In certain existing legacy 68K binary versions of AmigaOs, tools like MuGuardianAngel exist that do exactly that. Amongst other things. (-:

- for other tricky cases use gdb (the OS as a whole runs under gdb, not single tasks or apps) hardware watch points (no slowdown) if you know what is trashed, but not when and by whom.

Hardware watchpoints are unfortunately not commonly supported under 68K. Or rather, none of the legacy Motorola processors offer such features. However, some of them come with a MMU which can detect certain out-of-bounds accesses into free memory space.

This is something you can combine with a live or post-mortem debugger. But beware, it's not an easy terrain and source-level debugging is then out of the question.

[kamelito]

Isn't the Action Replay cartridge able to do HW watchpoint?

Kamelito

[asrael22]

OK. Another question regarding AllocPooled.

AllocPooled doesn't track the allocated size.
So, what's your advice for how to handle that, say, if the allocation has taken place in some function and the caller is responsible for freeing.
Would I pass an output parameter to the function that should be filled with the allocated size, or?


Manfred

[olsen]

OK. Another question regarding AllocPooled.

AllocPooled doesn't track the allocated size.
So, what's your advice for how to handle that, say, if the allocation has taken place in some function and the caller is responsible for freeing.
Would I pass an output parameter to the function that should be filled with the allocated size, or?


Manfred

You could roll your own AllocVecPooled(), like so:

Code: [Select]


APTR
AllocVecPooled(void * memory_pool, LONG size)
{
   APTR result = NULL;

   if (size > 0)
   {
      ULONG allocation_size;
      ULONG * allocation;

      allocation_size = sizeof(*allocation) + (ULONG)size;

      allocation = AllocPooled(memory_pool, allocation_size);
      if (allocation != NULL)
      {
          allocation[0] = allocation_size;
          result = &allocation[1];
      }
   }

   return(result);
}

VOID
FreeVecPooled(void * memory_pool, APTR memory)
{
   if (memory != NULL)
  {
     ULONG * allocation = memory;

     FreePooled(memory_pool, &allocation[-1], allocation[-1]);
  }
}


However, I would strongly recommend debugging the code which gives you trouble before you consider switching to a different memory management scheme.

[Georg]

before you consider switching to a different memory management scheme.

Btw, it sometimes does help to hack memory management functions ~"down" to lowest level, ie. AllocMem, FreeMem. For example AllocPooled -> AllocMem, FreePooled -> FreeMem (ignoring for a moment side effect like DeletePool no longer freeing all allocations still in the pool), but also malloc -> AllocMem or free -> FreeMem.

In case debugging tool (like original Mungwall?) otherwise does not see and monitor all the individual single allocations from the higher level, but only bigger chunks at lower level with multiple higher end allocations embedded in chunks.