How can executables work when being thrown into different addresses on 68000 ?

[matthey]

The main obstacle to get some Unix style environment is to be able to load arbitrary code into arbitrary locations. So I remembered that the 68000 lacked MMU but still provided arbitrary program loads and multitasking. But never investigated in depth how this was done.

The Amiga Hunk format uses a relocatable loader. It's convenient and more efficient in some cases but not necessary for the 68000 which can only do 16 bit PC relative addressing (plus or minus 32kB). It's possible to create larger position independent code with multiple branches and other means. The 68020+ has 32 bit branches and supports 32 bit offsets for full 32 bit PC relative addressing and unlimited size position independent code within it's addressable range while being reasonably efficient. It would not be efficient for a 64 bit (addressing) processor to provide for fully independent code through PC relative addressing as the offsets would grow too large. Programs aren't that big anyway. Each processor will have a different limit for the maximum position independent PC relative code and it may not match the 68k, especially with 32 bit fixed length RISC processor encodings which do not have encoding space for 32 bit offsets. The 68020+ is one of the easiest processors to create large position independent code for.

This insight of AmigaOS-68k and MMU less processors now means I know how to make this happen in other environments. Decoding ELF binaries with relocation table and making use of it seems however way more complicated than the Amiga Huff format. The loader has to copy the ELF binary code into memory and then find the relocation table and interpret it. This could form an self hosting ARM environment that works similar to AmigaOS 68k minus the custom graphics and sound.

Relocation like the Amiga Hunk format uses really isn't that complicated. Basic ELF or Amiga Hunk format support including relocation is not difficult. Supporting all the variations of hunk types in a robust way is more time consuming.

[matthey]

The target is ARM or MIPS and only the type of relocation table (list) that my compiler produce will be supported.

It looks like 32 bit ARM branches allow a signed 24 bit offset for forward and backward branches of +-32MB which is very good (conditional and subroutine branches have the same range as all branch instructions include a condition field). Thumb only allows 8 bit offsets for conditional branches (+-256 bytes), 11 bits for unconditional offsets (+-2KB) and 2 instructions for subroutines giving +-4MB which is awful for trying to produce position independent code (PIC). ARM v8/AARCH 64 (new 64 bit ARM) has a PC relative range of +-1MB for conditional branches (and literal loads/stores), unconditional and subroutines allow a range of +-128MB. There are other instructions that are helpful for PIC besides branches but they are the most important and they put a limit on the maximum size of PIC.

In comparison, the 68000 allows 16 bit offsets for all (implicit) PC relative branches giving +-32kB of range. The 68020 allows 32 bit offsets for all branches giving +-4GB for branches and has support for 32 bit offsets for all addressing mode types. It's easier to make a large PIC program for the 68020 than it is for any ARM processor.

Dunno if GCC/LLVM will produce PC relative code for ARM/MIPS though. m68k is not particular big in embedded for new designs..

I believe GCC can be told to produce PC relative (PIC) code with the -fPIC and -fPIE options. However, the code probably has to be written in a particular way to produce a position independent executable (PIE). Do a search for writing re-entrant code for the details. Amiga libraries require re-entrant code with the same restrictions. All support functions have to be re-entrant as well and may require linking with different support libraries.

The 68k is not popular for embedded because there aren't very many options or powerful 68k processors currently available. If an fpga 68k processor with 68060 like performance is good enough for your embedded needs then let me know. I might know where to find one .

[matthey]

Code that is given a different pointer to input and scratch variable space for each instance run ought to also be re-entrant code?

That's right. Global/static variables other than constants generally can't be shared by separate instances of tasks/processes using shared re-entrant code. They are usually allocated on the stack or dynamically on the Amiga. It should be possible for a loader to allocate certain designated hunks for each re-entrant task/process that is started but, as far as I know, the Amiga doesn't support this. It would have made it simpler to write re-entrant programs and have them supported by compilers. It would be possible to require position independent code without allowing for multiple simultaneous tasks/processes but it makes sense to share code if going to the trouble of making it position independent.

An FPGA would be more complicated and require a little bit to much standby current. FPGA Replay has a quite fast m68k CPU alike clone.

An fpga gives flexibility with all programmable inputs and outputs in a one chip solution *if* the processing power of a hard chip is not needed. Of course there are hard ARM processors available in fpgas but that is more expensive. I don't know about the current draw. The fpga power usage seems tiny to me but it's all relative when talking about embedded applications. The TG68 in the fpga Arcade/Replay is not particularly fast but it requires very few resources, it's fully 68020 compatible, it's mostly debugged, it has a friendly license and it should give fairly predictable and consistent performance. It can give 68030 to 68040 performance depending on the speed, resources and configuration in the fpga. The Phoenix/Apollo fpga 68k CPU is coming along and should be available in a mostly debugged state in the next 6 months. Performance ranging from 68040 to 68060+ should be possible but more fpga resources are required than the TG68. The only hard modern 68k processor worth considering is Innovasic's FIDO.

http://www.innovasic.com/Products/fido1100-communication-controller

It's an innovative design that would be awesome for some applications and not so good for others. It's a CPU32 (between 68000 and 68020 with full 32 bit branch and PC relative support) ISA at only 66MHz but then it gives very consistent performance with lower power consumption possible by using a lower clock speed. The 68k can be very powerful with a low clock speed due to efficient memory performance (ARM's load/store architecture is at a disadvantage working in memory without OoO).

[matthey]

The dos.library loader does not support this indeed, basically because its job is over once the code is loaded, and then it is too late. However, there are options for the SAS/C which do exactly that: Put all constants into the CODE (text) segment, and allocate all globals (DATA) manually when starting up the program. Globals are then accessed indirectly through (a4), and a4 is set to a dynamically allocated piece of memory each time the program is run, so code is "pure" and can be made resident.

My memory was a little fuzzy about how this works. The SAS/C option is convenient as it allows programs that weren't designed to be re-entrant to work with some caveats (+-32kB of global data addressing similar to Small Data as I recall). I once considered adding similar support to vbcc but it was so low of priority that I haven't got around to trying it.

How did that "PURE" executables work in relation to this?

Executables that are "pure" are re-entrant (by other tasks/processes) and share code (usually data constants also). The pure file protection bit can be set on any program and the AmigaOS will treat it as a re-entrant program but it will likely crash. Libraries are required to be pure/re-entrant but don't require the pure file protection bit to be set. The relocatable loader is used the first time a pure program is loaded so AmigaOS does not require pure programs to be position independent. However, absolute 32 bit addressing (commonly used for global variables of large programs) which is relocated is less useful with pure programs.

[matthey]

Pure flag only indicates this program can be made resident (preloaded to ram with resident command). Internally programs can have different means to achieve re-entrancy. It could be semaphore or dynamically allocate ram for each instance.

Code initialization may be different for a pure program but I believe the AmigaOS (dos.library?) has code that maintains a list of currently used pure programs (resident list?), allows for finding and reusing the code and flushes the code under certain conditions. ThoR probably knows all the details of how it works. The AmigaOS does have the Resident CLI command to make pure programs resident (it can also show the use count of resident commands) and the H (Hold not Hidden) file attribute flag to make them resident when first used (requires the Pure bit also) without using the Resident command (saves a few lines in the S:Startup-Sequence for making Assign, Mount and Execute resident for example). There appears to be automagic support from the AmigaOS.