GLFW (OpenGL toolkit) port for AmigaOS

[marcus256]

Hi everybody!

I'm new here, but I thought it's the best place to drop this info and get some feedback...

Anyway, I'm the developer of a portable OpenGL framework (or toolkit, if you like), called GLFW. It started out under Windows, and quickly got ported to Unix/X11. Now, when the API & code has matured a bit I thought it was apropriate to do an AmigaOS port too (after all, I'm an old AmigaOS fan!).

My problem is that I'm a bit rusty, so I would very much like to get some help with some things. Also, I'm conducting all development under WinUAE (since my A3000 does not fit in my tiny computer cabinet, unfortunately), so 68k only and no HW OpenGL support - speed is tremendous though (like 200-300 MHz '040).

Primarily I'm looking for some testers/users that are intererested in using and possibly aiding in the development of GLFW.

Short intro...

GLFW = OpenGL FrameWork

Window management (normal/fullscreen), keyboard/mouse/joystick input, timer (high precision - microsecond class), multi threading support, texture file loading support (TGA).

Of course, everything is 100% portable. Some of the supported OS:es are: Windows 9x/NT/2k/XP, Linux, IRIX, Solaris, QNX, Mac OS X and AmigaOS.

URL = http://glfw.sourceforge.net/

I'm especially interested in AmigaOS 4.0 / MorphOS compability, and support for different GL/Mesa implementations (I currently use StormMesa).

[marcus256]

> Have you thought about supporting AmigaOS4
> and/or MorphOS? They both involve new machines.

Yes, of course. I know these are new machines, but I believe that they are virtually 100% sourcecode compatible (of course I'm using pure OS friendly coding, no HW access, special assumptions etc) - and GLFW is distributed as source code. I read up on MorphOS, so I have actually included some MorphOS specifics regarding task creation (though I have not tested it myself).

I think both platforms are very interesting targets - especially when they get HW 3D support (that is a must for a system to be successful today, IMO). OS4 is rumored to get Mesa 5.0 (OpenGL 1.4 compliant). Does anyone know about HW support? Will it be through some new Warp3D interface?

[marcus256]

> I'm sure there will be increased need for this kind
> of libraries as soon as these OS:s have their 3D
> support nicely up and running.

Yes. I'd like to think of GLFW as a foundation for writing portable interactive 3D applications (most notably: games!). There is already one major application that uses GLFW as its "core", check out http://www.orkysquad.org/main - a very cool game.

The wording goes "Orky can be ported to any platform with GLFW support".

Personally, I think tools like this is absolutely necessary for a small market such as AmigaOS, since very few coders/companies are willing to spend time on supporting a "minority platform". By using portable technologies (OpenGL, GLFW, libMikMod etc), the hazzle of porting to AmigaOS is nearly negligable.

[marcus256]

Hi Karlos,

> I have written a C++ based portability layer (still in
> development) that encapsulates many things
> useful for your project

I believe we are doing pretty much the same thing (at least we have similar goals). GLFW is: portable, minimalist, very compact (a 68k Amiga executable is usually ~25 KB, the Windows DLL is 30 KB), extremely easy to use & powerful.

The major difference, as I see it, is that I use pure C, not C++. I don't like C++ for these kinds of things, since:

1) It's not as portable as C (no, not yet at least)
2) It's quite difficult to turn a C++ interface into a shared library
3) You can't use C++ interfaces from alternate languages (such as Borland Delphi, Visual Basic, Fortran, Python etc)

Maybe we can exchange experiences? You are of course free to "rip" the GLFW code if there is something that you find useful (the GLFW license is extremely liberal).

If you don't feel like downloading the whole pack, you can always browse the Sourceforge CVS tree: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/glfw/


> class MyApplication : private AppBase, FullscreenDB {
>
> };
>
> Does 90% of the work for you

In GLFW you usually do:

------------------------------------------------
glfwInit();
glfwOpenWindow( 640, 480, 8,8,8,0, 24,0, GLFW_FULLSCREEN );
do
{
    glfwSwapBuffers();
    running = !glfwGetKey( GLFW_KEY_ESC ) && glfwGetWindowParam( GLFW_OPENED );
}
while( running );
glfwTerminate();
------------------------------------------------

That's a complete GLFW application with input, double buffering and a "main loop" (you can do it differently too, e.g. with callback functions for input etc). I agree, not very OO, but I believe that it's not too ugly to write a C++ class on top of GLFW, if that suits your project better than the programmatic C interface.

> 6) Audio classes (fell a bit behind there).

Recommendation: stay away from audio! People tend to underestimate the complexity of audio. You have two choices:

1) Make a fullfeatured, complete audio interface (sound samples, MP3, mixing, 3D aureal emulation, music playback, CD music handling etc)

2) Ditch it and use a 3rd party toolkit (libMikMod, FMOD, OpenAL etc)

If you make a simplistic interface (a-la SDL), professional people will not want to use it anyway, rendering your effort useless and the interface bloated.

[marcus256]

> However, it is a link library so I don't get quite the compactness you do.

Actually, my lib is a static link library too. Under Windows I made it a DLL too (so you can select what suits your project the best). The 25 KB Amiga executable includes the GLFW static link library.

Ah, there's a question: How do you handle things like double floats as input/output arguments in an AmigaOS shared library??? I remember the ieeedoubbas libraries using dual 32-bit integer registers for passing 64-bit floats, which I don't find too appealing. Can't you mandate using the FPU registers? (the lib will probably not be used on non FPU-equipped CPUs anyway). How are the GL/Mesa shared libraries implemented under AmigaOS (they too use doubles)?

[marcus256]

> Yup, this is a problem, but only for 68K libraries as far as I know.

So how are things supposed to work under OS4, for instance?

What shared library standards exist except the old AmigaOS style (D0-D7,A0-A3 - I though A5?).

> I think you can also pass 32-bit floats in registers d0-d7 too

Obviously. It's just  a calling method - the caller & procedure just have to agree on the conventions. As I said, the old double precision IEEE libraries used two 32-bit integer registers per 64-bit float.

I'm also considering using referenced arguments (e.g. A0 pointing to a double). That does not go well with the GLFW API specification, but I suppose I can make wrapper macros or whatever.

I just thought that some new standard might have emerged, since the AmigaOS style does not accomodated for:

1) Floating point values
2) 64-bit floats/ints
3) Many arguments (exceedeing the number of available processor registers)

> Incidentally, are you using MesaGL? You may want to look at MiniGL for performance under current apps.

Does MiniGL work with WinUAE (i.e. without HW 3D acceleration)?

Worth supporting though - since some people seem to prefer it. Just wondering if I can test it myself. I have #ifdef:ed everything that is specific to the GL/Mesa implementation (currently only StormMesa), so that support for different implementations is just a matter of a simple #define or compiler-switch.

[marcus256]

I suppose that's one way to go (at least for AOS4 it's probably the prefered way). Another way is to ignore  the way the 68k assembly interface looks: write a library interface with "nonsense" arguments (e.g. D0/D1 = 64-bit float, or A0 points at function return value, etc), and write a wrapper C language interface library (libglfwsh.a or glfwsh.lib - sh = shared) that hides the library interface from the programmer.

More questions: For the multi threading part of GLFW I had to use the tc_UserData field of the Task structure for each GLFW thread (including the main thread, which is the one calling glfwInit()).

I know this is not good practice, but I see no other solution (each thread requires a lot of state that I have to store somewhere, and I do it in a struct pointed to by tc_UserData). If the GLFW program only uses GLFW threads, and not any custom AmigaOS tasks/processes etc, this will be no problem. Can anyone think of a situation where it COULD  be a problem???

[marcus256]

That's a good solution (at least as good as it gets, I suppose). Currently the interface is not very dependant on the custom GLFW thread structure, but I am planning to add support for multiple windows in a way that would require every single windowing function (event handling, window management etc) to access the thread private area.

Why? Thread aware window management!

The idea is very similar to OpenGL per thread rendering contexts (one context per thread - transparent from a coders point of view). For instance, I will have a function called glfwBindWindow(), which binds a window to the currently calling thread. All window operations will apply to that window. Since different threads can work on different windows, I need to store this "window binding" information in an easily accessible thread private area - my GLFW thread structure, pointed to by tc_UserData, is the obvious choice.

I can still use the "fool-proof" check that you described, but that means that using non-GLFW threads will limit the use of GLFW functions (e.g. you can't open a window from a non-GLFW thread).

I wish AmigaOS had something similar to POSIX pthread thread-specific data keys or Win32 thread local storage (TLS), which are basically indexed arrays of tc_UserData that you can dynamically allocate, per process. I suppose this does not work well without a  proper process class though...

[marcus256]

> Considered a port to AROS?

Is it worth it?

How is OpenGL support under AROS?

How different is AROS from AmigaOS?  I need AmigaDOS processes, intuition,  graphics and input.device. The rest is quite easy to replace (timer.device/ReadEClock, gameport.device, StormMESA etc).

Can I run AROS on my PC (natively, or under Windows, Linux or WinUAE)?

[marcus256]

What I meant with TLS, is that it would be nice if AmigaOS had support for TLS natively, so that I can use it to realize some of the GLFW threading things with it.

tc_UserData works very much as TLS, but the problem is that there is no OS-friendly way of allocating/deallocating it (as you said - exec simply does not care), so if GLFW is to use it, the application (that uses GLFW) can not use it. If AmigaOS provided proper TLS support, GLFW could allocate a TLS "key" or "index" private to GLFW, and the application is free to allocate other keys, meaning that there are no potential conflict situations.

By the way, I solved the condition variable (signalling primitive) support by adding a field to the GLFW thread structure called "waiting_for" (or something similar), so that when a thread is to signal or broadcast a condition to any waiting thread(s), it loops through all the known GLFW threads and checks the waiting_for field, to see if it is waiting for this particular condition. If so, a signal (that is private to the waiting thread, and whose ID is stored in its thread structure) is generated. Of course, critical sections (Forbid/Permit) is used wherever necessary.

I think this is the most viable solution for broadcasting signals to multiple threads in the way that is required for condition variables to work (it should be quite cheap too, since mostly you don't have more than a couple of threads, perhaps 10 at most or so). Have you done anything similar?

> I made a Threadable Rasterizer class (still in
> development) that has a double buffered vertex
> array / command queue. The rendering calls fill
> one buffer whilst the previous one is being
> rendered by the internal thread.

In the GLFW distribution I have an example program that works this way. It's a particle system, where the particle physics is carried out in one thread, and the rendering/billboarding is done in another thread.

First I did a straight forward solution without double buffering (meaning potential stalls). It roughly gave a 100% of the speed of a single threaded implementation on a single processor system, and 105-150% on dual CPU systems.

Then I added double buffering, actually resulting in a performance drop (about 95% on the single processor system, and slightly degraded performance on the SMP systems compared to single buffering).

I reccon the reason is that:

A) OpenGL hardware already runs asynchronously on most decent implementations (at least under Windows and Linux), so that there is no gain in using separate threads on single processor systems

B) Double buffering means more cache trashing, effectively degrading CPU performance

I still think multi threading like this is a good thing if it does not cost performance for single processor systems. Future systems are very likely to have multiple CPU cores (either SMP or SMT), meaning that multi threaded programs will gain performace "for free" on those systems. And, as you said, in many situations it can help the design to use multiple threads.

[marcus256]

oops - dual post...

[marcus256]

Daarrrgh!!!!

I wrote a leeengthy reply to this mail - but it didn't get posted (login timeout?). Anyway, this will be more brief...

> -edit-
> [snip]
> -end edit-

I agree...

I didn't quite understand your signalling policy. GLFW mimcs the POSIX pthread API (which rocks, IMHO), by supporting mutexes (AmigaOS signal semaphores) and condition variables (sleep/wake mechanism).

What's special about condition variables is that any numder of threads can be waiting for the same condition, and any numbder of threads can be signalling that condition (not knowing about which, if any, threads are waiting). Also, the condition variable does not maintain any state (it's like a strobe), so the actual condition has to be managed through mutex-protected shared variables. This means that the condition can be of arbitrary complexity (boolean, counter, combination of conditions etc).

The problem with AmigaOS is that each task has it's private set of allocated signals, and the signalling thread must know both which task to signal, and which signal ID that particular task is waiting for. That is why I need a loop to check each and every GLFW thread if it is waiting, and which signal ID it is waiting for (and of course, if the signal ID corresponds to the condition variable that is currently being signalled).

I actually don't use Forbid/Permit (I was confused with the joystick code I did recently, where joystick allocation needs Forbid/Permit). I use a global signal semaphore that protects all thread state (e.g. when a thread is added to the GLFW thread list).

Regarding task switch times: AmigaOS may be good, but Windows NT/2k/XP is really good! Windows 98 sucks big time though. I have a benchmark program in the GLFW example program collection which does forced context switching (two threads that signal/wait/signal/wait... in a loop). Here are some results:

AmigaOS (WinUAE, 68020 ~200 MHz): 50,000 switches/s
Windows 2000 (Athlon 700 MHz): 500,000 switches/s
Windows 98 (Athlon 700 MHz): 23,000 switches/s
Linux (Athlon 700 MHz): 160,000 switches/s
SunOS (6 x USPARC2 400 MHz): 120,000 switches/s
OSF/1 (1 x Alpha 21264 500 MHz): 130,000 switches/s
OSF/1 (2 x Alpha 21264 500 MHz): 40,000 switches/s

The signalling involves both mutex locking and condition signalling. The GLFW implementation of course has some kind of overhead to it too. I think that under Mac OS X (pthread) the figure is somewhere in the range 10-20 kswicthes/s. IRIX 5.3 also sucked if I remember correctly.

Do you have any similar benchmarking figures? (it would be interesting to compare)

Oh, and I have unconditional sleep too. It uses Amiga's Delay() - is that any good? (gives me a minimum of 40 ms sleep time in average - funny, I thought it would be 1000/50 = 20 ms)

I still haven't solved timed conditional waits. I suppose I would have to use timer.device to set up a timeout signal and wait for that too.