Where are the weak points?

[krashan]

Unfortunately it still looks limited to 8 bits per color channel, no HDRI

Edit: I do see 16 bits per channel! It's not float, but it's better than 8 bits.

Edit2: Video isn't going to be good through Reggae, it's documented as not supporting any YUV color spaces currently. It would be high CPU and low quality. This is really just for basic images and audio.

Reggae is an evolving framework. It is designed to be extendable. Even if it does not support some feature now, such a feature can be probably added.

Current Reggae supports only two video formats: "GRAY8" and "RGBA32". For serious video support at least one YUV format will be needed. For high dynamic range images probably another one. However additional formats should be added with care. Application programmer expects filters to support all the formats. Then adding many formats to Reggae means more work for filter programmers.

The question of HDR image format in Reggae is open. It may be ARGB64, it may be 4 x float too. Preliminary introduction of ARGB64 was inspired by PNG specification, and it is not widely supported by Reggae classes yet. We can discuss it.

[krashan]

I can work with 8bit for images for now, but the addition of YV12 for 4:2:0 streaming playback and UYVY for 4:2:2 editing would be crucial for video.

In my opinion Reggae should use one, or at most two YUV formats. Remember, more "common formats" means more work for filter implementors. Other YUV subformats should be supported with conversions.

For others to add to the list of available classes, it would be nice to have examples of a demux/decoder pair and possibly an encoder/mux pair.

Rzookol has plans to release source of some of his classes. I may add to this, that I work on LibMaker application, which is a GUI driven, parametrized code skeleton generator for shared libraries, BOOPSI classes, MUI classes and Reggae classes as well.

Splitting the demux and decoder is a great addition for video where you can have multiple containers (avi, mov, etc) share decoders, but I'm a little confused about the demux/decoder separation on many formats such as still images though.

For such formats demuxer decodes (and detaches) the header, extracts metadata (if any). Demuxers are also responsible for format recognition (Reggae performs content based recognition).

I assume layered formats, such as PSD and TIFF would somehow fall under the type MMT_DOCUMENT? Are they intended to expose a list of MMT_PICTURE or am I way off base here?

MMT_DOCUMENT is a postsign standing on a boundary of uncharted land ;-). In my opinion PSD (I guess you mean Photoshop format) is out of Reggae scope. Of course one can imagine a document, where multiple images are stacked one on another and alphablended. However PSD allows for much more than that. Layers can be combined in numerous, non-trivial ways. Do we really want the whole Photoshop compatible image composition engine in Reggae? Or it just should deliver layer images, their offsets and scaling factors and enumerate combiners used?

[krashan]

In similar frameworks, the color format is negotiated and most encoders/decoders would not support more than one or two color formats as needed by the type they (de)compress unless the author just wanted to do so.

Reggae features format negotiation as well. For example when we talk about RGB color formats, the only format which must be handled by filters, is ARGB32. However decoder can deliver (and encoder can request) any of {RGB24, ARGB32, BGR24, BGRA32, ABGR32}, decoder can also deliver LUT8 with palette. Reggae automatically inserts converter into the pipeline. Converter object is of either videopcm.decoder or videopcm.encoder class. They feature highly optimized conversions, including routines accelerated with AltiVec engine. Similar mechanism is implemented for audio formats. Additional formats may be added if needed.

A function in Reggae connecting two objects is tag based, so in the future format negotiation may be controlled by application.