AGA is a palette based video chipset. It has 256 palette entries that are each 24-bits wide (8-bits per channel). When using 8 bitplanes, each pixel only takes up 8-bits in chip memory, but this 8-bit value is only used as an index into the palette registers to lookup the appropriate 24-bit color. This is why you need a 24-bit video output to properly display AGA.
HAM8 doesn't change the output color width. It's still 24-bits. It just changes how you select a 24-bit color from the 8-bit data in chip memory. In HAM8 you can either use one of the first 64 entries from the palette or modify the upper 6-bits of one of the 8-bit channels of the 24-bit color that was used for the last pixel.
Well, maybe forget about HAM8 mode. I could see how that would Complicated to implement with only 12bit color.
AGA still offers up to 8bit color in much higher resolution than ECS. Perhaps the 24bit palette register could pick the closest of the available 4096 colors. That and the increased AGA speed might be worth it.
