The only limitation is that someone has to write the core code & drivers to support the particular screen modes we wish to display - and the FPGA needs enough spare capacity to output the mode we're looking to support. Yacubed estimated the Replay could support modes upto 1,280 x 1,024 - but I am guessing that's in full 24-bit mode, and that higher resolutions would be possible with less colours.
The hard limitations are:
* Primarily memory bandwidth, (screen size) * (color depth) * (updates) = (memfreq) * (bits)
* Memory size limits maximum resolution
* Memory access can be lessen by using HAM-style mode. Like HAM24 etc..
* I/O transition rate (333 Mbps/lane?) on the FPGA and number of bits wired to the video transceiver chip
Things like DDR on the memory access doubles number of bits per clock. While CPU accesses uses up some..
