Thinking I was clever, I hex-edited the executable, modifying each instance of 0x02C0 (704) to 0x0400 (1024), and rendered an image from a project that was set up for HIRES and OVERSCAN in the project's .SCR file, but it made no difference.
Well, obviously that will not work. You need to change the exact correct values or you'll just break the thing.
Even if would remove the limit, it still might fail to work for higher resolutions.