The really cheap ones have three RCA connectors: Composite video, left and right audio. To use these, the Amiga in question should be capable of producing a composite signal. These are the A1200, A600, and A500 if you have the composite adapter (A520). In either case you should see a yellow RCA plug to connect.
If you have composite output and an upscaler that does composite, then you are ready to go! However if you are not happy with the picture you should consider an upscaler that accepts RGB.