I'm not sure, but changing block size might help. (if that is even possible.. there is an option for it)
The CF (at least cheap ones) are reading and writing in blocks (and that's why it does make sense to defragment them from time to time!). if you could make the 'AmigaOS_size_of_block' same as physical, it should boost performance.
maybe I'm wrong, someone will correct me then..
