The "diskchange cd0:" command did not help. I also tried with an older 68040.library and CDFileSystem, same thing happened. With a newer 68040.library it didn't load the CD at all. To use the CacheCDFS i need the new version of the library.
As it hasn't always been like this I will try find another drive.