Is there any information or documentation on which files I should remove from my Coppermine directory after installation to secure my instance? I've moved the upgrade-1.0-to-1.2.php and install.php files to a different directory, are there other files I should be concerned with?
There is no need to remove any files after installation.
Is that in the documentation anywhere?
No. The fact it is not there indicates it is not necessary.