Be able to roll back to an earlier version of your site using a modern version control system, such as CVS, Subversion, or git. The Eclipse IDE indicated about includes a Subversion plugin. This allows you to work with the Joomla! source repository as well as other projects hosted on JoomlaCode.
Thou shalt at all time be able to return your site to a previous working state through regular use of a strong, off-site backup and recovery process. Be sure your backup and recovery process is in place and tested BEFORE you go live. This is the single best way (and often the only way) to recover from such inevitable catastrophes as:
A compromised/cracked site.
Broken site due to a faulty upgrade.
Hardware failure, such as dead hard drives, power failures, server theft, etc.
Authoritarian government intervention. (More common than some think.)
Needing to quickly relocate to a new server or hosting provider.