A backup is a copy of the data stored on a computer, which can be used to restore the computer to the state it was in at the time the backup was made. Useful in case of data loss or if the computer system's security measures are compromised by malicious attackers.
To backup your Joomla site you will need an FTP client. Log in to the FTP account of your site and download all the files and folders that exist in the main directory. Be sure that the file and directory structure remains the same as in the live site.
Alternatively you may use the backup functionality provided by many webhosters through their site management interfaces.
You will also need to create a copy of your database. The mechanism varies depending on the host of your site. Most hosting providers provide phpMyAdmin to help you manage your databases (provided of course that you use MySQL as your database).
Instructions on how to backup a database: http://www.phpmyadmin.net/documentation/#faq6_4