From Joomla! Documentation
This page documents the _JEXEC check that is used to protect many PHP files, explaining why it is used, when to use it and situations where it shouldn't be used.
The following line is commonly found at the start of Joomla! PHP files:
defined('_JEXEC') or die('Restricted access');
"_JEXEC" is a constant that is typically defined in the 'index.php' file at the root of the Joomla! instance and is used to mark a secure entry point into Joomla!. The defined or die check makes sure that _JEXEC has been defined in the pathway to get to the file. This is used to ensure that a file that could expose path information because functions, variables or classes aren't defined in that file trip PHP's error reporting and expose a path. It also prevents accidental injection of variables through a register globals attack that trick the PHP file into thinking it is inside the application when it really isn't.
Setting the error reporting down would have a similar effect, however there are configurations where changing PHP's INI settings aren't permitted. The JEXEC check works regardless of whether the configuration can be changed and has no other side effects (e.g. if you're debugging having every file reduce the error reporting would be annoying because you'd have to either set a debug flag to stop it or after each file is included reset error reporting, not fun!).
The check should be added to files that when accessed directly cause a path exposure. For example, the following error occurs when the Backlink System Plugin (/plugins/system/backlink.php file) has had the _JEXEC check disabled:
Fatal error: Call to undefined function jimport() in /Users/pasamio/Sites/workspace/joomla_15/plugins/system/backlink.php on line 18
As is evidenced, the 'jimport' function doesn't exist when the file is directly called so PHP raises an error and exposes the path to the file. Adding the defined or die check to this file will cause a "Restricted access" message to be displayed when the file is accessed.
So the general rule for the JEXEC check is if the PHP file depends on another file to operate properly. Typically if you access a file directly without the JEXEC check and a PHP error is raised (presuming your PHP error reporting is set to show errors by default) about a missing variable, function, object or similar then the file needs to be protected.