Actions

Setting up your workstation for extension development (build with Phing)

From Joomla! Documentation

Revision as of 06:08, 12 February 2009 by PtrNrs (Talk | contribs)

Copyedit.png
This Article Needs Your Help

This article is tagged because it NEEDS REVIEW. You can help the Joomla! Documentation Wiki by contributing to it.
More pages that need help similar to this one are here. NOTE-If you feel the need is satistified, please remove this notice.


This article explains one way in which you can set up your workstation for writing Joomla! extensions. It assumes that you are using Eclipse and that you are familiar with using Eclipse for working with Joomla! and Subversion. For more information about the basics of getting started with Eclipse for Joomla! development, see Setting up your workstation for Joomla! development.

Joomla! developers can and do use many different development tools, and there is no one way that is best for everyone. This article documents one way to use Eclipse and Phing to help streamline the process of writing and testing Joomla! extensions.

Contents

Overview

If you are writing a Joomla! extension, you have a challenge. On one hand, you need to keep the source code for the extension separate from the Joomla! source code. However, you also need to be able to run and perhaps debug the extension from within Joomla!. The problem is somewhat more difficult if you are using the Subversion (SVN) source code control system for the extension source code.

One simple way to do this is to create two separate projects in Eclipse. One project will be the "development" project. It will only contain the extension source code, and, if needed, can be controlled by SVN. The second project will be a "test" project. It will contain a full copy of Joomla! as well as a copy of your extension source code. Then we will create a Phing automatic build task that will allow us to keep the test copy of the extension current with the actual development copy.

Create the Development Project

The development project will contain the source code just for your Joomla! extension. If you are using SVN, then you will create this project in Eclipse by using File → Import → Checkout Projects from SVN. If you are not using SVN, you will just create a new PHP Project and create the appropriate folders and files for the extension. You should also have an install archive file so you can install the extension in Joomla! using the normal extension installer (Extensions → Install/Uninstall). We will see below how we can use Phing to create ZIP and "tar.az" archives automatically.

In our simple example, our extension is a plugin called "FJ Author Plugin". This has just two files called "authorunpublish.php" and "authorunpublish.xml". We also have created a zip archive file that contains these files, so we can install this extension on a Joomla! website.

Create the Test Project

Creating the test project is very easy. Just create a new PHP Project in Eclipse (in our case, called "Joomla_1.5.9_test"). Then unzip a full Joomla! zip archive into the top-level folder for this project (in our example, "C:\xampp\htdocs\joomla_development\Joomla_1.5.9_test"). Then navigate to the back end of this site and install Joomla!. (Note: If you already have a test environment set up you can just use this environment.)

Once the environment is set up, install the current version of the extension into the Joomla! test site in the normal way. In the back end of Joomla!, navigate to Extensions → Install/Uninstall, press Browse, find the archive file for your extension, and press "Upload File and Install".

  • Note that this step is needed so that the Joomla! installer can add the extension to the database. In this case, a row is added in the jos_plugins table to register this plugin in Joomla!.
  • Also note that, at this stage, the extension programs do not have to be working or finished. You just have the have a valid XML file for the installation and the appropriate files. The contents of the files, other than the XML file used for installation, does not matter. They can even be empty files.

Install Phing

Next, we need to add the Phing program to our PHP environment. Phing is an open-source project build system based on Apache Ant. In our example, we are going to use it to copy the program files from the development project to the test project every time we build the project.

Installing Phing is very easy.

  1. Go to the command line.
  2. Change directory to the Apache PHP folder. On my XAMPPP system, it is "C:\xampp\php".
  3. Enter the command pear channel-discover pear.phing.info
  4. Then enter the command pear install phing/phing
  5. The system will download Phing. On Windows, the executable "BAT" file will be in your PHP folder (for example, "C:\xampp\php").

If phing is already installed, you may see the error "Ignoring installed package phing/phing Nothing to install" at step 4. If so, proceed. You may though may need to repeat this step after deleting all phing files and folders in c:\xampp\php.

Create the "build.xml" File

Next, we need to create a file called "build.xml". This file will tell Phing what we want it to do during a project build -- in our case, just copy the two plugin files from the development project to the test project.

In Eclipse, select File → New → XML and create a file called "build.xml" in the root folder of your extension project. In our example, the file will contain the following code:

<?xml version="1.0" encoding="UTF-8"?>
<project name="FJ Author Plugin" default="copy_all" basedir=".">
        <property name="src"   value="."/>
        <property name="test"  value="../Joomla_1.5.9_test/plugins/content" />
        <target name="copy_all" description="Copies files to test project.">
                <echo message="Running build.xml. Copying files from dev to test..." />
                <copy file="${src}/authorunpublish.php" todir="${test}"/>
                <copy file="${src}/authorunpublish.xml" todir="${test}"/>
        </target>
</project>

Line 2 tells Phing that the default action to take is called "copy_all" and that the base directory for this is the current directory. Lines 3 and 4 just define the variables "src" (our current development folder) and "test" (where we want the files to be copied). Notice that "test" is set to copy the files to the "plugins/content" folder of the Joomla! test system. Of course, this will vary with different extension types. You can define as many variables as you need.

The next line creates the "target" element, in this case the default target called "copy_all". Inside the target element we have three lines. The first is just an echo command which will display the message in the Eclipse console when the build is run. The next two actually do the work. They copy the two files from the "src" folder to the "test" folder.

There are many things you can do inside a build file with Phing. See the Phing User Guide for more information.

Add Phing to the Eclipse Build

The last step is to set up the project in Eclipse so that our Phing build program is run automatically each time we build the project. To do this:

  1. Right-click on the project and select Properties. Then select "Builders" as shown below.
    Screenshot eclipse properties 20090122.png
  2. Press New to display the Choose Configuration Type dialog shown below.
    Screenshot eclipse phing1 20090122.png
  3. Select Program and press OK to show the Edit Configuration dialog shown below.
    Screenshot eclipse phing2 20090122.png
    • In the Name field, enter a descriptive name for the program.
    • In the Location field, press "Browse File System" and browse to the location of the "phing.bat" file (for example, "c:\xampp\php").
    • In the Working Directory, press "Browse Workspace" and select the root folder of the current project.
    • Note that we are not entering anything into the Arguments field. This is because Phing by default will run the file called "build.xml". Also, we specified the "copy_all" as the default task inside the "build.xml" file. So this executes by default when Phing is run.
  4. Click on the Refresh tab to display the screen below.
    Screenshot eclipse edit configuration 20090123.png
    Here you have the option of refreshing the workspace after the files are copied.

    Note about Eclipse Refresh: Eclipse keeps track of all local file changes. This allows it to restore files from it's local history repository. When you copy or change files outside of Eclipse (for example, with Phing), Eclipse needs to record these file changes. This is done with the Refresh command.

    If you check the options "Refresh resources upon completion" and "The entire workspace", then the workspace will be automatically refreshed after the build. On some systems, this might take a long time. In this case, you would uncheck the "Refresh resouces upon completion" box. If automatic refresh is not done, you just need to refresh the files (using F5 or right-click → Refresh) manually. If you try to open a file that hasn't been refreshed, you get a message "Resource is out of sync with the file system" and you can refresh at that time.
  5. Press the Build Options tab to show the screen below.
    Screenshot eclipse edit configuration2 20090123.png
    Here you can choose whether to have the Phing script run "During auto builds". If you set the project up for Automatic Builds (using the menu option Project → Build Automatically), and you check the "During auto builds" option, the Phing script will be run automatically every time you save an edited file.
  6. Press OK and the Properties dialog should now be as shown below.
    Screenshot eclipse phing3 20090122.png

Finally, let's test that this works. The idea here is that our Phing task should run whenever we build the Eclipse project. In the PHP Explorer, right-click on the project and select "Build". In the Eclipse console, you should see something similar to screen below.

Screenshot eclipse phing4 20090122.png

Note that, in this example, the source files in the development project were identical to the files in the test project, so we got the message "omitted, is up to date" for both. If you change one or both of these files and build the project again, you will see they have been copied.

More Phing Examples

Below are some more examples of things you can do with Phing. Please keep in mind that there are many advanced capabilities that you can use with Phing. See the Phing website for more information.

Run Phing File Outside of Build

In the previous example, we ran the Phing build.xml file automatically when the Eclipse project was built. We can also run any Phing file directly from Eclipse using the Eclipse External Tools Configuration. To do this:

  1. Select Run → External Tools → External Tools Configuration, as shown below.
    Screenshot eclipse external tools0 20090122.png
  2. Highlight the Program option and click on the New Launch Configuration button, as shown below.
    Screenshot eclipse external tools1 20090122.png
  3. Fill out the Externals Tools Configurations dialog box as shown below.
    Screenshot eclipse external tools2 20090122.png
    • Enter a descriptive name in the Name field.
    • In the Location field, again click on the Browse File System button and find the "phing.bat" file.
    • In the Working Directory field, click on the Browse Workspace button and select the root directory of the project.
    • In the Arguments field, enter "-f build.xml". Note that this is not strictly necessary, since "build.xml" is the default file name for Phing. However, this shows how you would run any Phing file.
  4. Select the Refresh tab and enter the desired refresh options. These are the same as discussed earlier.
  5. Select the Build tab to show the screen below.
    Screenshot eclipse external tools4 20090123.png
    Here you can select whether or not you want to automatically build the project before running this Phing script and, if so, which projects you want to build.
  6. Press the Run button to actually run the "build.xml" file. You should see the same output as you did when you ran the project build.

To re-run this file, you can just select it from the list using the Run External Programs button, as shown below.

Screenshot eclipse external tools3 20090122.png

Alternatively, you can select Run → External Tools and select it from that list.

Copy From Test Project to Development Project

In some situations, it may be more convenient to write your code in the test environment and copy files from the test project to the development project. Again, this is very easy to do with Phing. In this example, we will create a Phing file in our development project called "copy_to_dev.xml" that will copy the files from our test project to the development project.

  1. Highlight the test project (in our example, "Joomla_1.5.9_test"), right-click, and select New → XML.
  2. Place the file in the project's root directory and name it "copy_to_dev.xml".
  3. In our example, the code for this file is the following:
<?xml version="1.0" encoding="UTF-8"?>
<project name="Plugin Test" default="copy_to_dev" basedir=".">
        <property name="test"   value="./plugins/content"/>
        <property name="dev"  value="../FJ Author Plugin" />
        <target name="copy_to_dev" description="Copies files to dev project.">
                <echo message="Copying files from test to dev..." />
                <copy file="${test}/authorunpublish.php" todir="${dev}"/>
                <copy file="${test}/authorunpublish.xml" todir="${dev}"/>
        </target>
</project>
This code is very similar to the "build.xml" discussed above, except that it copies in the other direction.

To run this, add a new External Tools Run Configuration by following the steps outlined in the previous section. When you get to the External Tools Configurations dialog, enter the following in the Arguments field: -f copy_to_dev.xml. Check that the settings in the Refresh and Build tabs are what you want. For example, you may want to uncheck the "Build before launch" option if it takes a long time to build the test project.

When you run this, it should copy the files from the test project to the development project.

Alternate Copy File Syntax

Another way to use Phing to copy files is using the "fileset" property. The script below does the same thing as the "build.xml" file shown earlier.

<?xml version="1.0" encoding="UTF-8"?>
<project name="FJ Author Plugin" default="copy_all" basedir=".">
        <property name="test"  value="../Joomla_1.5.9_test/plugins/content" />
 
    <!-- Fileset for all files -->
    <fileset dir="${project.basedir}" id="allfiles">
        <include name="authorunpublish.*" />
    </fileset>
 
    <!-- copy_all Target -->
    <target name="copy_all" description="copy all files target">
        <echo message="Copying files to test project..." />
        <copy todir="${test}">
            <fileset refid="allfiles" />
        </copy>
    </target>
</project>

This code uses the Phing "fileset" property to select the files to be copied. With this property, you can include or exclude files using wildcards to create the list of files you need. You can read more about different Phing options at the Phing web site.

Create Archive Files In Phing

You can also use Phing to automate the creation of your extension archive files. For example, the script below creates a "tar.gz" archive using the built-in "tar" task in Phing:

<?xml version="1.0" encoding="UTF-8"?>
<project name="FJ Author Plugin" basedir="." default="copy">
 
    <property name="testdir" value="../Joomla_1.5.9_/plugins/content" override="true" />
    <property name="srcdir"   value="${project.basedir}" override="true" />
 
    <!-- Fileset for all files -->
    <fileset dir="." id="allfiles">
        <include name="authorunpublish.*" />
    </fileset>
 
    <!-- Copy Target -->
    <target name="copy" description="copy target">
        <echo>Running phing_test.xml. Copying files from dev to test...</echo>
        <copy todir="${testdir}">
            <fileset refid="allfiles" />
        </copy>
        <echo msg="Creating archive..." />
        <tar destfile="plg_authorunpublish.tar.gz" compression="gzip">
            <fileset refid="allfiles" />
        </tar>
        <delete file="plg_authorunpublish.zip" quiet="true" />
        <zip destfile="plg_authorunpublish.zip">
            <fileset refid="allfiles" />
        </zip>
         <echo msg="Files copied and archive file created OK!" />
        </target>
</project>

Notice that we can re-use the "fileset" we used for the copy task. The "tar" task creates a compressed "tar.gz" archive for our extension. The "zip" task creates a zip archive. Also note that we have a "delete" task before the "zip" task. This is because the zip task adds files to the archive instead of replacing files in the archive.

More Complex Phing Example

Here is a more complicated real-life example of a Phing script. The project is called FJEventsNews and has the following structure, typical of a simple Joomla! component extension:

Screenshot fjeventsnews project 20090122.png

Our Phing script will do two things:

  1. Copy each of these files to the appropriate folder in the test project (called JEvents_dev).
  2. Create a ZIP and "tar.gz" archive file for installation.

Here is the script:

<?xml version="1.0" encoding="UTF-8"?>
<project name="FJEventsNews" basedir="." default="copy_to_test">
 
        <property name="test" value="../Jevents_dev" override="true" />
        <property name="testadmincomp"
                value="../Jevents_dev/administrator/components/com_fjeventsnews"
                override="true" />
        <property name="src" value="${project.basedir}" override="true" />
        <fileset dir="./site" id="component_files">
                <include name="**" />
        </fileset>
        <fileset dir="${src}" id="allfiles">
                <include name="admin/*" />
                <include name="language/*.*" />
                <include name="language/admin/*.*" />
                <include name="site/helpers/*" />
                <include name="site/models/*" />
                <include name="site/views/fjeventsnews/tmpl/*.*" />
                <include name="site/views/fjeventsnews/*.*" />
                <include name="site/views/*.*" />
                <include name="site/*.*" />
                <include name="fjeventsnews.xml" />
        </fileset>
 
        <!-- Copy Target -->
        <target name="copy_to_test" description="copy target">
                <echo>Copying files from dev to test...
                </echo>
                <copy file="${src}/admin/admin.fjeventsnews.php" 
                        todir="${testadmincomp}" />
                <copy file="${src}/admin/index.html" todir="${testadmincomp}" />
                <copy file="${src}/language/admin/en-GB.com_fjeventsnews.ini"
                        todir="${test}/administrator/language/en-GB" />
                <copy file="${src}/language/en-GB.com_fjeventsnews.ini" 
                        todir="${test}/language/en-GB" />
                <echo msg="Copying site files if needed..." />
                <copy todir="${test}/components/com_fjeventsnews">
                        <fileset refid="component_files" />
                </copy>
                <copy file="${src}/fjeventsnews.xml" todir="${testadmincomp}" />
                <delete file="${src}/com_fjeventsnews_dev.zip" />
                <zip destfile="${src}/com_fjeventsnews_dev.zip">
                        <fileset refid="allfiles" />
                </zip>
                <tar destfile="${src}/com_fjeventsnews_dev.tar.gz" compression="gzip">
                        <fileset refid="allfiles" />
                </tar>
        </target>
</project>

This script uses the same commands we have seen in the earlier examples. It is just longer because we are working with more files. Here are some notes about the script:

  • The first three property elements just define variables we will use later in the script.
  • We define two "fileset" elements.
    • The first one, called "component_files", is used to copy all of the files under the "site" folder to the test folder, "components/com_fjeventsnews".
    • The second "fileset" element is "allfiles". We use this to create the two archive files. Notice that this uses "*.*" in some places. This is to exclude folder names. If the "*" is used, for example, in "site/*", then empty folders are included in the ZIP archive and this causes errors in the Joomla! extension install. By using "*.*", we only include files, since all of the files have extensions.
    • Notice also that we can have multiple include statements. We could also use "exclude" statements if we needed them to exclude one or more files or folders.
  • This file only has one "target" element, which is the default target. If we wanted, we could break this up into two or more "target" elements and execute them separately. Then we could use the command arguments -f filename targetname to call a specific target. For example, if we had a target in the file called "create_archive", we could enter -f copy_to_test.xml create_archive to execute the "create_archive" target instead of the default "copy_to_test" target.

As stated earlier, there are many more advanced capabilities of Phing which are documented in the Phing website.

Inserting Version Control Info in Your Files

If you ever edited one of Joomla's files you might certainly have noticed a comment block at the top of the file like this:

/**
* @version      $Id: index.php 10381 2008-06-01 03:35:53Z pasamio $
* @package      Joomla
* @copyright    Copyright (C) 2005 - 2008 Open Source Matters. All rights reserved.
* @license      GNU/GPL, see LICENSE.php
* Joomla! is free software. This version may have been modified pursuant
* to the GNU General Public License, and as distributed it includes or
* is derivative of works licensed under the GNU General Public License or
* other free or open source software licenses.
* See COPYRIGHT.php for copyright notices and details.
*/

What we are going to address here is the very first line of that block.

Note that it contains the name of the file, the build number of the last time the file was committed, the date and time of the last commit and the username of the person who committed the file.

The question is: "How can I use it in my own SVN managed extension project?"

Actually, it's quite easy to accomplish with the keyword replacement feature in SVN that we will explain here!

The first step is to put a comment block like the one above in your file. Actually, you don' have to use the same structure as the files in Joomla, but it would be better to maintain consistency and to make possible for your extension to be documented by phpDOC like the Joomla! 1.5 API Reference. But, to achieve our goal, it's enough to have a comment block like:

/**
* $Id$
*/

Notice the two "$" signs and the uppercase "I" letter. Place this comment block in every file you want to have the keyword replacement to take palace. It doesn't matter where you put the block, but the top of the file seams to be more logical.

After placing the comment block in all the files, you are ready for the replacement to occur, but SVN won't do anything without being explicitly told to. So, let's move to next step.

For the keyword replacement to occur, it's necessary to instruct SVN through a special property named "svn:keywords". Notice that this property applies only to files, not folders, so you have to set it for each and every file you want to have the replacement to take place, just as you did with the comment block. Also, never set this property for binary files. If you do, Eclipse may behave oddly when you make a commit.

To do it just highlight the file you want to set the property for.

Screenshot eclipse svn property 01.png

If the file isn't versioned yet you'll have to make it versioned to be able to set the property. For this, right-click the file and select Team → Add To Version Control. You'll see that a blue plus sign will appear at the bottom-right-hand of the file's icon.

Screenshot eclipse svn property 02.png

Now it's time to set the property for the file. At the bottom of the edit area, there's a set of tabs. One of them might read "SVN Properties". Select that tab, right click inside of it and select "Add a Property" from the menu. If this tab isn't showing you can always right-click the file and choose Team → Set Property.

Screenshot eclipse svn property 03.png

In the window that opens just type svn:keywords for the property's name and Id (notice the capitalized I letter) in the "Enter a text property" text box. Select OK.

Screenshot eclipse svn property 04.png
Screenshot eclipse svn property 05.png

You're set. Next time you make a commit, right after finishing the commit process, SVN will replace the $Id$ tag in your comment block with all the information you see in Joomla! files. Also, after committing the first time, if anybody else makes a change to that file and commits it, the next time you update your working copy the changes will be reflected in the comment block.

There is a quicker way to set the property for a set of files at once. Instead of highlighting each file, highlight the folder that contains them. Then proceed as explained above with the difference that in the "Set Property" window you'll have to check the "set property recursively" select box at the bottom of the window. In this case, make sure the folder doesn't contain any binary files for the reason we mentioned before. If inadvertedly you set the property for a binary file, just select the file/files, right-click svn:keywords property in the "SVN properties" tab and delete it.

For more information about SVN properties, check Version Control with Subversion - Chapter 9

Tips

  • If you accidentally overwrite or delete a file, remember that Eclipse automatically keeps a local history of all files in a project. You can right-click on any file and select either Compare With → Local History or Replace With → Local History to recover any lost edits.
  • If you delete a file and need to restore it, you can right-click on a project and select Restore From Local History. This will let you recover any deleted files.
  • If you have multiple Phing scripts, including a "build.xml" script, you may need to check the setting "Build before launch" on the Build tab of the External Tools Configuration. For example, if this is checked for a Phing file called "copy_to_test.xml", the "build.xml" will be run each time you run "copy_to_test.xml", which may not be what you want.