Forrestbot - automated building and deploying

Forrestbot lets you automate building and deploying websites. The whole process gets the source docs, builds the site, then deploys it where you want it to go. Forrestbot can also notify you afterwards, and it keeps a log of the build process. To accomplish these tasks, Forrestbot uses four "workstages" (getsrc, build, deploy, notify). Each workstage has various implementations (e.g., getsrc has getsrc.cvs or getsrc.svn or getsrc.local implementations), which have various properties that may be set, depending upon the implementations chosen.

You need to create a customized buildfile directing Forrestbot's work and then simply execute:

forrest -f mybuildfile.xml

This project buildfile is simply an Ant buildfile with specific targets that control Forrestbot's operation. The next section explains how to create such a buildfile. For details on the syntax of Ant buildfiles and the operation of Ant itself consult the Ant documentation.

Within the new buildfile, you need to first set properties needed by the workstages you are going to use and then specify which implementations will be used by each workstage. Note that the properties need to be set at the global scope (as children of <project>, i.e., outside of <target> elements) in order for your settings to override the defaults in the Forrestbot implementation. Other than that, the property definitions can appear anywhere before the <import> task.

This sample buildfile can be used as a base from which to customize your own buildfile. The file starts with the project name and default target, then sets the specific properties we need to get the source, indicate the deployment location, and set up notification. It then specifies which implementations we will use and finishes up with importing the forrestbot.xml file. The 'main' target, which is specified as the default here, is a convenience target (defined in forrestbot.xml) that executes the four workstages (getsrc, build, deploy, notify) sequentially.

<project name="mysampleproject" default="main">
  <property name="notify.email.host" value="smtp.example.org"/>
  <property name="notify.email.to" value="me@domain.com"/>
  <property name="notify.administrator" value="Your Name"/>
  <property name="getsrc.cvs.user" value="anoncvs"/>
  <property name="getsrc.cvs.password" value="anoncvs"/>
  <property name="getsrc.cvs.root" value="/home/cvspublic"/>
  <property name="getsrc.cvs.host" value="cvs.example.org"/>
  <property name="getsrc.cvs.module" value="myproject"/>
  <property name="deploy.scp.dest" 
      value="username@example.org:/var/www/example.org/htdocs"/>

  <!-- here we declare the ssh keyfile and passphrase in an external file -->
  <import file="../deploy-settings.xml" optional="true"/>

  <!-- here we specify to use two notification implementations -->
  <target name="notify" depends="notify.local, notify.email"/>

  <!-- here we specify to deploy with the scp implementation -->
  <target name="deploy" depends="deploy.scp"/>

  <!-- the default implementation for getsrc is getsrc.cvs,
       which is what we want -->

  <!-- assumes FORREST_HOME has been set as an environment variable -->
  <property environment="env"/>
  <import file="${env.FORREST_HOME}/tools/forrestbot/core/forrestbot.xml"/>
</project>

      

Workstages

It is only necessary to include specific target implementations in the buildfile if we want to override the default implementations. The following table shows the various workstages, which implementations may be used for each, and which one is the default.

If you want to do more advanced processing for your project, you can override the 'main' target, which by default is <target name="main" depends="getsrc, build, deploy, notify"/>, create your own implementation of a workstage, or use any other ant tasks to do additional work. In order to create your own workstage implementation, define the workstage target in question in your mybuildfile.xml anywhere before the <import> task. This will override the default implementation provided by Forrestbot.

Also, you can choose a different target as the default by changing the default attribute of <project>. For example, you will much more frequently do a 'build' without a 'deploy' during the development of your website, and only at the end do an actual 'deploy', so you might want to choose 'build' as your default target.

Deploying only modified files

Use the checkums feature of the Cocoon CLI. This enables Forrest to keep track of which generated files have actually been changed. The Ant tasks used by Forrestbot will then deploy only the modified files. Ant keeps a cache.properties file. If you need to deploy all files then remove this file and let it be re-generated.

Note that the deplo.svn and deploy.cvs workstages handle modified files with their own mechanism.

Correct Use of getsrc.local

There is a wrinkle when using the 'getsrc.local' implementation of the 'getsrc' workstage. If you define your own 'getsrc.local' target, make sure it starts with the <property> task given here:

<target name="getsrc.local">
  <property name="build.home-dir" location="${getsrc.local.root-dir}"/>
  [...]
</target>

Alternatively (and preferably), define your 'getsrc' target like this:

<target name="getsrc" depends="getsrc.clean-workdir, getsrc.get, getsrc.local"/>

and then implement the actual fetching of the sources in the 'getsrc.get' target. This latter approach is safer since it is more likely to be forward-compatible with future versions of Forrestbot.

Workstage Properties

Each workstage implementation is configurable with properties. The following tables describe each property and whether or not you are required to set it in your buildfile.

Many workstage properties use usernames and passwords. You may want to keep them out of your project's Ant buildfile (especially if you store that file in CVS or SVN). A nice way to do this is to create a separate properties file (e.g., auth.properties) that just sets those properties (don't include it in CVS/SVN). Then, at the top of your project buildfile, have <property file="auth.properties"/>.

Misc. Properties

getsrc.clean-workdir

This should be executed before a getsrc implementation is executed, e.g., <target name="getsrc" depends="getsrc.clean-workdir, getsrc.svn"/>.

getsrc.local

getsrc.cvs

getsrc.svn

build.forrest

deploy.local

deploy.scp

The ${user.home}/.ssh/known_hosts must properly recognize the host, so you should manually make an ssh connection to the host if you never have before.

If ${deploy.scp.keyfile}is defined, then it will use key-based authentication in preference. Otherwise it will use ${deploy.scp.password}

deploy.cvs

This is only available on *nix operating systems.

deploy.svn

This uses the "svn" executable expected to be present, and its authorisation mechanism.

If your "svn username" is different from your local system username create $FORREST_HOME/deploy.svn.settings file like:

<?xml version="1.0"?>
<project>
  <property name="deploy.svn.user" value="myApacheUsername"/>
</project>

deploy.ftp

notify

These settings are used by all notify implementations.

notify.local

No properties.

notify.email

Forrest and Forrestbot use Ant buildfiles extensively. Ant 1.6's <import> task is used to import multiple buildfiles into a single build. The following is the flow of control when running Forrestbot:

• Your project buildfile (mybuildfile.xml)
  • $FORREST_HOME/tools/forrestbot/core/forrestbot.xml
    • Workstage buildfiles ($FORREST_HOME/tools/forrestbot/core/{getsrc,build,deploy,notify}.xml)
    • $FORREST_HOME/main/forrest.build.xml

The workstage buildfiles define the default workstage implementations and set up the properties and files so that targets in the main Forrest buildfile (forrest.build.xml) will run. After those targets are executed, the targets in the workstage buildfiles can perform reporting, deployment, or other post-build activities.

Your project buildfile specifies the workstages you want to use, sets properties for them, and does any additional pre- and post-processing. In addition, you can override the default workstage implementations by defining the relevant targets in your project buildfile before the <import> task (see the example above).

• How to deploy documentation with the Forrestbot "svn" workstage
• How to deploy documentation with the Forrestbot "scp" workstage