Test::AutoBuild(3) User Contributed Perl Documentation Test::AutoBuild(3)NAMETest::AutoBuild - Automated build engine controller
SYNOPSIS
use Test::AutoBuild;
use Config::Record;
my $config = new Config::Record (file => $filename);
my $builder = new Test::AutoBuild (config => $config);
my ($status, $log) = $builder->run($timestamp);
if ($status) {
print STDERR $log, "\n";
}
exit $status;
DESCRIPTION
This module provides the build controller, tieing together various
subsystems to form an integrated engine. It is wholely reponsible for
loading the various runtime objects (stages, modules, repositories,
package types, monitors, publishers) based on their definitions in the
configuration file and then invoking the build. This object does not,
however, contain any logic pertaining to how the build is run, since
this is all delegated to the stages defined in the configuration file.
SETUP
After installing the modules, the first setup step is to create an
unprivileged user to run the build as. By convention the user is called
'builder', in a similarly named group and a home directory of
/var/lib/builder. So as root, execute the following commands:
$ groupadd builder
$ useradd -g builder -m -d /var/lib/builder builder
NB, with the combined contents of the source checkout, the cache and
the virtual installed root, and HTTP site, the disk space requirements
can be pretty large for any non-trivial software. Based on the
applications being built, anywhere between 100MB and many GB of disk
space make be neccessary. For Linux, making /var/lib/builder a
dedicated partition with LVM (Logical Volume Manager) will enable
additional space to be easily grafted on without requiring a re-build.
The next step is to create the basic directory structure within the
user's home directory for storing the various files. There are
directories required for storing the source code, a virtual root
directory for installing files, a build archive, package spool
directories, and publishing directories for HTTP and FTP servers. To
facilitate quick setup, a script is provided to create all the required
directories. Run this script as the unprivileged user
$ su - builder
$ auto-build-make-root /var/lib/builder
It will display a list of all the directories it creates, but for
advance reference, they are
/var/lib/builder/install-root
/var/lib/builder/source-root
/var/lib/builder/log-root
/var/lib/builder/build-archive
/var/lib/builder/package-root
/var/lib/builder/package-root/rpm
/var/lib/builder/package-root/rpm/BUILD
/var/lib/builder/package-root/rpm/RPMS
/var/lib/builder/package-root/rpm/RPMS/noarch
/var/lib/builder/package-root/rpm/RPMS/i386
/var/lib/builder/package-root/rpm/RPMS/i486
/var/lib/builder/package-root/rpm/RPMS/i586
/var/lib/builder/package-root/rpm/RPMS/i686
/var/lib/builder/package-root/rpm/RPMS/x86_64
/var/lib/builder/package-root/rpm/RPMS/ia32e
/var/lib/builder/package-root/rpm/RPMS/ia64
/var/lib/builder/package-root/rpm/RPMS/sparc
/var/lib/builder/package-root/rpm/SPECS
/var/lib/builder/package-root/rpm/SOURCES
/var/lib/builder/package-root/rpm/SRPMS
/var/lib/builder/package-root/zips
/var/lib/builder/package-root/tars
/var/lib/builder/package-root/debian
CONFIGURATION
The configuration file determines all aspects of operation of the build
engine, from the modules built, through the package types detected,
archival method, to build workflow stages, and much more. The example
build configuration file installed by default should provide a fully
functional build instance running under /var/lib/builder, which is
capable of building Test-AutoBuild, versions 1.0.x and 1.1.x, along
with the AutoBuild-Applet. A good sanity check for correct
installation, is to ensure that the example build configuration
succeeds when run.
The configuration file is split into a number of logical groups, which
will be considered in turn below. The minimal level of configuration to
get started involves editing the list of modules, along with the source
repository definitions.
General runtime
The following options define miscellaneous aspects of the build engine
runtime environment.
root
The "root" option is a grouping under which core directories of the
build engine are defined.
root = {
... nested options...
}
The following nested options are permitted with the "root" option
source
The location into which modules' source code will be checked
out from version control. If not specified this option defaults
to the location $HOME/source-root
root = {
...
source = /var/lib/builder/source-root
...
}
Thus, a module with a name of 'dbus-dev' would be checked out
into the directory
/var/lib/builder/source-root/dbus-dev
install
The location into which a module's build process would install
files to be used by dependant modules later in the build cycle.
This location is made available to a module's build control
file via the environment variable $AUTOBUILD_INSTALL_ROOT. If
not specified this option defaults to the location
$HOME/install-root
root = {
...
install = /var/lib/builder/install-root
...
}
Consider, for example, a module 'mozilla' which depends on a
library 'openssl'. The 'openssl' module would be listed as a
dependant module so that it is built first. The build of
'openssl' would install itself into the install root, perhaps
by passing the 'prefix' argument to a configure script:
./configure --prefix=$AUTOBUILD_INSTALL_ROOT
The later build of mozilla, would then build against this
version of openssl, by using
./configure --with-openssl=$AUTOBUILD_INSTALL_ROOT
package
The location in which a module's build process will create any
binary packages it generates, for example RPMs, or Debian
packages. The packages are typically placed into a package
type specific sub-directory. This location is made available to
a module's build control file via the environment variable
$AUTOBUILD_PACKAGE_ROOT. If not specified, this option
defaults to the location $HOME/package-root
root = {
...
package = /var/lib/builder/package-root
...
}
Consider, for example, a module which generates an RPM, of
itself. The $AUTOBUILD_PACKAGE_ROOT directory would be used to
set the '_topdir' macro for the RPM build process
rpmbuild --define '_topdir $AUTOBUILD_PACKAGE_ROOT/rpm' -ta foo.tar.gz
log The location in which the output from a module's build control
file will be spooled during execution. If not specified, this
option defaults to the location $HOME/log-root. The control
file's standard output and error streams will be combined into
one.
root = {
...
log = /var/lib/builder/log-root
...
}
admin-email
The email address of the build engine administrator, typically
linked from the bottom of the HTML status pages. This is also the
address spammed with build status alerts if the
Test::AutoBuild::Stage::EmailAlert module is in use.
admin-email = admin@example.com
admin-name
The full name of the build engine administrator, typically
displayed on the bottom of the HTML status pages.
admin-name = John Doe
log4perl
A configuration block controlling the output of debug information
during execution of the build. The data here is passed straight
through to the "init" method in the Log::Log4perl module, so
consult that module's manual page for possible configuration
options. The example setting, enables display of progress through
the build workflow. To get maximum possible debug information,
change the "log4perl.rootLogger" option to 'DEBUG' instead of
'WARN'.
log4perl = {
log4perl.rootLogger = WARN, Screen
# To get progress updates
log4perl.logger.Test.AutoBuild.Monitor.Log4perl = INFO
log4perl.appender.Screen = Log::Log4perl::Appender::Screen
log4perl.appender.Screen.stderr = 1
log4perl.appender.Screen.layout = Log::Log4perl::Layout::SimpleLayout
}
counter
This configuration block determines the module used to generate the
unique build cycle counter.
counter = {
..nested options..
}
The nested options allowed within this block are
module
The full package name of the subclass of
Test::AutoBuild::Counter used to generate the build cycle
counter. Consult that module for a list of known
implementations. The exmaple configuration sets the build
counter to match the timestamp taken at the start of the build
cycle.
module = Test::AutoBuild::Counter::Timestamp
options
This is defines a set of options as key, value pairs which are
passed to the counter object created. The valid keys for this
are specific to the package specified in the "module" parameter
above, so consult the manual page corresponding to the module
defined there. If using the
"Test::AutoBuild::Counter::ChangeList" class, and there is a
source repository named, "mysvn", one would set an option such
as
options = {
repository = mysvn
}
Source repositories
The "repositories" configuration block, defines the source repositories
from which modules are checked out. The keys in the block form the
short name of the repository, and is referenced later when defining the
paths for modules' source checkout
repositories = {
myrepo = {
... definition of myrepo ...
}
otherrepo = {
... definition of otherrepo ...
}
}
Within each repository definition the following options are supported
label
The free-text display name of the repository, typically used in the
HTML status pages.
module
The full package name of a subclass of
"Test::AutoBuild::Repository" which implements the checkout
procedure for the repository type. There are repository types for
common version control systems such as Subversion, CVS, Perforce
and GNUArch, as well as a simple non-versioned repository. Refer
to the Test::AutoBuild::Repository module for a list of known
repository types and their corresponding package names.
module = Test::AutoBuild::Repository::Subversion
env Lists a set of environment variables which will be set whenever
running any repository commands. The possible environment variable
names vary according to the type of repository, so refer to the
manual page for the repository module defined in the "module"
option. For example, the CVS repository type uses the CVSROOT
environment variable to specify the repository location.
env = {
CVSROOT = :pserver:nonymous@cvs.gna.org:/cvs/testautobuild
}
option
A set of configuration options specific to the type of repository
configure. Again, refer to the manual page for the repository
module defined in the "module" option. For example, the GNU Arch
repository type supports the 'archive-name' option
options = {
archive-name = lord@emf.net--2004
}
Modules
The "modules" configuration block defines the list of modules to be
checked out of source control and built. The keys in the block form the
short names for the modules, used, for example, in creating filenames
for assets relating to the module, and the name of the checkout
directory under the source root. If building multiple branches of a
module, it is common to post-fix the module name with a version /
branch name.
modules = {
mymod-1.0 = {
.. definition of mymod version 1.0..
}
mymod-dev = {
.. definition of mymod development snapshot...
}
}
Within the configuration block of an individual module the following
options are permitted
label
The free-text display name for the module, typically used in HTML
status pages, and email alerts.
label = Test-AutoBuild (Development branch)
source
This block defines the repository containing the source to be
checked out for the module. There are two keys in the block, the
value associated with the key "repository", is the name of a
repository previously defined in the config file. The value
associated with the key "path" is the path to checkout within the
repository. The syntax for path values is dependant on the type of
repository being accessed. For details refer to the manual pages
for the corresponding modules:
CVS Refer to Test::AutoBuild::Repository::CVS
GNU Arch
Refer to Test::AutoBuild::Repository::GNUArch
Subversion
Refer to Test::AutoBuild::Repository::Subversion
Perforce
Refer to Test::AutoBuild::Repository::Perforce
Mercurial
Refer to Test::AutoBuild::Repository::Mercurial
SVK Refer to Test::AutoBuild::Repository::SVK
Local disk
Refer to Test::AutoBuild::Repository::Disk
An example config entry for a module checked out of CVS is
source = {
repository = gna-cvs
path = /testautobuild
}
sources
If a module's source is split amongst several locations, this block
is used instead of the "source" block. It allows defintion of a
list of source paths to checkout. It is a list, where each entry
matches the format of the "source" parameter. For example
sources = (
{
repository = gna-cvs
path = /testautobuild
}
{
repository = local-disk
path = /testautobuild-autobuild.sh -> autobuild.sh
}
)
NB, not all repository types play nicely together when checking out
from multiple paths. Consult manual pages for individual repository
types for futher information
groups
Lists the groups to which the module belongs. The values in the
list must be group names, specified earlier in the top level
"groups" configuration block.
groups = (
software
perl
)
env Defines a set of environment variables which will be set whenever
running the build/test control files for the module. The only
restriction on variables set here, are that none should be named
with the prefix AUTOBUILD_, otherwise they are liable to be
overridden by variables set by the build engine.
env = (
SKIP_TESTS = 1
)
options
The options parameter is used to specify module specific data which
will be used by stages in the workflow engine. Consult the manual
pages for individual stages in use, for further details on which
options are possible. The most common option is "control-file"
which can be used to override the default name of the command to
invoke to run the build. For compatability with version 1.0.x of
autobuild, this should be set to 'rollingbuild.sh'
options = {
control-file = rollingbuild.sh
}
links
The links configuration block defines a simple list of hyperlinks
relating to the module. This is typically used to provide links to
a graphical front end for the source repository, or a link to a
project homepage. The two keys with the block are "href" and
"label"
links = (
{
href = http://www.autobuild.org/
label = Project homepage
}
{
href = http://cvs.gna.org/viewcvs/testautobuild/testautobuild/?only_with_tag=RELEASE_1_0_0
label = Browse source code
}
)
artifacts
The artifacts configuration block defines a list of build artifacts
which will be published to the distribution sites. This is
typically used to provide access to items such as build reports on
code coverage, code complexity, bug analysis, etc, or metadata
files such as the module's README, or copyright notices. With the
block, the "src" parameter is a filename glob relative to the base
of the module's code checkout; the "dst" parameter is the name of
the destination file (or directory if the source glob matches
multiple files), and will also form the URL string; the "label" key
gives a label for hyperlinks to the artifact, and finally the
"publisher" is the name of a file publisher, as defined in the top
level "publishers" config block.
artifacts = (
{
src = README
dst = README
label = Readme
publisher = copy
}
{
src = blib/coverage/*
dst = coverage/
label = Code Test & POD coverage Reports
publisher = copy
}
)
Groups
The following options define grouping of modules, primarily used for
grouping modules in the HTML status display. The keys in the
configuration block for the short group name, used when defining group
membership in the module configuration.
groups = {
perl = {
label = Perl modules
}
software = {
label = Software
}
docs = {
label = Documentation
}
}
The following options are allowed within each group definition
label
The free-text display name of the group
Package types
The following options define binary package types to detect and
publish.
Publishers
The following options define mechanisms for publishing files to
distribution directories.
Platforms
The following options define aspects of the host platform
Build archive
The following options define the mechanism used for archiving module
build output between build cycles.
Workflow stages
The following options defined the workflow followed for a build cycle
METHODS
$builder = Test::AutoBuild->new(config => $config);
Creates a new builder instance, loading configuration parameters
from the $config parameter, whose value is an instance of the
"Config::Record" module.
$config = $builder->config([$name, [$default]]);
If invoked with no arguments returns the Config::Record object
storing the builder configuration. If invoked with a single
argument, returns the configuration value with the matching name.
An optional default value can be provided in the second argument
$builder->run();
Executes the build process. This is the heart of the auto build
engine. It performs the following actions:
* Reads the list of modules, source control repositories,
package types and output modules from the configuration
file
* Initializes the build cache
* Takes out an exclusive file lock to prevent > 1 builder
running at the same time.
* Changes the (nice) priority of the AutoBuild process
* Checks the code for each module out of its respective
source control repository.
* Does a topological sort to determine the build order
for all modules
* For each module to be built:
- Take a snapshot of the package & virtual root install
directories
- Change to the top level source directory of the module
- Run the rollingbuild.sh script
- Take another snapshot & compare to determine which
files were install in the virtual root & which packages
were generated
- Save the intsalled files and packages in the cache.
* Invoke each requested output module, for example, HTML
status generator, package & log file copiers, email
alerts
my \%groups = $autobuild->load_groups();
Creates the "Test::AutoBuild::Publisher" objects for each publisher
defined in the build configuration.
my \%publishers = $autobuild->load_publishers();
Creates the "Test::AutoBuild::Publisher" objects for each publisher
defined in the build configuration.
my \%monitors = $autobuild->load_monitors();
Creates the "Test::AutoBuild::Monitor" objects for each monitor
defined in the build configuration.
my \%platforms = $autobuild->load_platforms();
Creates the "Test::AutoBuild::Platform" objects for each platform
defined in the build configuration.
my \@repositories = $autobuild->load_repositories();
Creates the "Test::AutoBuild::Repository" objects for each
repository defined in the build configuration.
my \%package_types = $autobuild->load_package_types();
Creates the "Test::AutoBuild::PackageType" objects for each package
type defined in the build configuration.
my \%modules = $autobuild->load_modules();
Creates the "Test::AutoBuild::Module" obkjects for each module
defined in the build configuration.
TODO
The task tracker on the Gna! project site (www.autobuild.org) contains
a list of all things we'd like to do.
Oh and 100% Pod & code test coverage - Devel::Cover WILL EAT YOUR
BRAAAAANE!
BUGS
Probably a few, so report them to the bug tracker linked from the Gna!
project site www.autobuild.org.
AUTHORS
Daniel P. Berrange, Dennis Gregorovic
COPYRIGHT
Copyright (C) 2002-2005 Daniel Berrange <dan@berrange.com>
SEE ALSOperl(1), <http://www.autobuild.org>, Test::AutoBuild::Runtime,
Test::AutoBuild::Module, Test::AutoBuild::Stage,
Test::AutoBuild::Repository, Test::AutoBuild::PackageType,
Test::AutoBuild::Publisher, Test::AutoBuild::Repository,
Test::AutoBuild::Counter, Test::AutoBuild::Group,
perl v5.14.1 2011-09-01 Test::AutoBuild(3)
