3. Using the autotools element¶
In the last chapter we observed how the
manual element works, allowing one to specify and
run commands manually in the process of constructing an artifact.
In this chapter, we’ll go over a mostly automated build of a similar
hello world example. We will observe how our configurations of the
autotools element translate to configurations
on the manual element, and observe how
variable substitution works.
Note
This example is distributed with BuildStream in the doc/examples/autotools subdirectory.
3.1. Overview¶
Instead of using the local source as we have been using
in the previous examples, we’re going to use a tar source
this time to obtain the automake release tarball directly from the upstream
hosting.
In this example we’re going to build the example program included in the
upstream automake tarball itself, and we’re going to use the automated
autotools build element to do so.
3.2. Project structure¶
3.2.1. project.conf¶
# Unique project name
name: autotools
# Minimum required BuildStream version
min-version: 2.0
# Subdirectory where elements are stored
element-path: elements
# Define some aliases for the tarballs we download
aliases:
alpine: https://bst-integration-test-images.ams3.cdn.digitaloceanspaces.com/
gnu: http://ftpmirror.gnu.org/gnu/automake/
Like the last project.conf, we’ve
added another source alias for gnu, the location
from which we’re going to download the automake tarball.
3.2.2. elements/base/alpine.bst and elements/base.bst¶
The alpine base and base stack element are defined in the same way as in the last chapter: Running commands.
3.2.3. elements/hello.bst¶
kind: autotools
description: |
Hello world example from automake
variables:
# The hello world example lives in the doc/amhello folder.
#
# Set the %{command-subdir} variable to that location
# and just have the autotools element run its commands there.
#
command-subdir: doc/amhello
sources:
- kind: tar
url: gnu:automake-1.16.tar.gz
ref: 80da43bb5665596ee389e6d8b64b4f122ea4b92a685b1dbd813cd1f0e0c2d83f
depends:
- base.bst
In this case, we haven’t touched the element’s config section
at all, instead we just slightly override the bahavior of the
autotools build element by overriding
the command-subdir variable
3.2.3.1. Looking at variables¶
Let’s take a moment and observe how element composition works with variables.
As the documentation mentions:
The initial settings of the
project.confvariables are setup using BuildStream’s builtin defaults.After this, your local
project.confmay override some variables on a project wide basis. Those will in turn be overridden by any defaults provided by element classes, such as the variables set in the documentation of theautotoolsbuild element. The variables you set in your final<element.bst>element declarations, will have the final say on the value of a particular variable.Finally, the variables, which may be composed of other variables, are resolved after all composition has taken place.
The variable we needed to override was command-subdir, which is an
automatic variable provided by the BuildElement
abstract class. This variable simply instructs the BuildElement
in which subdirectory of the %{build-root} to run its commands in.
One can always display the resolved set of variables for a given element’s configuration using bst show:
user@host:~/autotools$ bst show --deps none --format "%{vars}" hello.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
prefix: /usr
exec_prefix: /usr
bindir: /usr/bin
sbindir: /usr/sbin
libexecdir: /usr/libexec
datadir: /usr/share
sysconfdir: /etc
sharedstatedir: /usr/com
localstatedir: /var
lib: lib
libdir: /usr/lib
debugdir: /usr/lib/debug
includedir: /usr/include
docdir: /usr/share/doc
infodir: /usr/share/info
mandir: /usr/share/man
build-root: /buildstream/autotools/hello.bst
conf-root: .
install-root: /buildstream-install
strip-binaries: ''
fix-pyc-timestamps: "find \"/buildstream-install\" -name '*.pyc' -exec \\\n dd if=/dev/zero\
\ of={} bs=1 count=4 seek=4 conv=notrunc ';'"
project-name: autotools
max-jobs: '2'
autogen: "export NOCONFIGURE=1;\n\nif [ -x ./configure ]; then true;\nelif [ -x ./autogen\
\ ]; then ./autogen;\nelif [ -x ./autogen.sh ]; then ./autogen.sh;\nelif [ -x ./bootstrap\
\ ]; then ./bootstrap;\nelif [ -x ./bootstrap.sh ]; then ./bootstrap.sh;\nelse autoreconf\
\ -ivf .;\nfi"
conf-global: ''
conf-local: ''
conf-extra: ''
conf-cmd: ./configure
conf-args: "--prefix=/usr \\\n--exec-prefix=/usr \\\n--bindir=/usr/bin \\\n--sbindir=/usr/sbin\
\ \\\n--sysconfdir=/etc \\\n--datadir=/usr/share \\\n--includedir=/usr/include \\\
\n--libdir=/usr/lib \\\n--libexecdir=/usr/libexec \\\n--localstatedir=/var \\\n\
--sharedstatedir=/usr/com \\\n--mandir=/usr/share/man \\\n--infodir=/usr/share/info\
\ "
configure: "./configure --prefix=/usr \\\n--exec-prefix=/usr \\\n--bindir=/usr/bin\
\ \\\n--sbindir=/usr/sbin \\\n--sysconfdir=/etc \\\n--datadir=/usr/share \\\n--includedir=/usr/include\
\ \\\n--libdir=/usr/lib \\\n--libexecdir=/usr/libexec \\\n--localstatedir=/var \\\
\n--sharedstatedir=/usr/com \\\n--mandir=/usr/share/man \\\n--infodir=/usr/share/info\
\ "
make: make
make-install: make -j1 DESTDIR="/buildstream-install" install
remove-libtool-modules: 'false'
remove-libtool-libraries: 'false'
delete-libtool-archives: "if false || false; then\n find \"/buildstream-install\"\
\ -name \"*.la\" -print0 | while read -d '' -r file; do\n if grep '^shouldnotlink=yes$'\
\ \"${file}\" &>/dev/null; then\n if false; then\n echo \"Removing ${file}.\"\
\n rm \"${file}\"\n else\n echo \"Not removing ${file}.\"\n \
\ fi\n else\n if false; then\n echo \"Removing ${file}.\"\n \
\ rm \"${file}\"\n else\n echo \"Not removing ${file}.\"\n \
\ fi\n fi\n done\nfi"
command-subdir: doc/amhello
element-name: hello.bst
As an exercise, we suggest that you modify the hello.bst
element to set the prefix like so:
variables:
prefix: "/opt"
And rerun the above bst show command to observe how this changes the output.
Observe where the variables are declared in the builtin defaults and autotools element
documentation, and how overriding these effects the resolved set of variables.
3.3. Using the project¶
3.3.1. Build the hello.bst element¶
To build the project, run bst build in the following way:
user@host:~/autotools$ bst build hello.bst
[--:--:--][ ][ main:core activity ] START Build
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:core activity ] START Checking sources
[00:00:00][ ][ main:core activity ] SUCCESS Checking sources
BuildStream Version 1.93.5
Session Start: Wednesday, 19-08-2020 at 14:56:07
Project: autotools (/home/user/autotools)
Targets: hello.bst
User Configuration
Configuration File: /home/user/.config/buildstream.conf
Cache Directory: /home/user/.cache/buildstream
Log Files: /home/user/.cache/buildstream/logs
Source Mirrors: /home/user/.cache/buildstream/sources
Build Area: /home/user/.cache/buildstream/build
Strict Build Plan: Yes
Maximum Fetch Tasks: 10
Maximum Build Tasks: 4
Maximum Push Tasks: 4
Maximum Network Retries: 2
Project: autotools
Element Plugins
autotools: core plugin
stack: core plugin
import: core plugin
Source Plugins
tar: core plugin
Pipeline
buildable 179c6ae937e8e2ece3192ab8dc2a55053134a591c9ccd37507b56d11885fae23 base/alpine.bst
waiting 7e7bdf36e3d6947ebf279f651fc492c0d20f919e114fb3af8196ac6d2863a8b7 base.bst
waiting 396a624b197576abc3bc6fc7a4162417c83fc1fc08685285f99b3bcaef60f575 hello.bst
===============================================================================
[--:--:--][179c6ae9][ build:base/alpine.bst ] START autotools/base-alpine/179c6ae9-build.7016.log
[--:--:--][179c6ae9][ build:base/alpine.bst ] START Staging sources
[00:00:00][179c6ae9][ build:base/alpine.bst ] SUCCESS Staging sources
[--:--:--][179c6ae9][ build:base/alpine.bst ] START Caching artifact
[00:00:00][179c6ae9][ build:base/alpine.bst ] SUCCESS Caching artifact
[00:00:00][179c6ae9][ build:base/alpine.bst ] SUCCESS autotools/base-alpine/179c6ae9-build.7016.log
[--:--:--][7e7bdf36][ build:base.bst ] START autotools/base/7e7bdf36-build.7023.log
[--:--:--][7e7bdf36][ build:base.bst ] START Caching artifact
[00:00:00][7e7bdf36][ build:base.bst ] SUCCESS Caching artifact
[00:00:00][7e7bdf36][ build:base.bst ] SUCCESS autotools/base/7e7bdf36-build.7023.log
[--:--:--][396a624b][ build:hello.bst ] START autotools/hello/396a624b-build.7027.log
[--:--:--][396a624b][ build:hello.bst ] START Staging dependencies
[00:00:00][396a624b][ build:hello.bst ] SUCCESS Staging dependencies
[--:--:--][396a624b][ build:hello.bst ] START Staging sources
[00:00:00][396a624b][ build:hello.bst ] SUCCESS Staging sources
[--:--:--][396a624b][ build:hello.bst ] START Running commands
export NOCONFIGURE=1;
if [ -x ./configure ]; then true;
elif [ -x ./autogen ]; then ./autogen;
elif [ -x ./autogen.sh ]; then ./autogen.sh;
elif [ -x ./bootstrap ]; then ./bootstrap;
elif [ -x ./bootstrap.sh ]; then ./bootstrap.sh;
else autoreconf -ivf .;
fi
./configure --prefix=/usr \
--exec-prefix=/usr \
--bindir=/usr/bin \
--sbindir=/usr/sbin \
--sysconfdir=/etc \
--datadir=/usr/share \
--includedir=/usr/include \
--libdir=/usr/lib \
--libexecdir=/usr/libexec \
--localstatedir=/var \
--sharedstatedir=/usr/com \
Message contains 23 additional lines
[00:00:04][396a624b][ build:hello.bst ] SUCCESS Running commands
[--:--:--][396a624b][ build:hello.bst ] START Caching artifact
[00:00:00][396a624b][ build:hello.bst ] SUCCESS Caching artifact
[00:00:04][396a624b][ build:hello.bst ] SUCCESS autotools/hello/396a624b-build.7027.log
[00:00:05][ ][ main:core activity ] SUCCESS Build
Pipeline Summary
Total: 3
Session: 3
Fetch Queue: processed 0, skipped 3, failed 0
Build Queue: processed 3, skipped 0, failed 0
3.3.2. Run the hello world program¶
We probably know by now what’s going to happen, but let’s run the program we’ve compiled anyway using bst shell:
user@host:~/autotools$ bst shell hello.bst -- hello
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:hello.bst ] START Staging dependencies
[00:00:00][ ][ main:hello.bst ] SUCCESS Staging dependencies
[--:--:--][ ][ main:hello.bst ] START Integrating sandbox
[00:00:00][ ][ main:hello.bst ] SUCCESS Integrating sandbox
[--:--:--][ ][ main:hello.bst ] STATUS Running command
hello
2020-08-19T14:56:13.074+0000 [8030:140267088308288] [buildboxcommon_runner.cpp:547] [WARNING] The --use-localcas option will be deprecated. LocalCAS support is now enabled by default.
2020-08-19T14:56:13.075+0000 [8030:140267088308288] [buildboxcommon_client.cpp:87] [INFO] Setting d_maxBatchTotalSizeBytes = 4128768 bytes by default
Hello World!
This is amhello 1.0.
3.4. Summary¶
Now we’ve used a builtin build element, and we’ve taken a look into how variables work.
When browsing the build elements in the documentation, we are now equipped with a good idea of what an element is going to do, based on their default YAML configuration and any configurations we have in our project. We can also now observe what variables are in effect for the build of a given element, using bst show.