Declaring elements¶

Element basics¶

Here is a rather complete example using the autotools element kind and git source kind:

# Specify the kind of element this is
kind: autotools

# Specify some dependencies
depends:
- element1.bst
- element2.bst

# Specify the source which should be built
sources:
- kind: git
  url: upstream:modulename.git
  track: master
  ref: d0b38561afb8122a3fc6bafc5a733ec502fcaed6

# Override some variables
variables:
  sysconfdir: "%{prefix}/etc"

# Tweak the sandbox shell environment
environment:
  LD_LIBRARY_PATH: /some/custom/path

# Specify the configuration of the element
config:

  # Override autotools element default configure-commands
  configure-commands:
  - "%{configure} --enable-fancy-feature"

# Specify public domain data, visible to other elements.
public:
  bst:
    integration-commands:
    - /usr/bin/update-fancy-feature-cache

# Specify a user id and group id to use in the build sandbox.
sandbox:
  build-uid: 0
  build-gid: 0

For most use cases you would not need to specify this much detail, we’ve provided details here in order to have a more complete initial example.

Let’s break down the above and give a brief explanation of what these attributes mean.

Element names and paths¶

An element name is the filename of an element relative to the project’s element path.

Element names are the identifiers used to refer to elements, they are used to specify an element’s dependencies, to select elements to build on the command line, and they are arbitrarily used in various element specific configuration surfaces, for example the target configuration of the link element is also an element name.

Addressing elements¶

When addressing elements in a single project, it is sufficient to use the element name as a dependency or configuration parameter.

When muliple projects are connected through junction elements, there is a need to address elements which are not in the same project but in a junctioned subproject. In the case that you need to address elements across junction boundaries, one must use element paths.

An element path is a path to the element indicating the junction elements leading up to the project, separated by : symbols, e.g.: junction.bst:element.bst.

Elements can be address across multiple junction boundaries with multiple : separators, e.g.: junction.bst:junction.bst:element.bst.

Element naming rules¶

When naming the elements, use the following rules:

The name of the file must have .bst extension.
All characters in the name must be printable 7-bit ASCII characters.
Following characters are reserved and must not be part of the name:
- < (less than)
- > (greater than)
- : (colon)
- " (double quote)
- / (forward slash)
- \ (backslash)
- | (vertical bar)
- ? (question mark)
- * (asterisk)

Kind¶

# Specify the kind of element this is
kind: autotools

The kind attribute specifies which plugin will be operating on the element’s input to produce its output. Plugins define element types and each of them can be referred to by name with the kind attribute.

To refer to a third party plugin, prefix the plugin with its package, for example:

kind: buildstream-plugins:dpkg_build

Depends¶

# Specify some dependencies
depends:
- element1.bst
- element2.bst

Relationships between elements are specified with the depends attribute. Elements may depend on other elements by specifying the element names they depend on here.

See Dependencies for more information on the dependency model.

Build-Depends¶

# Specify some build-dependencies
build-depends:
- element1.bst
- element2.bst

Build dependencies between elements can be specified with the build-depends attribute. The above code snippet is equivalent to:

# Specify some build-dependencies
depends:
- filename: element1.bst
  type: build
- filename: element2.bst
  type: build

See Dependencies for more information on the dependency model.

Runtime-Depends¶

# Specify some runtime-dependencies
runtime-depends:
- element1.bst
- element2.bst

Runtime dependencies between elements can be specified with the runtime-depends attribute. The above code snippet is equivalent to:

# Specify some runtime-dependencies
depends:
- filename: element1.bst
  type: runtime
- filename: element2.bst
  type: runtime

See Dependencies for more information on the dependency model.

Sources¶

# Specify the source which should be built
sources:
- kind: git
  url: upstream:modulename.git
  track: master
  ref: d0b38561afb8122a3fc6bafc5a733ec502fcaed6

Here we specify some input for the element, any number of sources may be specified. By default the sources will be staged in the root of the element’s build directory in the build sandbox, but sources may specify a directory attribute to control where the sources will be staged. The directory attribute may specify a build sandbox relative subdirectory.

For example, one might encounter a component which requires a separate data package in order to build itself, in this case the sources might be listed as:

sources:

# Specify the source which should be built
- kind: git
  url: upstream:modulename.git
  track: master
  ref: d0b38561afb8122a3fc6bafc5a733ec502fcaed6

# Specify the data package we need for build frobnication,
# we need it to be unpacked in a src/frobdir
- kind: tarball
  directory: src/frobdir
  url: data:frobs.tgz
  ref: 9d4b1147f8cf244b0002ba74bfb0b8dfb3...

Like Elements, Source types are plugins which are indicated by the kind attribute. Asides from the common kind and directory attributes which may be applied to all Sources, refer to the Source specific documentation for meaningful attributes for the particular Source.

Variables¶

# Override some variables
variables:
  sysconfdir: "%{prefix}/etc"

Variables can be declared or overridden from an element. Variables can also be declared and overridden in the Project configuration

See Using variables below for a more in depth discussion on variables in BuildStream.

Environment¶

# Tweak the sandbox shell environment
environment:
  LD_LIBRARY_PATH: /some/custom/path

Environment variables can be set to literal values here, these environment variables will be effective in the Sandbox where build instructions are run for this element.

Environment variables can also be declared and overridden in the Project configuration

Config¶

# Specify the configuration of the element
config:

  # Override autotools element default configure-commands
  configure-commands:
  - "%{configure} --enable-fancy-feature"

Here we configure the element itself. The autotools element provides sane defaults for building sources which use autotools. Element default configurations can be overridden in the project.conf file and additionally overridden in the declaration of an element.

For meaningful documentation on what can be specified in the config section for a given element kind, refer to the element specific documentation.

Public¶

# Specify public domain data, visible to other elements.
public:
  bst:
    integration-commands:
    - /usr/bin/update-fancy-feature-cache

Metadata declared in the public section of an element is visible to any other element which depends on the declaring element in a given pipeline. BuildStream itself consumes public data from the bst domain. The integration-commands demonstrated above for example, describe commands which should be run in an environment where the given element is installed but before anything should be run.

An element is allowed to read domain data from any element it depends on, and users may specify additional domains to be understood and processed by their own element plugins.

The public data keys which are recognized under the bst domain can be viewed in detail in the builtin public data section.

Sandbox¶

Configuration for the build sandbox (other than environment variables) can be placed in the sandbox configuration. The UID and GID used by the user in the group can be specified, as well as the desired OS and machine architecture. Possible machine architecture follow the same list as specified in the architecture option.

# Specify a user id and group id to use in the build sandbox.
sandbox:
  build-uid: 1003
  build-gid: 1001

BuildStream normally uses uid 0 and gid 0 (root) to perform all builds. However, the behaviour of certain tools depends on user id, behaving differently when run as non-root. To support those builds, you can supply a different uid or gid for the sandbox. Only bwrap-style sandboxes support custom user IDs at the moment, and hence this will only work on Linux host platforms.

# Specify build OS and architecture
sandbox:
  build-os: AIX
  build-arch: power-isa-be

When building locally, if these don’t match the host machine then generally the build will fail. The exception is when the OS is Linux and the architecture specifies an x86-32 build on an x86-64 machine, or aarch32 build on a aarch64 machine, in which case the linux32 command is prepended to the bubblewrap command.

When building remotely, the OS and architecture are added to the Platform field in the Command uploaded. Whether this actually results in a building the element for the desired OS and architecture is dependent on the server having implemented these options the same as buildstream.

Dependencies¶

The dependency model in BuildStream is simplified by treating software distribution and software building as separate problem spaces. This is to say that one element can only ever depend on another element but never on a subset of the product which another element produces.

In this section we’ll quickly go over the few features BuildStream offers in its dependency model.

Expressing dependencies¶

Dependencies in BuildStream are parameterizable objects, however as demonstrated in the above example, they can also be expressed as simple strings as a convenience shorthand in most cases, whenever the default dependency attributes are suitable.

Note

Note the order in which element dependencies are declared in the depends, build-depends and runtime-depends lists are not meaningful.

Dependency dictionary:

# Fully specified dependency
depends:
- filename: foo.bst
  type: build
  junction: baseproject.bst
  strict: false

Attributes:

filename

The element name to depend on.
type

This attribute is used to express the dependency type. This field is not permitted in Build-Depends or Runtime-Depends.
junction

This attribute can be used to specify the junction portion of the element name separately from the project local element name.

This should be the element name of the junction element in the local project, possibly followed by other junctions in subprojects leading to the project in which the element you want to depend on resides.

In the case that a junction is specified, the filename attribute indicates an element in the junctioned project.
strict

This attribute can be used to specify that this element should be rebuilt when the dependency changes, even when strict mode has been turned off.

This is appropriate whenever a dependency’s output is consumed verbatim in the output of the depending element, for instance when static linking is in use.

Cross-junction dependencies¶

As explained in the element name section on element addressing, elements can be addressed across junction boundaries using element paths such as junction.bst:element.bst. An element at any depth can be specified by specifying multiple junction elements.

For example, one can specify a subproject element dependency with the following syntax:

build-depends:
- baseproject.bst:element.bst

And one can specify an element residing in a sub-subproject as a dependency like so:

depends:
- baseproject.bst:middleproject.bst:element.bst

Dependency types¶

The dependency type attribute defines what the dependency is required for and is essential to how BuildStream plots a build plan.

There are three types which one can specify for a dependency:

build

A build dependency type states that the given element’s product must be staged in order to build the depending element. Depending on an element which has build dependencies will not implicitly depend on that element’s build dependencies.
runtime

A runtime dependency type states that the given element’s product must be present for the depending element to function. An element’s runtime dependencies are not available to the element at build time.
all

An all dependency is the default dependency type. If all is specified, or if type is not specified at all, then it is assumed that the dependency is required both at build time and runtime.

Note

It is assumed that a dependency which is required for building an element must run while building the depending element. This means that build depending on a given element implies that that element’s runtime dependencies will also be staged for the purpose of building.

Using variables¶

Variables in BuildStream are a way to make your build instructions and element configurations more dynamic.

Referring to variables¶

Variables are expressed as %{...}, where ... must contain only alphanumeric characters and the separators _ and -. Further, the first letter of ... must be an alphabetic character.

This is release version %{version}

Declaring and overriding variables¶

To declare or override a variable, one need only specify a value in the relevant variables section:

variables:
  hello: Hello World

You can refer to another variable while declaring a variable:

variables:
  release-text: This is release version %{version}

The order in which you declare variables is arbitrary, so long as there is no cyclic dependency and that all referenced variables are declared, the following is fine:

variables:
  release-text: This is release version %{version}
  version: 5.5

Note

It should be noted that variable resolution only happens after all Element Composition has already taken place.

This is to say that overriding %{version} at a higher priority will affect the final result of %{release-text}.

Example:

kind: autotools

# Declare variable, expect %{version} was already declared
variables:
  release-text: This is release version %{version}

config:

  # Customize the installation
  install-commands:
  - |
    %{make-install} RELEASE_TEXT="%{release-text}"

Variables declared by BuildStream¶

BuildStream declares a set of builtin variables that may be overridden. In addition, the following read-only variables are also dynamically declared by BuildStream:

element-name

The name of the element being processed (e.g base/alpine.bst).
project-name

The name of project where BuildStream is being used.
max-jobs

Maximum number of parallel build processes within a given build, support for this is conditional on the element type and the build system used (any element using ‘make’ can implement this).