Plugin - Base plugin class

BuildStream supports third party plugins to define additional kinds of Elements and Sources.

The common API is documented here, along with some information on how external plugin packages are structured.

Abstract Methods

For both Elements and Sources, it is mandatory to implement the following abstract methods:

  • Plugin.configure()

    Loads the user provided configuration YAML for the given source or element

  • Plugin.preflight()

    Early preflight checks allow plugins to bail out early with an error in the case that it can predict that failure is inevitable.

  • Plugin.get_unique_key()

    Once all configuration has been loaded and preflight checks have passed, this method is used to inform the core of a plugin’s unique configuration.

Configurable Warnings

Warnings raised through calling Plugin.warn() can provide an optional parameter warning_token, this will raise a PluginError if the warning is configured as fatal within the project configuration.

Configurable warnings will be prefixed with Plugin.get_kind() within buildstream and must be prefixed as such in project configurations. For more detail on project configuration see Configurable Warnings.

It is important to document these warnings in your plugin documentation to allow users to make full use of them while configuring their projects.

Example

If the git plugin uses the warning "inconsistent-submodule" then it could be referenced in project configuration as "git:inconsistent-submodule".

Plugin Structure

A plugin should consist of a setuptools package that advertises contained plugins using entry points.

A plugin entry point must be a module that extends a class in the Plugin API reference to be discovered by BuildStream. A YAML file defining plugin default settings with the same name as the module can also be defined in the same directory as the plugin module.

Note

BuildStream does not support function/class entry points.

A sample plugin could be structured as such:

.
├── elements
│   ├── autotools.py
│   ├── autotools.yaml
│   └── __init__.py
├── MANIFEST.in
└── setup.py

The setuptools configuration should then contain at least:

setup.py:

from setuptools import setup, find_packages

setup(name='BuildStream Autotools',
      version="0.1",
      description="A better autotools element for BuildStream",
      packages=find_packages(),
      include_package_data=True,
      entry_points={
          'buildstream.plugins.elements': [
              'autotools = elements.autotools'
          ]
      })

MANIFEST.in:

global-include *.yaml

Class Reference

class Plugin

Bases: object

Base Plugin class.

Some common features to both Sources and Elements are found in this class.

Note

Derivation of plugins is not supported. Plugins may only derive from the base Source and Element types, and any convenience subclasses (like BuildElement) which are included in the buildstream namespace.

BST_MIN_VERSION = None

The minimum required version of BuildStream required by this plugin.

The version must be expressed as the string “<major>.<minor>”, where the major version number is the API version and the minor version number is the revision of the same BuildStream API where new symbols might have been added to the API.

Example:

The following statement means that this plugin works with BuildStream 2.X, only if X >= 2:

class Foo(Source):

    # Our plugin requires 2.2
    BST_MIN_VERSION = "2.2"

Note

This version works exactly the same was as the min-version which must be specified in the project.conf file.

BST_PLUGIN_DEPRECATED = False

True if this element plugin has been deprecated.

If this is set to true, BuildStream will emit a deprecation warning in any place where this plugin is used.

The deprecation warnings can be suppressed when defining the plugin origins in your project configuration

BST_PLUGIN_DEPRECATION_MESSAGE = None

An additional message to report when a plugin is deprecated

This can be used to refer the user to a suitable replacement or alternative approach when the plugin is deprecated.

name

The plugin name

For elements, this is the project relative bst filename, for sources this is the owning element’s name with a suffix indicating its index on the owning element.

For sources this is for display purposes only.

configure(node: buildstream.node.MappingNode) → None

Configure the Plugin from loaded configuration data

Parameters

node – The loaded configuration dictionary

Raises

Plugin implementors should implement this method to read configuration data and store it.

The MappingNode.validate_keys() method should be used to ensure that the user has not specified keys in node which are unsupported by the plugin.

preflight() → None

Preflight Check

Raises

This method is run after Plugin.configure() and after the pipeline is fully constructed.

Implementors should simply raise SourceError or ElementError with an informative message in the case that the host environment is unsuitable for operation.

Plugins which require host tools (only sources usually) should obtain them with utils.get_host_tool() which will raise an error automatically informing the user that a host tool is needed.

get_unique_key() → Union[None, int, List[Any], Dict[str, Any]]

Return something which uniquely identifies the plugin input

Returns

A string, list or dictionary which uniquely identifies the input

This is used to construct unique cache keys for elements and sources, sources should return something which uniquely identifies the payload, such as an sha256 sum of a tarball content.

Elements and Sources should implement this by collecting any configurations which could possibly affect the output and return a dictionary of these settings.

For Sources, this is guaranteed to only be called if Source.is_resolved() has returned True which is to say that the Source is expected to have an exact ref indicating exactly what source is going to be staged.

get_kind() → str

Fetches the kind of this plugin

Returns

The kind of this plugin

node_get_project_path(node, *, check_is_file=False, check_is_dir=False)

Fetches a project path from a dictionary node and validates it

Paths are asserted to never lead to a directory outside of the project directory. In addition, paths can not point to symbolic links, fifos, sockets and block/character devices.

The check_is_file and check_is_dir parameters can be used to perform additional validations on the path. Note that an exception will always be raised if both parameters are set to True.

Parameters

  • node (ScalarNode) – A Node loaded from YAML containing the path to validate

  • check_is_file (bool) – If True an error will also be raised if path does not point to a regular file. Defaults to False

  • check_is_dir (bool) – If True an error will also be raised if path does not point to a directory. Defaults to False

Returns

The project path

Return type

(str)

Raises

LoadError – In the case that the project path is not valid or does not exist

Example:

path = self.node_get_project_path(node, 'path')
debug(brief: str, *, detail: Optional[str] = None) → None

Print a debugging message

Parameters

  • brief – The brief message

  • detail – An optional detailed message, can be multiline output

status(brief: str, *, detail: Optional[str] = None) → None

Print a status message

Parameters

  • brief – The brief message

  • detail – An optional detailed message, can be multiline output

Note: Status messages tell about what a plugin is currently doing

info(brief: str, *, detail: Optional[str] = None) → None

Print an informative message

Parameters

  • brief – The brief message

  • detail – An optional detailed message, can be multiline output

Note: Informative messages tell the user something they might want

to know, like if refreshing an element caused it to change. The instance full name of the plugin will be generated with the message, this being the name of the given element, as appose to the class name of the underlying plugin __kind identifier.

warn(brief: str, *, detail: Optional[str] = None, warning_token: Optional[str] = None) → None

Print a warning message, checks warning_token against project configuration

Parameters

  • brief – The brief message

  • detail – An optional detailed message, can be multiline output

  • warning_token – An optional configurable warning assosciated with this warning, this will cause PluginError to be raised if this warning is configured as fatal.

:raises (PluginError): When warning_token is considered fatal by the project configuration

log(brief: str, *, detail: Optional[str] = None) → None

Log a message into the plugin’s log file

The message will not be shown in the master log at all (so it will not be displayed to the user on the console).

Parameters

  • brief – The brief message

  • detail – An optional detailed message, can be multiline output

timed_activity(activity_name: str, *, detail: Optional[str] = None, silent_nested: bool = False) → Generator[None, None, None]

Context manager for performing timed activities in plugins

Parameters

  • activity_name – The name of the activity

  • detail – An optional detailed message, can be multiline output

  • silent_nested – If specified, nested messages will be silenced

This function lets you perform timed tasks in your plugin, the core will take care of timing the duration of your task and printing start / fail / success messages.

Example

# Activity will be logged and timed
with self.timed_activity("Mirroring {}".format(self.url)):

    # This will raise SourceError on its own
    self.call(... command which takes time ...)
call(*popenargs, fail: Optional[str] = None, fail_temporarily: bool = False, **kwargs) → int

A wrapper for subprocess.call()

Parameters

  • popenargs (list) – Popen() arguments

  • fail – A message to display if the process returns a non zero exit code

  • fail_temporarily – Whether any exceptions should be raised as temporary.

  • rest_of_args (kwargs) – Remaining arguments to subprocess.call()

Returns

The process exit code.

:raises (PluginError): If a non-zero return code is received and fail is specified

Note: If fail is not specified, then the return value of subprocess.call()

is returned even on error, and no exception is automatically raised.

Example

# Call some host tool
self.tool = utils.get_host_tool('toolname')
self.call(
    [self.tool, '--download-ponies', self.mirror_directory],
    "Failed to download ponies from {}".format(
        self.mirror_directory))
check_output(*popenargs, fail=None, fail_temporarily=False, **kwargs) → Tuple[int, str]

A wrapper for subprocess.check_output()

Parameters

  • popenargs (list) – Popen() arguments

  • fail (str) – A message to display if the process returns a non zero exit code

  • fail_temporarily (bool) – Whether any exceptions should be raised as temporary.

  • rest_of_args (kwargs) – Remaining arguments to subprocess.call()

Returns

A 2-tuple of form (process exit code, process standard output)

:raises (PluginError): If a non-zero return code is received and fail is specified

Note: If fail is not specified, then the return value of subprocess.check_output()

is returned even on error, and no exception is automatically raised.

Example

# Get the tool at preflight time
self.tool = utils.get_host_tool('toolname')

# Call the tool, automatically raise an error
_, output = self.check_output(
    [self.tool, '--print-ponies'],
    "Failed to print the ponies in {}".format(
        self.mirror_directory),
    cwd=self.mirror_directory)

# Call the tool, inspect exit code
exit_code, output = self.check_output(
    [self.tool, 'get-ref', tracking],
    cwd=self.mirror_directory)

if exit_code == 128:
    return
elif exit_code != 0:
    fmt = "{plugin}: Failed to get ref for tracking: {track}"
    raise SourceError(
        fmt.format(plugin=self, track=tracking)) from e