1. Workspaces¶
In this section we will cover the use of BuildStream’s workspaces feature when devloping a BuildStream project.
Note
This example is distributed with BuildStream in the doc/examples/developing subdirectory.
We will start with the project used in the running commands tutorial. Recall the element hello.bst, which builds the bellow C file:
/*
* hello.c - Simple hello world program
*/
#include <stdio.h>
int main(int argc, char *argv[])
{
printf("Hello World\n");
return 0;
}
Suppose we now want to alter the functionality of the hello command. We can make changes to the source code of Buildstream elements by making use of BuildStream’s workspace command.
1.1. Opening a workspace¶
First we need to open a workspace, we can do this by running
user@host:~/developing$ bst workspace open --directory workspace_hello hello.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:core activity ] START Checking sources
[00:00:00][ ][ main:core activity ] SUCCESS Checking sources
[--:--:--][ ][ main:core activity ] INFO Creating workspace for element hello.bst
[--:--:--][ ][ main:hello.bst ] START Staging sources to /home/user/developing/workspace_hello
[--:--:--][ ][ main:hello.bst ] START Staging local files into CAS
[00:00:00][ ][ main:hello.bst ] SUCCESS Staging local files into CAS
[00:00:00][ ][ main:hello.bst ] SUCCESS Staging sources to /home/user/developing/workspace_hello
[--:--:--][ ][ main:core activity ] INFO Created a workspace for element: hello.bst
This command has created the workspace_hello directory in which you can see the source for the hello.bst element, i.e. hello.c and the corresponding makefile.
You can view existing workspaces using
user@host:~/developing$ bst workspace list
workspaces:
- element: hello.bst
directory: /home/user/developing/workspace_hello
1.2. Making code changes¶
Let’s say we want to alter the message printed when the hello command is run. We can open workspace_hello/hello.c and make the following change:
-- hello.c 2018-06-25 14:48:32.077568920 +0100
+++ hello.c 2018-06-25 14:49:23.025553785 +0100
@@ -5,6 +5,6 @@
int main(int argc, char *argv[])
{
- printf("Hello World\n");
+ printf("Hello World\nWe can use workspaces!\n");
return 0;
}
Now, rebuild the hello.bst element.
user@host:~/developing$ bst build hello.bst
[--:--:--][ ][ main:core activity ] START Build
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:core activity ] START Checking sources
[00:00:00][ ][ main:core activity ] SUCCESS Checking sources
BuildStream Version 1.93.5
Session Start: Wednesday, 19-08-2020 at 14:53:09
Project: developing (/home/user/developing)
Targets: hello.bst
User Configuration
Configuration File: /home/user/.config/buildstream.conf
Cache Directory: /home/user/.cache/buildstream
Log Files: /home/user/.cache/buildstream/logs
Source Mirrors: /home/user/.cache/buildstream/sources
Build Area: /home/user/.cache/buildstream/build
Strict Build Plan: Yes
Maximum Fetch Tasks: 10
Maximum Build Tasks: 4
Maximum Push Tasks: 4
Maximum Network Retries: 2
Project: developing
Element Plugins
manual: core plugin
stack: core plugin
import: core plugin
Source Plugins
workspace: core plugin
tar: core plugin
Pipeline
buildable 179c6ae937e8e2ece3192ab8dc2a55053134a591c9ccd37507b56d11885fae23 base/alpine.bst
waiting 125099c65de3a18455f11c5a4a7379df93d5638460906c7f183cfae1b21a651b base.bst
waiting c77a106972d75d5efcac7b9c2312ef817a15c8691a69ea01ce714b80ce94d36e hello.bst Workspace: /home/user/developing/workspace_hello
===============================================================================
[--:--:--][179c6ae9][ build:base/alpine.bst ] START developing/base-alpine/179c6ae9-build.826.log
[--:--:--][c77a1069][ fetch:hello.bst ] START developing/hello/c77a1069-fetch.827.log
[--:--:--][179c6ae9][ build:base/alpine.bst ] START Staging sources
[00:00:00][c77a1069][ fetch:hello.bst ] SUCCESS developing/hello/c77a1069-fetch.827.log
[00:00:00][179c6ae9][ build:base/alpine.bst ] SUCCESS Staging sources
[--:--:--][179c6ae9][ build:base/alpine.bst ] START Caching artifact
[00:00:00][179c6ae9][ build:base/alpine.bst ] SUCCESS Caching artifact
[00:00:00][179c6ae9][ build:base/alpine.bst ] SUCCESS developing/base-alpine/179c6ae9-build.826.log
[--:--:--][125099c6][ build:base.bst ] START developing/base/125099c6-build.836.log
[--:--:--][125099c6][ build:base.bst ] START Caching artifact
[00:00:00][125099c6][ build:base.bst ] SUCCESS Caching artifact
[00:00:00][125099c6][ build:base.bst ] SUCCESS developing/base/125099c6-build.836.log
[--:--:--][c77a1069][ build:hello.bst ] START developing/hello/c77a1069-build.840.log
[--:--:--][c77a1069][ build:hello.bst ] START Staging dependencies
[00:00:00][c77a1069][ build:hello.bst ] SUCCESS Staging dependencies
[--:--:--][c77a1069][ build:hello.bst ] START Staging sources
[00:00:00][c77a1069][ build:hello.bst ] SUCCESS Staging sources
[--:--:--][c77a1069][ build:hello.bst ] START Running commands
make PREFIX="/usr"
make -j1 PREFIX="/usr" DESTDIR="/buildstream-install" install
[00:00:00][c77a1069][ build:hello.bst ] SUCCESS Running commands
[--:--:--][c77a1069][ build:hello.bst ] START Caching artifact
[00:00:00][c77a1069][ build:hello.bst ] SUCCESS Caching artifact
[00:00:00][c77a1069][ build:hello.bst ] SUCCESS developing/hello/c77a1069-build.840.log
[00:00:00][ ][ main:core activity ] SUCCESS Build
Pipeline Summary
Total: 3
Session: 3
Fetch Queue: processed 1, skipped 2, failed 0
Build Queue: processed 3, skipped 0, failed 0
Note that if you run the command from inside the workspace, the element name is optional.
user@host:~/workspace_hello$ bst build
[--:--:--][ ][ main:core activity ] START Build
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:core activity ] START Checking sources
[00:00:00][ ][ main:core activity ] SUCCESS Checking sources
BuildStream Version 1.93.5
Session Start: Wednesday, 19-08-2020 at 14:53:10
Project: developing (/builds/BuildStream/buildstream/doc/examples/developing)
Targets: hello.bst
User Configuration
Configuration File: /home/user/.config/buildstream.conf
Cache Directory: /home/user/.cache/buildstream
Log Files: /home/user/.cache/buildstream/logs
Source Mirrors: /home/user/.cache/buildstream/sources
Build Area: /home/user/.cache/buildstream/build
Strict Build Plan: Yes
Maximum Fetch Tasks: 10
Maximum Build Tasks: 4
Maximum Push Tasks: 4
Maximum Network Retries: 2
Project: developing
Element Plugins
manual: core plugin
stack: core plugin
import: core plugin
Source Plugins
workspace: core plugin
tar: core plugin
Pipeline
cached 179c6ae937e8e2ece3192ab8dc2a55053134a591c9ccd37507b56d11885fae23 base/alpine.bst
cached 125099c65de3a18455f11c5a4a7379df93d5638460906c7f183cfae1b21a651b base.bst
cached c77a106972d75d5efcac7b9c2312ef817a15c8691a69ea01ce714b80ce94d36e hello.bst Workspace: /home/user/workspace_hello
===============================================================================
[00:00:00][ ][ main:core activity ] SUCCESS Build
Pipeline Summary
Total: 3
Session: 0
Fetch Queue: processed 0, skipped 0, failed 0
Build Queue: processed 0, skipped 0, failed 0
Now running the hello command using bst shell:
user@host:~/developing$ bst shell hello.bst -- hello
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:hello.bst ] START Staging dependencies
[00:00:00][ ][ main:hello.bst ] SUCCESS Staging dependencies
[--:--:--][ ][ main:hello.bst ] START Integrating sandbox
[00:00:00][ ][ main:hello.bst ] SUCCESS Integrating sandbox
[--:--:--][ ][ main:hello.bst ] STATUS Running command
hello
2020-08-19T14:53:10.699+0000 [908:140188886145088] [buildboxcommon_runner.cpp:547] [WARNING] The --use-localcas option will be deprecated. LocalCAS support is now enabled by default.
2020-08-19T14:53:10.700+0000 [908:140188886145088] [buildboxcommon_client.cpp:87] [INFO] Setting d_maxBatchTotalSizeBytes = 4128768 bytes by default
Hello World
We can use workspaces!
This gives us the new message we changed in hello.c.
From this point we have several options. If the source is under version control we can commit our changes and push them to the remote repository.
1.3. Making bigger changes¶
The first time you build with your workspace BuildStream behaves very similarly to normal. However for subsequent builds it does not run the configuration commands and only runs the build commands. This can result in significant speed ups as the configuration commands can be slow (maybe 20+ minutes for a moderate sized element)
Sometimes you do need to rerun the configuration command for a open workspace, this can be done by resetting the workspace, and then running a build. However this will cause all of your incremental work to be lost. In this case running a soft reset will reset the trigger to run the configuration commands but will not change any files in you workspace.
- Reasons to soft reset a workspace include:
changing the configuration command of a element.
adding a new source that the configure command will spot and enable more code.
The soft reset can be performed by:
user@host:~/developing$ bst workspace reset --soft hello.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:core activity ] INFO Reset workspace state for hello.bst at: /home/user/developing/workspace_hello
Then the next build will include the configuration commands. You must reset the workspace every time you wish to trigger the configuration commands as only the first build after the reset will run them.
1.4. Closing your workspace¶
If we want to close the workspace and come back to our changes later, we can
user@host:~/developing$ bst workspace close hello.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:core activity ] INFO Closed workspace for hello.bst
We can then reopen the workspace later using:
user@host:~/developing$ bst workspace open --no-checkout --directory workspace_hello hello.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:core activity ] INFO Creating workspace for element hello.bst
[--:--:--][ ][ main:core activity ] INFO Created a workspace for element: hello.bst
The –no-checkout option tells BuildStream not to check the source out but to instead hard-link to the workspace_hello directory.
Alternatively, if we wish to discard the changes we can use
user@host:~/developing$ bst workspace reset hello.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:core activity ] START Removing workspace directory /home/user/developing/workspace_hello
[00:00:00][ ][ main:core activity ] SUCCESS Removing workspace directory /home/user/developing/workspace_hello
[--:--:--][ ][ main:core activity ] INFO Closed workspace for hello.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:core activity ] START Checking sources
[00:00:00][ ][ main:core activity ] SUCCESS Checking sources
[--:--:--][ ][ main:core activity ] INFO Creating workspace for element hello.bst
[--:--:--][ ][ main:hello.bst ] START Staging sources to /home/user/developing/workspace_hello
[--:--:--][ ][ main:hello.bst ] START Staging local files into CAS
[00:00:00][ ][ main:hello.bst ] SUCCESS Staging local files into CAS
[00:00:00][ ][ main:hello.bst ] SUCCESS Staging sources to /home/user/developing/workspace_hello
[--:--:--][ ][ main:core activity ] INFO Created a workspace for element: hello.bst
This resets the workspace to its original state.
To discard the workspace completely we can do:
user@host:~/developing$ bst workspace close --remove-dir hello.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Resolving cached state
[00:00:00][ ][ main:core activity ] SUCCESS Resolving cached state
[--:--:--][ ][ main:core activity ] START Removing workspace directory /home/user/developing/workspace_hello
[00:00:00][ ][ main:core activity ] SUCCESS Removing workspace directory /home/user/developing/workspace_hello
[--:--:--][ ][ main:core activity ] INFO Closed workspace for hello.bst
This will close the workspace and completely remove the workspace_hello directory.