.. _handling_files_overlaps: Overlapping files ================= In this chapter, we will discuss what happens when files from multiple element artifacts conflict with eachother, and what strategies we can use to resolve these situations. .. note:: This example is distributed with BuildStream in the `doc/examples/overlaps `_ subdirectory. Overview -------- This project builds on the previous chapter on :ref:`composition `, and as such we'll only go over what has changed from there, which is not much. Project structure ----------------- In this example we've just extended the ``libhello.bst`` and the ``hello.bst`` elements such that they both install an additional file: ``%{docdir}/hello.txt``. We've updated the following Makefiles: files/libhello/Makefile ~~~~~~~~~~~~~~~~~~~~~~~ .. literalinclude:: ../../examples/overlaps/files/libhello/Makefile :language: Makefile files/hello/Makefile ~~~~~~~~~~~~~~~~~~~~ .. literalinclude:: ../../examples/overlaps/files/hello/Makefile :language: Makefile As you can see, this now presents a conflict of sorts, where multiple elements in the pipeline want to install the *same file*. Using the project ----------------- In this chapter, we're only going to present the warning and then discuss how to mitigate this situation. See what happens when we try to build the ``runtime-only.bst`` :mod:`compose ` element: .. raw:: html :file: ../sessions/overlaps-build.html Notice the warning message about the conflicting file, it is there to inform the user about which files are *overlapping*, and also which elements are being staged in which order. Note also that BuildStream does not discover the overlap until the moment that you build a reverse dependency which will require staging of both artifacts. .. tip:: The ``overlaps`` warning discussed here can be configured to be a :ref:`fatal warning `. This is useful in the case that you want to be strict about avoiding overlapping files in your project. Mitigating overlapping files ---------------------------- Since we recently discussed :ref:`filtering of artifacts `, we should note that it is of course possible to handle this case by having ``hello.bst`` depend on a *filtered* version of ``libhello.bst`` with the offending file excluded. However, working with :mod:`filter ` elements just for the sake of handling a conflicting artifact would be quite inconvenient, so we have other means. Whitelisting overlapping files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ BuildStream allows explicitly ignoring such errors by adding the files to an :ref:`overlap whitelist `, you could achieve this in the given example by adding the following to the ``hello.bst`` element: .. code:: yaml public: bst: overlap-whitelist: - | %{docdir}/hello.txt .. note:: Note that :func:`glob patterns ` are also supported in the whitelist. Artifact munging ~~~~~~~~~~~~~~~~ Another way around this situation is the *munge* the artifacts at install time such that there is no conflict. This is the easiest approach in the case that you might want to keep the underlying ``%{docdir}/hello.txt`` from ``libhello.bst`` and discard the same file from ``hello.bst``. In this case, we might modify the ``hello.bst`` file so that it's install command contain an ``rm`` statement, as such: .. code:: yaml install-commands: - make -j1 PREFIX="%{prefix}" DESTDIR="%{install-root}" install - | # Rid ourselves of the unwanted file at install time rm -f %{install-root}%{docdir}/hello.txt This would cause later builds of ``runtime-only.bst`` to no longer conflict on the given file. Summary ------- In this chapter we've presented a situation where an artifact can *conflict* with another artifact by way of providing the same files. We've presented the :ref:`overlap whitelist ` public data which is the typical solution for silencing the error when the outcome is desired, and also presented a strategy to deal with cases where you want to keep files from the *overlapped* artifact instead.