Debug Build Farm on qastaging
=============================

Access a builder
----------------

The builders are ephemeral, meaning they fully reset after a build.
Therefore, to access the builder, one should either:

* access it during the duration of a build.

* ``disable`` the builder after starting a build so that it doesn't
  automatically trigger the reset when the build finishes.

.. note::

   You should wait until you see some output in the build log before you
   ``disable`` the builder, otherwise the build might not get dispatched
   properly.


Disable a builder
~~~~~~~~~~~~~~~~~

You need to have the ubuntu-archive-tools installed.

.. code-block:: sh

    $ git clone https://git.launchpad.net/ubuntu-archive-tools
    $ sudo apt install python3-launchpadlib python3-ubuntutools


For example, you want to debug the ``lcy02-amd64-004`` instance.

.. code-block:: sh

    ❯ ./manage-builders -l qastaging -b qastaging-lcy02-amd64-004 --disable
    Updating 1 builders.
    * qastaging-lcy02-amd64-004
    Changed: 1. Unchanged: 0.


You can look up the builder name on https://qastaging.launchpad.net/builders or
via ``manage-builders -l qastaging``.


SSH into the builder
~~~~~~~~~~~~~~~~~~~~

Once you are ready to access a builder, follow these steps:

1. SSH into bastion ``ssh launchpad-bastion-ps5`` and switch to the
   ``stb-vbuilder-qastaging`` user by running ``sudo -iu stg-vbuilder-qastaging``.

2. Juju ssh into the ``vbuilder-manage`` for the region your builder is, for
   example ``juju ssh vbuilder-manage-bos03/leader``.

3. Switch to the ``ppa`` user by running ``sudo -iu ppa``.

4. SSH into builder instance. For example, to access the ``bos03-amd64-004``
   builder, you should run ``ssh
   ubuntu@qastaging-bos03-amd64-004.vbuilder.qastaging.bos03.scalingstack``.
   This builder address can be found when click on the builder on
   https://qastaging.launchpad.net/builders.

From there, you can check the ``launchpad-buildd`` logs in
``/var/log/launchpad-buildd/`` or run commands within the builder.

The build itself should be done within a container within the builder. If you
need to look into the container, you can ``lxc list`` to list the lxc
containers present in the builder, and then run ``lxc shell`` to start a shell
session from within the container.


Clean-up
~~~~~~~~

Once you have finished, please remember to enable the builder again.

.. code-block:: sh

    ❯ ./manage-builders -l qastaging -b qastaging-lcy02-amd64-004 --enable
    Updating 1 builders.
    * qastaging-lcy02-amd64-004
    Changed: 1. Unchanged: 0.