How to use Pebble to manage a remote system

Managing server clusters remotely takes time and effort. Servers need regular updates and configuration changes, which typically involves transferring files and running commands. Directly managing remote servers or using tools that require SSH access (such as Ansible) increases overhead and security risks. The XZ Utils backdoor incident in 2024 highlighted vulnerabilities associated with SSH access.

Pebble offers commands and an HTTP API over Unix socket for remote system management, avoiding the need for extra open ports.

Note: By “remote system”, we mean that the Pebble daemon is running in a separate system and there’s a Unix socket for clients to interact with the daemon.

Run commands in a remote system

One common task in system administration is updating and installing packages. With Pebble, we can use the pebble exec command to achieve this.

Note: Set the environment variable PEBBLE_SOCKET to override the Unix socket used for the API (defaults to $PEBBLE/.pebble.socket, or to /var/lib/pebble/default/.pebble.socket if PEBBLE is not set).

For example, if Pebble is running as a user with root privileges, we can use this command to update and install packages in a remote system:

user@host:~$ pebble exec apt update

Hit:1 http://ports.ubuntu.com/ubuntu-ports noble InReleaseGet:2 http://ports.ubuntu.com/ubuntu-ports noble-updates InRelease [126 kB]Get:3 http://ports.ubuntu.com/ubuntu-ports noble-backports InRelease [126 kB]...Fetched 3121 kB in 38s (81.5 kB/s)Reading package lists... DoneBuilding dependency tree... DoneReading state information... Done41 packages can be upgraded. Run 'apt list --upgradable' to see them.

To install a package, run:

pebble exec apt install cowsay

To confirm the package is successfully installed in the remote system, run:

user@host:~$ pebble exec cowsay moo

_____< moo > -----        \   ^__^         \  (oo)\_______            (__)\       )\/\                ||----w |                ||     ||

For more information, see exec.

Manage files in a remote system

Another common task in a remote system is configuration management. For example, updating configuration files, pushing files to servers, pulling files generated by running services, and so on.

With Pebble, this can be done using its file commands. For example, if we want to install Nginx, create our own website, and serve it on a newly created virtual host, we can first install Nginx:

pebble exec apt install nginx

Then create our website locally:

echo "Hello, Pebble!" > /tmp/index.html

Create a directory in the remote system:

pebble mkdir -p /var/www/demo

Push our local website file to the remote system:

pebble push /tmp/index.html /var/www/demo/index.html

Create a virtual host configuration locally:

cat <<EOF > /tmp/demo
server {
       listen 81;
       listen [::]:81;

       server_name example.ubuntu.com;

       root /var/www/demo;
       index index.html;

       location / {
               try_files \$uri \$uri/ =404;
       }
}
EOF

Push the file to the remote system:

pebble push /tmp/demo /etc/nginx/sites-enabled/demo

Activate the newly added virtual host in the remote system by restarting Nginx:

pebble exec service nginx restart

Finally, we can test the result:

user@host:~$ curl localhost:81

Hello, Pebble!

For more information about related Pebble commands, see:

• ls

• mkdir

• pull

• push

• rm

See more

You can also use the Pebble API to run command and manage files. This is normally done using the Go or Python client libraries. For more information, see How to use the Pebble API.