Getting started with COS Lite on MicroK8s¶
In this tutorial you deploy a single-node COS Lite appliance, backed by hostPath storage.
Prerequisites¶
This tutorial assumes you have:
A Juju controller bootstrapped on a MicroK8s cloud, running on a node with at least 4 CPU, 8 GB RAM, and 40 GB of disk space.
MicroK8s must be installed from a strict channel (not
--classic): Juju 3.x is itself a strictly confined snap and cannot bootstrap a classic MicroK8s installation. See the MicroK8s strict install guide for channel selection, and the Juju MicroK8s cloud reference for how Juju integrates with it. Your user must also be a member of thesnap_microk8sgroup.
Add your user to the snap_microk8s group
Strict MicroK8s uses the snap_microk8s group rather than the microk8s
group used by the classic snap. To add your current user and pick up the new
membership in the current shell:
sudo usermod -a -G snap_microk8s $USER
newgrp snap_microk8s
jqinstalled on the machine running the Juju client. It is used for JSON parsing in themetallbstep below and is not preinstalled on a fresh Ubuntu image. Install it with:
sudo apt-get update && sudo apt-get install -y jq
Introduction¶
The COS Lite bundle is a Juju-based observability stack, running on Kubernetes. The bundle consists of Prometheus, Loki, Alertmanager, and Grafana.
Let’s go and deploy that bundle!
Configure MicroK8s¶
For the COS Lite bundle deployment to go smoothly, make sure the following MicroK8s add-ons are enabled: dns, hostpath-storage and metallb.
You can check this with microk8s status, and if any are missing, enable them with
microk8s enable dns
Note
While the following setup is sufficient for non-production environments, if you’re looking for a more resilient storage option, consider deploying MicroCeph on MicroK8s using this guide.
microk8s enable hostpath-storage
The bundle comes with Traefik to provide ingress, for which you’ll need a load balancer controller.
If you don’t have one already, the metallb add-on should be enabled:
IPADDR=$(ip -4 -j route get 2.2.2.2 | jq -r '.[] | .prefsrc')
microk8s enable metallb:$IPADDR-$IPADDR
To wait for all the add-ons to be rolled out, then run:
microk8s kubectl rollout status deployments/hostpath-provisioner -n kube-system -w
microk8s kubectl rollout status deployments/coredns -n kube-system -w
microk8s kubectl rollout status daemonset.apps/speaker -n metallb-system -w
Note
If you have an HTTP proxy configured, you will need to give this information to MicroK8s. See the proxy documentation for details.
Note
By default, MicroK8s will use 8.8.8.8 and 8.8.4.4 as DNS servers, which can be adjusted. See the DNS documentation for details.
Deploy the COS Lite bundle¶
It is usually a good idea to create a dedicated model for the COS Lite bundle. So let’s do just that and call the new model cos:
juju add-model cos
juju switch cos
Next, deploy the bundle with:
juju deploy cos-lite --trust
Now you can sit back and watch the deployment take place:
juju status --relations --watch=5s
The status of your deployment should eventually be very similar to the following:
juju status --relations
Model Controller Cloud/Region Version SLA Timestamp
cos microk8s microk8s/localhost 3.6.4 unsupported 15:48:47+04:00
App Version Status Scale Charm Channel Rev Address Exposed Message
alertmanager 0.27.0 active 1 alertmanager-k8s latest/edge 158 10.152.183.34 no
catalogue active 1 catalogue-k8s latest/edge 83 10.152.183.128 no
grafana 9.5.3 active 1 grafana-k8s latest/edge 146 10.152.183.110 no
loki 2.9.6 active 1 loki-k8s latest/edge 193 10.152.183.108 no
prometheus 2.52.0 active 1 prometheus-k8s latest/edge 240 10.152.183.55 no
traefik 2.11.0 active 1 traefik-k8s latest/edge 236 10.152.183.211 no Serving at 10.211.88.149
Unit Workload Agent Address Ports Message
alertmanager/0* active idle 10.1.157.91
catalogue/0* active idle 10.1.157.81
grafana/0* active idle 10.1.157.93
loki/0* active idle 10.1.157.92
prometheus/0* active idle 10.1.157.94
traefik/0* active idle 10.1.157.90 Serving at 10.211.88.149
Integration provider Requirer Interface Type Message
alertmanager:alerting loki:alertmanager alertmanager_dispatch regular
alertmanager:alerting prometheus:alertmanager alertmanager_dispatch regular
alertmanager:grafana-dashboard grafana:grafana-dashboard grafana_dashboard regular
alertmanager:grafana-source grafana:grafana-source grafana_datasource regular
alertmanager:replicas alertmanager:replicas alertmanager_replica peer
alertmanager:self-metrics-endpoint prometheus:metrics-endpoint prometheus_scrape regular
catalogue:catalogue alertmanager:catalogue catalogue regular
catalogue:catalogue grafana:catalogue catalogue regular
catalogue:catalogue prometheus:catalogue catalogue regular
catalogue:replicas catalogue:replicas catalogue_replica peer
grafana:grafana grafana:grafana grafana_peers peer
grafana:metrics-endpoint prometheus:metrics-endpoint prometheus_scrape regular
grafana:replicas grafana:replicas grafana_replicas peer
loki:grafana-dashboard grafana:grafana-dashboard grafana_dashboard regular
loki:grafana-source grafana:grafana-source grafana_datasource regular
loki:metrics-endpoint prometheus:metrics-endpoint prometheus_scrape regular
loki:replicas loki:replicas loki_replica peer
prometheus:grafana-dashboard grafana:grafana-dashboard grafana_dashboard regular
prometheus:grafana-source grafana:grafana-source grafana_datasource regular
prometheus:prometheus-peers prometheus:prometheus-peers prometheus_peers peer
traefik:ingress alertmanager:ingress ingress regular
traefik:ingress catalogue:ingress ingress regular
traefik:ingress-per-unit loki:ingress ingress_per_unit regular
traefik:ingress-per-unit prometheus:ingress ingress_per_unit regular
traefik:metrics-endpoint prometheus:metrics-endpoint prometheus_scrape regular
traefik:peers traefik:peers traefik_peers peer
traefik:traefik-route grafana:ingress traefik_route regular
Now COS Lite is good to go: you can relate software with it to begin the monitoring!
Alternatively, you may want to deploy the bundle with one or more of our readily available overlays, which is what we’ll cover next.
Deploy the COS Lite bundle with overlays¶
An overlay is a set of model-specific modifications that avoid repetitive overhead in setting up bundles like COS Lite.
Specifically, we offer the following overlays:
the offers overlay makes your COS model ready for cross-model relations
the storage-small overlay applies some defaults for the various storage used by the COS Lite components.
Note
You can apply the offers overlay to an existing COS Lite bundle by executing the juju deploy command.
The storage-small overlay, however, is applicable only on the first deployment.
So, if you were following the previous steps you would first need to switch to a new Juju model or remove all applications from the current one.
To use any of the overlays above, you need to include an --overlay argument per overlay (applied in order):
curl -L https://raw.githubusercontent.com/canonical/cos-lite-bundle/main/overlays/offers-overlay.yaml -O
curl -L https://raw.githubusercontent.com/canonical/cos-lite-bundle/main/overlays/storage-small-overlay.yaml -O
juju deploy cos-lite \
--trust \
--overlay ./offers-overlay.yaml \
--overlay ./storage-small-overlay.yaml
Deploy COS Lite using Terraform¶
To deploy the COS Lite solution, create this root module and set the model to cos-lite:
terraform {
required_version = ">= 1.5"
required_providers {
juju = {
source = "juju/juju"
version = ">= 1.0"
}
}
}
variable "model" {
type = string
}
data "juju_model" "model" {
name = var.model
owner = "admin"
}
module "cos-lite" {
source = "git::https://github.com/canonical/observability-stack//terraform/cos-lite?ref=track/3.0"
model = { uuid = data.juju_model.model.uuid }
risk = "stable"
internal_tls = true
}
Customize COS Lite
You can customize the deployment of COS Lite by overriding the defaults of these variables:
# We use constraints to set AntiAffinity in K8s
# https://discourse.charmhub.io/t/pod-priority-and-affinity-in-juju-charms/4091/13?u=jose
# FIXME: Passing an empty constraints value to the Juju Terraform provider currently
# causes the operation to fail due to https://github.com/juju/terraform-provider-juju/issues/344
# Therefore, we set a default value of "arch=amd64" for all applications.
variable "risk" {
description = "Risk level that the applications are (unless overwritten by individual channels) deployed from"
type = string
default = "stable"
}
variable "model" {
description = "Model configuration. When `uuid` is set, an existing model is looked up; otherwise a new model is created with the given fields. For more details: https://registry.terraform.io/providers/juju/juju/latest/docs/resources/model"
type = object({
uuid = optional(string)
name = optional(string, "cos-lite")
cloud = optional(object({
name = string
region = optional(string)
}))
annotations = optional(map(string))
config = optional(map(string))
constraints = optional(string)
credential = optional(string)
target_controller = optional(string)
})
default = {}
validation {
condition = var.model.uuid == null || (
var.model.annotations == null &&
var.model.cloud == null &&
var.model.config == null &&
var.model.constraints == null &&
var.model.credential == null &&
var.model.target_controller == null
)
error_message = "When `model.uuid` is set, the model already exists; do not also set `annotations`, `cloud`, `config`, `constraints`, `credential`, or `target_controller`."
}
validation {
condition = var.model.uuid != null || (var.model.name != null && length(var.model.name) > 0)
error_message = "`model.name` must be non-empty when creating a model (i.e. when `model.uuid` is null)."
}
validation {
condition = var.model.uuid == null || can(regex("^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$", var.model.uuid))
error_message = "`model.uuid` must be a valid UUID."
}
}
# -------------- # Network configurations --------------
variable "internal_tls" {
description = "Specify whether to use TLS or not for internal COS communication. By default, TLS is enabled using self-signed-certificates"
type = bool
default = true
}
variable "external_certificates_offer_url" {
description = "A Juju offer URL (e.g. admin/external-ca.certificates) of a CA providing the 'tls_certificates' integration for Traefik to supply it with server certificates."
type = string
default = null
validation {
condition = (
(var.external_certificates_offer_url == null && var.external_ca_cert_offer_url == null) ||
(var.external_certificates_offer_url != null && var.external_ca_cert_offer_url != null)
)
error_message = "external_certificates_offer_url and external_ca_cert_offer_url must be supplied together (either both set or both null)."
}
}
variable "external_ca_cert_offer_url" {
description = "A Juju offer URL (e.g. admin/external-ca.send-ca-cert) of a CA providing the 'certificate_transfer' integration for applications to trust ingress via Traefik."
type = string
default = null
}
variable "postgresql_offer_url" {
description = "A Juju offer URL (e.g. admin/postgresql.database) of a PostgreSQL service providing the 'postgresql_client' integration for applications to connect to the database."
type = string
default = null
validation {
condition = !(var.postgresql_offer_url == null && var.grafana.units > 1)
error_message = "postgresql_offer_url must be supplied when Grafana is scaled > 1 due to its database requirements."
}
}
# -------------- # Ingress configurations --------------
variable "ingress" {
description = "Per-component toggle for ingress integrations"
type = object({
alertmanager = optional(bool, true)
catalogue = optional(bool, true)
grafana = optional(bool, true)
loki = optional(bool, true)
prometheus = optional(bool, true)
})
default = {}
}
# -------------- # Application configurations --------------
variable "alertmanager" {
type = object({
app_name = optional(string, "alertmanager")
config = optional(map(string), {})
constraints = optional(string, "arch=amd64")
resources = optional(map(string), {})
revision = optional(number, null)
storage_directives = optional(map(string), {})
units = optional(number, 1)
})
default = {}
description = "Application configuration for Alertmanager. For more details: https://registry.terraform.io/providers/juju/juju/latest/docs/resources/application"
}
variable "catalogue" {
type = object({
app_name = optional(string, "catalogue")
config = optional(map(string), {})
constraints = optional(string, "arch=amd64")
resources = optional(map(string), {})
revision = optional(number, null)
storage_directives = optional(map(string), {})
units = optional(number, 1)
})
default = {}
description = "Application configuration for Catalogue. For more details: https://registry.terraform.io/providers/juju/juju/latest/docs/resources/application"
}
variable "grafana" {
type = object({
app_name = optional(string, "grafana")
config = optional(map(string), {})
constraints = optional(string, "arch=amd64")
resources = optional(map(string), {})
revision = optional(number, null)
storage_directives = optional(map(string), {})
units = optional(number, 1)
})
default = {}
description = "Application configuration for Grafana. For more details: https://registry.terraform.io/providers/juju/juju/latest/docs/resources/application"
}
variable "loki" {
type = object({
app_name = optional(string, "loki")
config = optional(map(string), {})
constraints = optional(string, "arch=amd64")
resources = optional(map(string), {})
revision = optional(number, null)
storage_directives = optional(map(string), {})
units = optional(number, 1)
})
default = {}
description = "Application configuration for Loki. For more details: https://registry.terraform.io/providers/juju/juju/latest/docs/resources/application"
}
variable "prometheus" {
type = object({
app_name = optional(string, "prometheus")
config = optional(map(string), {})
constraints = optional(string, "arch=amd64")
resources = optional(map(string), {})
revision = optional(number, null)
storage_directives = optional(map(string), {})
units = optional(number, 1)
})
default = {}
description = "Application configuration for Prometheus. For more details: https://registry.terraform.io/providers/juju/juju/latest/docs/resources/application"
}
variable "ssc" {
type = object({
app_name = optional(string, "ca")
config = optional(map(string), {})
constraints = optional(string, "arch=amd64")
revision = optional(number, null)
storage_directives = optional(map(string), {})
units = optional(number, 1)
})
default = {}
description = "Application configuration for self-signed-certificates. For more details: https://registry.terraform.io/providers/juju/juju/latest/docs/resources/application"
}
variable "traefik" {
type = object({
app_name = optional(string, "traefik")
config = optional(map(string), {})
constraints = optional(string, "arch=amd64")
resources = optional(map(string), {})
revision = optional(number, null)
storage_directives = optional(map(string), {})
units = optional(number, 1)
})
default = {}
description = "Application configuration for Traefik. For more details: https://registry.terraform.io/providers/juju/juju/latest/docs/resources/application"
}
It is usually a good idea to create a dedicated model for COS Lite. This module creates one named cos-lite.
If there are multiple clouds registered in the controller or a default cloud is not setup, an explicit reference of the cloud in the base terraform file needs to exist. Look at how to reference a K8s cloud section.
By default, COS uses local files on k8s node to store data. To use a custom storage class for container’s persistent volume during deployment, refer to guide: how to configure custom storage options
Next, deploy COS Lite in the new model, run:
terraform init
terraform apply -var="model=cos-lite"
Now you can sit back and watch the deployment take place:
juju switch cos
juju status --relations --watch=5s
Browse dashboards¶
When all the charms are deployed you can head over to browse their built-in web interfaces. You can find out their addresses from the show-proxied-endpoints Traefik action.
For example:
juju run traefik/0 show-proxied-endpoints --format=yaml \
| yq '."traefik/0".results."proxied-endpoints"' \
| jq
…should return output similar to:
{
"prometheus/0": {
"url": "http://10.43.8.34:80/cos-prometheus-0"
},
"loki/0": {
"url": "http://10.43.8.34:80/cos-loki-0"
},
"catalogue": {
"url": "http://10.43.8.34:80/cos-catalogue"
},
"alertmanager": {
"url": "http://10.43.8.34:80/cos-alertmanager"
}
}
In the output above,
10.43.8.34is Traefik’s IP address.Applications that are ingresses “per app”, such as Alertmanager, are accessible via the
model-apppath (i.e.http://10.43.8.34:80/cos-alertmanager).Applications that are ingresses “per unit”, such as Loki, are accessible via the
model-app-unitpath (i.e.http://10.43.8.34:80/cos-loki-0).
Note that Grafana does not appear in the list. Currently, to obtain Grafana’s proxied endpoint you would need to look at catalogue’s relation data directly - try running:
juju show-unit catalogue/0 | grep url
…which should return a list of the endpoints like this:
url: http://10.43.8.34:80/cos-catalogue
url: http://10.43.8.34/cos-grafana
url: http://10.43.8.34:80/cos-prometheus-0
url: http://10.43.8.34:80/cos-alertmanager
With ingress in place, you can still access the workloads via pod IPs, but you will need to include the original port, as well as the ingress path. For example:
curl 10.1.55.34:9093/cos-alertmanager/-/ready
The default password for Grafana is automatically generated for every installation. To
access Grafana’s web interface, use the username admin, and the password obtained
from the get-admin-password action, e.g:
juju run grafana/leader get-admin-password --model cos
Enjoy!
Next steps¶
Use the scrape target charm to have the COS stack scrape any Prometheus compatible target.
Relate your own charm to the COS stack with relation interfaces such as prometheus_scrape.
Configure alertmanager to have alerts routed to your receivers.
Use the cos-proxy machine charm to observe LMA-enabled machine charms.
Use the grafana-agent machine charm to observe charms on other substrates than Kubernetes.
If you need support, the Charmhub community is the best place to get all your questions answered and get in touch with the community.