When you have a clear understanding of what metrics to gather for SLO, the next question is how to obtain and gather those metrics. Basically the metrics can be obtained by the following methods.
Metrics from commercial APM/ Observability tools,
Commercial monitoring tools offer one stop shop to gather infrastructure metrics from machine agents, cluster agents or in terms of public cloud platforms, those will be metrics from AWS CloudWatch or Azure Monitor. For application metrics, usually the same APM agent can generate application specific metrics derived from logs or traces. Usually these contribute to the 4 golden signals we discussed in the previous post.
You can now build dashboards directly from the commercial APM tools to utilize those metrics. Metrics can also be retrieved through APIs for external consumption.
Metrics from siloed data sources
They are usually from systems in your organization that have no full featured APM tools in place. Today it is common for applications to expose Prometheus compatible metrics ( a de facto standard to allow metrics to be scrapped for monitoring by Prometheus ) to fulfill simple monitoring use cases. Grafana then fills the gap on time series visualization and alerting.
Metrics from E2E testing and synthetic tools
Synthetic testing tools can provide valuable insight on service availability, response time and customer experience. These metrics can also be a source of SLO. Commercial solutions include ThousandEyes and SolarWinds PingDom.
If you have multiple monitoring tools it is wise to consolidate all these metrics into a single dashboard for alerting and visualization. Examples are Grafana and Nobl9 which can help to
Consolidate metrics from multiple tools into a single dashboard.
Offers pre-built SLO dashboards or flexibility to easily build SLO dashboards
As a summary, the goal is to streamline the process of obtaining the metrics from multiple systems and quickly realize the benefits of SLO tracking. In the next article, we will look at a real example of utilizing Nobl9 for a simple service availability SLO.
New to SLO?
#SLOconf is a free, virtual event focused on #SLOs! 🔥
Whether you are doing SRE, SLO, or DevOps, or Ops, or a Dev – SLOconf is the perfect platform to share insights and ideas on the latest trends and developments in SRE/SLO.
Vsceptre is a sponsor at SLOconf 2023, hosted by Nobl9! 📢
For more details & speaker lineup, register here: 👇
www.sloconf.com
