SRE and SLO Design
Reliability targets that change decisions. Objectives set against what users actually experience, with a written agreement about what happens when the budget is spent.
- Who
- Founder holds the engineering leadership seat; the delivery team executes underneath it.
What you're seeing
- An enterprise contract names an availability figure and nobody can measure it.
- Usually means A number was committed commercially without a corresponding indicator. Defining what is being measured, and where, is now urgent and should have been the easy part.
- There is an availability dashboard and it has never changed a decision.
- Usually means A target with no policy attached is decoration. Reliability and feature work are still being traded implicitly, by whoever is most persuasive that week.
- Server-side availability looks excellent and customers are complaining.
- Usually means The measurement is taken where the server is rather than where the user is. The load balancer, the CDN and the client are all outside it and the user experiences all three.
- Every incident produces a debate about whether reliability work should take priority.
- Usually means The trade has no agreed rule, so it is renegotiated from scratch each time — under pressure, with the outcome decided by seniority rather than by a policy anyone agreed to in advance.
A target that changes nothing is decoration
The useful part of a service level objective is not the number. It is the sentence attached to it.
That sentence says what happens when the error budget for the period is spent: feature work pauses, reliability work takes priority, and it resumes when the budget recovers. Without it, the target is a chart, and the trade between shipping and stability continues to be made implicitly by whoever is most persuasive in a given week.
The sentence has to be agreed in advance, and it has to be agreed with the people who own the roadmap rather than inside engineering. It cannot be negotiated credibly during the incident that spends the budget — at that moment everyone has an interest, the pressure runs one way, and whatever gets decided is decided by rank.
That conversation is the substance of this work. The arithmetic of the budget is trivial by comparison.
Measure where the user is
Server-side availability misses the load balancer, the CDN, DNS resolution and everything happening in the client.
Users experience all of them. A system reporting excellent availability while customers complain is not a paradox and not a reporting artefact — it is measuring the wrong location, and the honest number is the worse one.
That makes indicators harder to instrument, and it makes the resulting figures less flattering. Both are the point. An indicator a customer would not recognise as describing their experience is not an indicator of service quality, whatever it is an indicator of.
The target comes from the commitment
Three nines and four nines differ by roughly an order of magnitude in engineering cost and are largely indistinguishable to a user.
So the number is derived rather than chosen: what has been promised commercially, what the architecture can actually hold, and what the business is prepared to spend to close the gap between them. A target selected because it sounds appropriately serious commits the organisation to spending that buys nothing, and it is very difficult to revise downward once it has been said out loud.
Where it sits
This capability sits under Platform Engineering, where it is the mechanism that makes reliability a stated trade rather than an implicit one.
It depends on two neighbours. Observability Stack has to exist first — an objective for something you are not measuring at the right point is not a target, it is a hope. And On-Call & Incident Response is where the budget is actually spent, which is why the severity definitions and the objectives should be designed in the same conversation rather than separately.
How the work runs
-
Pick indicators users would recognise
Availability and latency of the journeys that matter, measured where the user is rather than where the server is.
-
Set targets against the commitment
Objectives derived from what has been promised commercially and what the architecture can hold. A number chosen for how it looks is not a target.
-
Derive the error budget
The permitted unreliability, made explicit — because a budget is what turns a target into a decision-making tool.
-
Write the policy in advance
What changes when the budget is exhausted, agreed with whoever owns the roadmap before it happens. Negotiated during an incident, it means nothing.
What arrives
- Service level indicators measured from the user's position
- Objectives with the reasoning and the commercial commitment behind them
- Error budget tracking on a dashboard product can read
- A signed policy for budget exhaustion
What it costs your team
Two workshops with engineering and product leadership, then around two hours a month to review.
How we decide
Indicators are measured where the user is
Costs It is harder to instrument and the resulting numbers are worse than server-side ones.
Server-side availability excludes the load balancer, the CDN, DNS and the client, and users experience all of them. A figure that looks excellent while customers complain is not a measurement problem to be explained away — it is a measurement of the wrong thing, and the worse number is the true one.
The target comes from the commitment, not from how it looks
Costs It occasionally produces a target lower than the organisation would like to publish.
Three nines and four nines are separated by roughly an order of magnitude in engineering cost, and the difference is invisible to most users. A target chosen because it sounds serious commits the organisation to spend that buys nothing, and it is the kind of commitment that is very hard to walk back later.
The exhaustion policy is written and signed before it is needed
Costs It commits the roadmap in advance to something nobody wants to commit to.
The error budget is what turns a target into a decision-making tool, and the budget only works if there is an agreed consequence. That agreement is impossible to reach honestly during the incident that spends it — everybody has an interest, the pressure is asymmetric, and the outcome is decided by whoever outranks the room.
Where this has run
Frequently Asked Questions
Sources
- Google — Implementing SLOssre.google
- Google — Site Reliability Engineeringsre.google
- OpenTelemetry — documentationopentelemetry.io
- Principles of Chaos Engineeringprinciplesofchaos.org
Page reviewed
