Lesson 2: The Standard Container Workload Profile

Why a profile comes before a product

Every bad container platform choice I have seen started the same way. Someone said "we need Kubernetes" or "let's go serverless" before anyone had written down what the workload actually does. Then the team spends a year working around the platform.

The fix is boring but it works. Before you look at any service from Level 1, fill in a standard workload profile. It is like a doctor's intake form: the same questions every time, so you can compare patients, which here means workloads. Every later lesson in this course reads from it, and so does the weighted scorecard near the end.

The profile has eight fields:

1. Services: how many deployable units, and how they talk to each other
2. State: stateless, or holding data on local disk or in memory
3. Traffic and scaling: requests per second, peak-to-average ratio, and the cold start you can accept
4. GPUs: none, occasional, or always on, plus the type you need
5. Skills and operating model: the Kubernetes depth your team can keep up at 3 a.m., not on a good day
6. Portability: must it move to another cloud or on-premises?
7. Compliance: regulated landing zone, data residency, supported-version windows
8. Cost model: pay per use, flat reserved capacity, or a fixed budget ceiling

The rule for each field is evidence, not opinion. "High traffic" is opinion. "Peak 240 req/s from last November's gateway logs" is evidence.

Walkthrough: profiling a sample system

Our sample is an order-tracking system: a public API, a web front end, a background worker and a nightly report job. Keep the profile as a file in the repo, so it gets reviewed like code:

# workload-profile.yaml: one per workload, kept next to the code
workload: order-tracking
services:
  count: 4                      # api, web, worker, nightly-report
  evidence: "repo /deploy folder, 4 Dockerfiles"
state:
  kind: stateless               # all state lives in Azure SQL + Service Bus
  evidence: "no volumes in docker-compose.yml"
traffic:
  avg_rps: 30                   # from the calculation below
  peak_rps: 240
  peak_to_avg: 8
  idle_hours_per_day: 7         # 00:00-07:00 near-zero traffic
  acceptable_cold_start_s: 5    # web/api; the report job can wait minutes
  evidence: "App Gateway logs, Nov 2025 to Oct 2026"
gpus:
  needed: false
skills:
  kubernetes_level: basic       # 1 engineer has run a cluster; none on call for it
  ops_model: "product team, no platform team"
portability:
  required: low                 # OCI images only; no multi-cloud mandate
compliance:
  regulated_landing_zone: false
  data_residency: "EU regions"
cost_model: pay-per-use preferred; budget ceiling 1,500 EUR/month

Turning "needs to scale" into numbers

When a stakeholder says "it needs to scale", ask three questions. Here is the arithmetic for each, in code rather than formulas:

# 1. Average requests per second, from daily volume in the logs
daily_requests = 2_600_000
avg_rps = daily_requests / 86_400        # = 30.1 req/s

# 2. Peak-to-average ratio. Take the busiest minute, not the busiest day
peak_rps = 240                           # busiest minute / 60
peak_to_avg = peak_rps / avg_rps         # = 8.0

# 3. Concurrency at peak = rate x time each request takes
p95_latency_s = 0.2
peak_concurrency = peak_rps * p95_latency_s   # = 48 requests in flight

The third number matters because many autoscalers count concurrent requests, not requests per second. Azure Container Apps' HTTP scale rule, for example, adds a replica each time concurrent requests pass a threshold, and that threshold defaults to 10. At 48 in flight, that is about 5 replicas at peak.

The last figure is acceptable cold start: how long the first user after a quiet spell may wait while a replica starts. Ask the business directly: "Is 5 seconds on the first request after midnight acceptable?" If the answer is "no, never", then scale to zero is off the table for that service, whatever the savings.

Gotcha: a ratio of 8 with 7 idle hours a day is a very different workload from a ratio of 8 with no idle time. Record both. Idle hours are where scale to zero pays off.

Knock-out fields: rule services in or out straight away

Some fields don't just add points. They end the discussion. Check these first, before any detailed scoring:

Trade-off: knock-outs are blunt on purpose. A small team with no on-call cover may still pick a six-node OpenShift cluster because the company standard demands it. Write that down as a compliance answer, not a skills answer, so the cost is visible.

Best practices from production

• Profile per workload, not per company. The nightly report job and the public API have different traffic shapes. One profile hides that.
• Date every number. Traffic grows, and GPU regions and service features change. A profile from 2024 is a historical document.
• Be honest about skills. "We could learn Kubernetes" isn't an operating model. Who patches, upgrades and answers the pager? Name the people.
• Keep an evidence column. Logs, invoices and audit findings beat meetings.

Key takeaways

• Fill in the same eight fields for every workload: services, state, traffic and scaling, GPUs, skills and operating model, portability, compliance, cost model.
• Turn "needs to scale" into average req/s, peak-to-average ratio, peak concurrency (rate × latency) and an agreed cold start.
• Check knock-out fields first. Scale to zero, GPUs (ACI's GPU support retired on 14 July 2025), an OpenShift standard and long version-support needs remove options at once.
• Back every field with dated evidence, and store the profile next to the code.