Why you need a profile before a product
Most bad platform choices start with a demo. Someone watches a slick notebook or dashboard, and three months later the team finds out the service can't keep data in the region they need, or bills every query against a table that dashboards hit 5,000 times a day.
The fix is boring and it works. Before you look at any service, write down what the workload actually needs, using numbers. Treat this standard analytics profile like a load-bearing spec in building work: you don't choose the beam and then ask what it has to carry. Every service lesson in this course gets scored against the same ten fields, so filling them in well is the most useful thing you'll do in the course.
The ten fields
| # | Field | Write it as | Not as |
|---|---|---|---|
| 1 | Sources and volume | 4 sources, 180 GB/day, 2,000 events/s peak |
"lots of data" |
| 2 | Batch or streaming | streaming in, hourly batch out |
"real-time-ish" |
| 3 | Freshness target | p95 < 2 min from event to dashboard |
"fast" |
| 4 | Query patterns and concurrency | 40 concurrent dashboard queries, 5 ad hoc |
"many users" |
| 5 | Users | 30 analysts (SQL), 4 engineers, 2 scientists |
"the business" |
| 6 | Open formats | Delta required; Iceberg readable by Snowflake |
"open" |
| 7 | Governance and lineage | column lineage for 12 regulated fields |
"compliant" |
| 8 | Residency | UK South only; no copies outside UK |
"in Europe probably" |
| 9 | Team skills | strong T-SQL, basic PySpark, no Scala |
"capable" |
| 10 | Cost model | prefer a fixed monthly budget, ~£6k cap |
"cheap" |
Gotcha: an adjective can't be wrong, so it can't be tested either. Write
p95 < 2 minand you can measure a proof of concept against it. Write "fast" and no service ever fails.
Walkthrough: turning adjectives into numbers
Take a typical brief: "Our IoT fleet sends a lot of telemetry and ops want it live." Pull a sample hour from the source and do the arithmetic:
# profile_math.py: derive profile numbers from one measured sample
peak_events_per_s = 2_000 # measured from gateway logs at the busiest hour
avg_event_bytes = 1_000 # sample 10k events, take the mean payload size
mb_per_s = peak_events_per_s * avg_event_bytes / 1_000_000
gb_per_day = mb_per_s * 86_400 / 1_000 # worst case: assumes peak all day
print(f"Peak ingress: {mb_per_s:.1f} MB/s")
print(f"Upper-bound volume: {gb_per_day:.0f} GB/day")
print(f"Upper-bound yearly: {gb_per_day*365/1000:.1f} TB/year")
Peak ingress: 2.0 MB/s
Upper-bound volume: 173 GB/day
Upper-bound yearly: 63.1 TB/year
These numbers start doing work straight away. Event Hubs Standard, for example, sizes in throughput units of 1 MB/s or 1,000 events/s of ingress each, whichever limit you hit first. So 2 MB/s at 2,000 events/s already needs two units before you add any headroom. You don't have to choose Event Hubs yet. The point is that a measured number lines up with real pricing and limits, and an adjective doesn't.
Then record the result as a file you keep under version control:
# analytics-profile.yaml (reviewed with the workload owner, 2026-10-29)
workload: fleet-telemetry
sources: {count: 4, peak_events_per_s: 2000, gb_per_day: 173}
mode: streaming-in, batch-out
freshness: {target: "p95 < 2 min", hard: true}
concurrency: {dashboards: 40, ad_hoc: 5}
users: {analysts_sql: 30, engineers: 4, scientists: 2}
formats: {required: [delta], must_be_readable_as: [iceberg]}
governance: {column_lineage: true, regulated_fields: 12}
residency: {region: uksouth, hard: true}
skills: {tsql: strong, pyspark: basic, scala: none}
cost_model: {preference: fixed-monthly, cap_gbp: 6000}
Knock-outs: the two or three answers that rule services out
Most fields are weights. They make a service better or worse, and you'll turn them into a scorecard later in the course. A few are knock-outs: if a service can't meet one, it's out, however good its other scores are. Mark them hard: true and check them first.
Typical knock-outs:
- Freshness in seconds or less. A design built around hourly batch can't be tuned into sub-second insight. That one answer pushes you towards streaming services.
- A required open format. If another engine (Snowflake, Trino) must read your tables as Iceberg, ask exactly how each service supports Iceberg: natively, through metadata conversion, or not at all. Fabric's OneLake, for instance, can present Delta tables to Iceberg readers by generating the metadata, without copying the data. That level of detail decides whether a service passes.
- A fixed residency region. If the service or one of its features isn't available in
uksouth, it's out. Check the feature as well as the service.
Best practice: keep knock-outs to two or three. If everything is marked hard, nothing is, and you end up with no candidates. Team skills, for example, are usually a weight, because people can learn PySpark. A regulator can't make an exception to residency for you.
Turning the cost model into pricing-page questions
Azure analytics services bill in three basic ways. Each one comes with its own questions to ask:
| Model | Example (list prices, US, Oct 2026) | Questions to ask any pricing page |
|---|---|---|
| Reserved or provisioned capacity | Fabric F SKUs at ~$0.18 per CU-hour pay-as-you-go, cheaper with a 1-year reservation | What does it cost when idle? Can I pause it? Does storage keep billing while paused? What happens when I go over capacity? |
| Pay per job or compute time | Databricks bills DBUs per second plus the VMs underneath; Stream Analytics bills streaming units hourly at the highest count in that hour | Is the infrastructure billed separately? What's the billing granularity? Can it scale to zero? |
| Pay per query (data scanned) | Synapse serverless SQL at ~$5 per TB processed, 10 MB minimum per query | Am I billed for data scanned or data returned? Is there a per-query minimum? Do file format and partitioning cut the bill? |
Run your profile numbers through the model. 40 dashboards refreshing 5 times a day, each scanning 50 GB, comes to 200 queries × 50 GB = 10 TB a day. At pay per query that's about $50 a day, every day. With a fixed capacity the cost would be flat. Concurrency (field 4) and cost model (field 10) depend on each other.
Trade-off: reserved capacity gives a predictable bill but charges you while it sits idle. Pay per query costs nothing when idle but can surprise you when usage grows. Write down which risk the budget holder would rather carry.
Key takeaways
- Fill all ten fields with measured numbers taken from real samples, not adjectives.
- Derive volume from events per second × event size, then check it against real service limits.
- Mark two or three knock-outs (freshness, required format, residency) and check them first. Everything else is a weight.
- Each cost model has its own questions: idle cost and pausing for capacity, separate infrastructure billing for pay per job, scanned bytes and minimums for pay per query.
- Keep the profile as a versioned file. Every later service lesson gets scored against it.