Why a profile comes before a product
In Level 1 you saw every Azure storage service on one page. The obvious next step is to compare prices, but that is the wrong order. Pricing pages only make sense once you know what you are buying for. Teams that skip this step end up paying premium-disk prices for log files that get read once a month, or putting a database on a share that cannot reach its latency target.
The fix is a standard data profile: one short, fixed-format record per workload. A doctor takes your blood pressure before prescribing anything, and the profile plays the same role here. Every service lesson after this one will be scored against it, so you fill it in once and reuse it many times.
The profile template
Save this as profile.yaml next to your project. Every field needs a number or a named choice. Adjectives are not allowed.
# Standard Data Profile v1 - one per workload
workload: invoice-archive # short, unique name
owner: finance-platform-team
size:
today_tib: 4.2 # measured, not guessed
growth_tib_per_month: 0.15 # from last 6 months of data
horizon_months: 36 # plan for this far ahead
access:
type: object # object | block | file
protocol: [REST, SFTP] # e.g. REST, SFTP, NFS 3.0, SMB, NFS 4.1, iSCSI
performance:
iops_peak: 200
throughput_mibps_peak: 50
latency_p99_ms: 500 # the slowest 1% may take up to this long
pattern:
read_write_ratio: "95:5"
io_size: large-sequential # small-random | large-sequential | mixed
hot_window_days: 30 # most reads happen within this many days
protection:
durability: "no loss beyond RPO"
rpo_minutes: 60 # max data you can afford to lose
rto_hours: 8 # max time to be back online
redundancy_target: zone-and-region # single-site | zone | region | zone-and-region
availability_target: "99.9%"
consistency: strong-read-after-write # or eventual is acceptable
sharing: many-readers-one-writer # single-VM | many-VMs | many-users
retention:
keep_years: 7
immutability: time-based-WORM # none | time-based-WORM | legal-hold
deletion: "delete within 30 days of retention expiry"
residency: [UK South, UK West] # where data may legally live
compliance: [UK GDPR, HMRC records]
cost_model: lowest-capacity-cost # capacity | performance | predictable-monthly
Turning vague words into numbers
Stakeholders will tell you that a system must be "fast" or "must never lose data". Your job is to translate those words into figures someone can measure.
| They say | You write | How you check it |
|---|---|---|
| "It must be fast" | p99 latency ≤ 5 ms, 8,000 IOPS peak | Measure on the current system |
| "Never lose data" | RPO 15 min, RTO 4 h | Agree with the business owner and sign it off |
| "Keep it forever" | Retain 7 years, then delete within 30 days | Check the legal or regulatory rule |
| "Lots of data" | 4.2 TiB today, +0.15 TiB per month | Storage reports, du, billing exports |
Measure on the system you have today. On Linux, iostat shows the real numbers:
# Sample disk stats every 60 s, 60 times (one busy hour)
iostat -dxm 60 60 > io-sample.txt
# Columns to read:
# r/s + w/s -> IOPS
# rMB/s + wMB/s -> throughput
# r_await/w_await -> average latency in ms
# Example busy-hour line:
# Device r/s w/s rMB/s wMB/s r_await w_await
# sdb 6120 1880 95.6 29.4 1.8 3.1
# => ~8,000 IOPS, ~125 MiB/s, 2-3 ms average
Gotcha: an average hides the slowest requests, and those are what users notice. Size for the busy-hour peak, and ask for p99 latency, not the mean. If your tooling only gives averages, write that down in the profile.
Gotcha: compound growth catches people out. 4.2 TiB plus 0.15 TiB a month over 36 months is about 9.6 TiB, more than double what you have now.
Use access type and protocol to rule out whole groups
The access field is the cheapest filter you have. Apply it before you look at any price.
- Block (a raw disk that the operating system formats): Managed Disks and Elastic SAN. Elastic SAN volumes connect over iSCSI.
- File (shared folders): Azure Files (SMB and NFS 4.1), Azure NetApp Files, and Azure Managed Lustre for high-performance computing.
- Object (HTTP access to whole objects): Blob Storage over REST. Blob can also be reached over NFS 3.0 and SFTP.
Worked example: a Windows app that maps a drive letter needs type: file, protocol: [SMB]. That one line rules out the block services, Lustre and plain Blob. You now compare two or three services, not ten.
Trade-off: a protocol that "also works" is not always a good fit. Blob over NFS 3.0 lets Linux mount a container, but it does not behave exactly like a classic file server. Record the protocol the application needs, not one it could be made to use.
Protection: durability is not availability
Keep these four fields separate in the profile:
- Durability means the data is not lost. Azure documents at least 11 nines a year for LRS (three copies in one location), 12 nines for ZRS (three availability zones) and 16 nines for GRS (adds an asynchronous copy in a second region).
- Availability means you can reach the data right now. Durable data can still be unreachable during an outage.
- RPO and RTO are your recovery promises. Geo-replication is asynchronous, and Microsoft says the lag is typically under 15 minutes but does not guarantee it unless you pay for the optional Geo Priority Replication feature. An RPO of 0 across regions therefore needs a different design.
- Redundancy target is the failure you must survive: one site, one zone, or a whole region.
Put immutability next to retention. On Blob Storage, a time-based retention policy lets you create and read data but not change or delete it until the period ends. A legal hold blocks changes until someone explicitly clears it. Write down which one the rule requires, and how quickly data must be deleted afterwards, because under GDPR keeping data too long can be as much of a breach as losing it.
Key takeaways
- Fill in the profile before you compare prices, using numbers and named choices, never adjectives.
- Measure busy-hour peaks and p99 latency on the system you have today, and project growth over your planning horizon.
- Access type and protocol come first because they rule out whole groups of services.
- Record durability, availability, RPO/RTO and redundancy target as separate fields.
- Record retention, immutability type (time-based vs legal hold) and the deletion deadline next to each other.