Lesson 2: The Standard Data Profile

Why you need a profile before you need a product

In Level 1 you saw every Google Cloud storage service on one page. That list doesn't choose anything for you. Ask five engineers which service suits a workload and you'll get five answers, because each one pictures a different workload. The fix is boring and it works: write the workload down once, in a fixed format, with numbers and units. That record is the data profile. Every later lesson in this course measures a service against it.

Think of it like a patient's chart. A doctor doesn't prescribe from "I feel unwell". They record temperature, blood pressure and history first. "Fast", "safe" and "big" are how the workload feels. The profile records the measurements.

The fields, and the questions behind them

Gotcha: units. MB/s (10^6) and MiB/s (2^20) differ by about 5%. Vendor docs mix them, so pick one (this course uses MiB/s and TiB) and convert everything else to it. An IOPS figure without an I/O size means little: 20,000 IOPS at 4 KiB is about 78 MiB/s, while at 64 KiB it is about 1,250 MiB/s.

Turning vague words into targets

• "Fast" becomes a latency figure at a percentile, plus IOPS or throughput. Use p99, not the average. An average of 1 ms can hide 1% of requests taking 200 ms, and users notice those.
• "Safe" splits into three figures. RPO (recovery point objective) is how much recent data you can afford to lose. RTO (recovery time objective) is how long recovery may take. Copies is how many independent copies you keep and how far apart they are, for example the 3-2-1 rule: three copies, on two kinds of media, one off-site.
• "Always on" becomes an availability percentage. 99.9% allows about 8.8 hours of downtime a year. 99.99% allows about 53 minutes.
• "Durable" describes something different from availability. Cloud Storage, for example, is designed for 99.999999999% (11 nines) annual durability. That figure measures whether your bytes survive, not whether you can reach them right now. The profile records both.

Walkthrough: measure, then write it down

Get your numbers from the running system, not from memory. On an existing Linux database server:

# Extended device stats every 60 s, 5 samples. Run it at peak hour, not at 3 a.m.
iostat -x 60 5
# Read: r/s + w/s = IOPS; rkB/s + wkB/s = throughput;
#       r_await / w_await = average latency in ms (an average, so add headroom for p99)

# Capacity in use on the data volume
df -h /var/lib/postgresql

If some data already sits in a bucket:

# Total size of a bucket, human-readable
gcloud storage du gs://acme-invoices --summarize --readable-sizes

Then record it in a file that lives in the repo next to the infrastructure code:

# data-profile.yaml: card-payments ledger (profile dated 2026-10-24)
workload: payments-ledger
size: {today_tib: 2.5, growth_pct_month: 4, at_36_months_tib: 10.3}
access: {type: block, protocol: none, consumer: "1 PostgreSQL primary VM"}
performance:
  iops: {peak: 18000, io_size_kib: 8}      # from iostat at month-end peak
  throughput_mib_s: {read: 250, write: 90}
  latency_p99_ms: 2
pattern: {read_write: "70/30", access: random}
protection: {rpo_minutes: 5, rto_minutes: 60, copies: 3, regions: 2}
availability_target: "99.95%"
consistency: strong                         # ledger: no stale reads
sharing: {writers: 1, readers: 1}           # replicas get their own storage
retention: {keep_years: 7}
immutability: {required: true, form: "WORM, locked retention period"}
deletion: {requirement: "erase on request within 1 month (GDPR Art. 17)"}
residency: {allowed: ["europe-west1", "europe-west4"]}
compliance:
  - {regime: PCI DSS, control: "PAN unreadable wherever stored (Req. 3.5.1)"}
  - {regime: GDPR, control: "EU residency, erasure on request"}
cost_model: {prefer: provisioned, budget_usd_month: 4000}

Trade-off to flag early: immutability and erasure can clash. A locked WORM retention period means nobody can delete the data early, you included. If personal data might be subject to an erasure request, record that conflict now. A common answer is to keep personal data out of the immutable store, or to encrypt it per subject and destroy the key. Settle it in the profile so it doesn't surface in an audit.

Compliance next to residency

Residency answers where the data may live. Compliance answers what controls must surround it. Record both, and record each regime with the specific storage control it requires:

• GDPR: lawful location and transfers, and erasure "without undue delay". In practice that means within one month of the request, extendable by two months for complex cases.
• HIPAA: in Google Cloud, protected health information may only go into services covered by Google's Business Associate Addendum, and you must have that agreement in place.
• PCI DSS v4.0.1: card numbers (PAN) must be unreadable wherever they are stored. Disk-level encryption alone is not enough on non-removable media.

On Google Cloud you can enforce residency with the organization policy constraint gcp.resourceLocations. It applies only to resources created after you set it.

Using the profile to rule out whole groups

Before you look at any single product, eliminate groups:

1. Access type first. The ledger needs block access with 2 ms p99. That rules out object storage and archive straight away.
2. Sharing next. One writer on one VM means you don't need shared file storage, so file and high-performance file drop out.
3. Protection and compliance last. RPO 5 minutes across 2 regions means block storage alone isn't enough. Note "needs replication or backup" for the transfer and backup group.

Three questions take you from seven groups down to block, plus backup. Comparing products inside those groups starts in Level 3.

Key takeaways

• Write the workload down once, in a fixed format with units, before you pick any service.
• Replace adjectives with targets: p99 latency, IOPS at an I/O size, MiB/s, RPO/RTO, copies, availability %.
• Get the numbers from a running system at peak, and date the profile.
• Record immutability and deletion next to retention, and flag any conflict between them.
• Record each compliance regime with the storage control it requires, next to residency.
• Use access type, sharing and protection to rule out whole storage groups first.