Lesson 2: The Standard AI Workload Profile

Why profile before you pick

In Level 1 you saw the whole Google Cloud AI map, including the April 2026 move that turned Vertex AI into the Gemini Enterprise Agent Platform. A map that size makes it easy to choose too early. A stakeholder says "we need AI for claims", someone says "Gemini can do that", and three weeks later you find the data was never allowed to leave the EU.

The fix is boring and it works. Before you open a pricing page, write a workload profile: one page that describes the job, not the tool. It works like a structural survey on a house you're thinking of buying. The survey doesn't choose the house for you. It tells you which houses you can't buy.

The profile has twelve fields. Some of them are knock-outs that remove services outright. The rest are weights you'll score later.

The twelve fields

# workload-profile.yaml: fill in one per workload, not one per project
name: claims-form-extraction
task_type: perception          # generative | predictive | perception
build_mode: buy                # buy an API | adapt a model | train one
data:
  sensitivity: confidential    # public | internal | confidential | regulated
  residency: EU-only           # KNOCK-OUT if strict
latency_p95: 30s               # interactive (<2s) vs batch (minutes/hours)
throughput: 40000 pages/month, peaks 4000/day
volume_unit: pages             # tokens | requests | characters | pages | minutes | node-hours
lock_in_tolerance: medium      # can we rewrite the integration in a quarter?
evaluation_and_safety:
  quality_bar: ">=98% field accuracy on 500 labelled forms"
  human_review: yes, below confidence 0.8
mlops_maturity: 1              # 0 none, 1 scripts, 2 pipelines, 3 automated retrain+monitor
accelerators: none needed      # GPUs/TPUs only matter if you train or self-host
team_skills: [python, sql, terraform]   # no ML engineers
cost_model: pay-per-use        # pay-per-use | reserved/provisioned | fixed hosting

Three fields carry most of the decision, so it's worth slowing down on them.

Task type. Generative tasks produce new content: drafting, summarising, chat, code. Predictive tasks output a number or a label from tabular history: churn, fraud score, demand. Perception tasks turn raw media into structure: OCR, speech to text, image labels, translation. Many requests are really two workloads. "Summarise customer calls" is perception (speech to text) feeding generative (the summary). Profile each one separately.

Buy, adapt or train. Buy means calling a ready-made API or a foundation model as it ships. Adapt means prompting with your own data (retrieval), or tuning an existing model. Train means building your own model from your data. Each step down that list costs more people, more MLOps maturity, and often accelerators. The default is to buy, and you need evidence before you move down a step: a measured accuracy gap, a cost curve, or a regulatory reason.

Volume unit. Every AI service bills in some unit, and the profile should name it in that unit. Here are dated list-price examples (October 2026, USD, check before you quote them):

Worked example: from request to profile in 15 minutes

"We get about 40,000 scanned claim forms a month from EU customers. Pull out policy number, claim date and amount so the team stops retyping them."

1. Task: perception (scans in, fields out). Nothing here asks for new text.
2. Build mode: buy. Form extraction is a solved, packaged problem, and the team has no ML engineers.
3. Data: confidential personal data, which must be processed in the EU. This is a knock-out.
4. Latency and throughput: nobody is waiting on screen, so batch within 30 seconds is fine. That allows async and batch pricing.
5. Volume and cost: a rough estimate is enough at this stage:

pages = 40_000

# Option A: page-priced document OCR / extraction
ocr = pages / 1000 * 1.50            # -> $60.00/month

# Option B: a token-priced model reading each page
# ASSUMPTION to verify: ~1,500 input + 200 output tokens per page
gen = pages*1500/1e6*0.30 + pages*200/1e6*2.50   # -> $18 + $20 = $38.00/month

print(f"OCR ${ocr:.2f}  vs  LLM ${gen:.2f}")
# OCR $60.00  vs  LLM $38.00

Gotcha: don't stop at "the LLM is cheaper". At this volume both cost less than an hour of the team's time. The profile tells you that cost is not the deciding field here. Accuracy, residency and review effort are.

6. Evaluation: 500 labelled forms, a 98% field-accuracy bar, and low-confidence results routed to a human. Write this down before you test anything, or you'll end up grading whichever tool you already like.
7. MLOps 1, no accelerators, skills Python/SQL/Terraform, cost model pay-per-use. Together these rule out anything you'd have to train or host yourself.

Knock-outs: rule services out before you compare prices

Work through these in order. Each one is a yes/no gate, not a score.

1. Residency and sovereignty. Gemini on the Agent Platform offers regional endpoints and a global endpoint. Google's own docs say requests to global "may be processed in any Google Cloud location" and give no residency guarantee. So an EU-only workload must use an EU regional endpoint. The other check is whether that model or feature exists in that region at all, and that alone can remove an option.
2. Data handling and sensitivity. Can this data class go to a managed API under your contracts? If it can't, every buy option is out.
3. Latency floor. A voice agent that must reply in under a second rules out batch modes and slow, large models.
4. Skills and MLOps maturity. A team at level 0–1 can't run a train-your-own option safely, however cheap the compute looks.
5. Lock-in tolerance. If you must be able to leave within a quarter, prefer open models or a thin abstraction over proprietary end-to-end features.

Only the services that pass every gate go on to the scorecard you'll build later in this course.

Best practice: keep the profile in the repo next to the Terraform, and change it through review. When someone asks a year later why you didn't use X, the answer is a line in a file, not something one person remembers.

Trade-off: Provisioned Throughput turns token spend into a fixed fee and reserves capacity. That's good for steady, high-volume, latency-sensitive traffic. For spiky or small workloads like the claims example, you'd be paying for idle capacity.

Key takeaways

• Profile the workload, not the project. Requests that mix tasks split into several profiles.
• Sort every task as generative, predictive or perception, then default to buy and step down to adapt or train only when you have evidence.
• Write volume in the service's billing unit (tokens, characters, pages, minutes, node-hours), and treat prices as dated examples.
• Residency, data handling, latency floor, skills and lock-in are knock-outs. Apply them before you compare prices.
• The global Gemini endpoint gives no residency guarantee. Strict-residency workloads need a regional endpoint.
• Set the evaluation bar before you test candidates.