Lesson 1: Anatomy of an HTTP Request and Response

What actually goes over the wire

Every API call you'll ever make, whether from curl, Python, a browser or a phone app, comes down to the same exchange. The client sends one request message and the server sends back one response message. RFC 9110 (HTTP Semantics, June 2022) defines what these messages mean. RFC 9112 defines how HTTP/1.1 writes them out as text.

Think of it like posting a letter. The envelope has an address and some labels, then there's the letter itself. A request has three parts:

1. Request line: the method, the target and the HTTP version, e.g. GET /get?lang=en HTTP/1.1
2. Headers: Name: value lines that carry metadata (who you are, what formats you accept)
3. Body (optional): the payload. A GET usually has none.

The response has the same layout:

1. Status line: the version, then a numeric status code, then (in HTTP/1.1) a short reason phrase, e.g. HTTP/1.1 200 OK
2. Headers: metadata about the response (content type, length, date)
3. Body: the data, often JSON

An empty line separates the headers from the body. That's how the receiver knows the metadata is finished and the payload is starting. Once you can read these parts, debugging an API stops being guesswork.

Walkthrough: watch a request with curl -v

We'll use httpbin.org, a free service that echoes your request back as JSON.

# -v (verbose) prints the conversation, not just the body
# --http1.1 forces the classic text format so you see the reason phrase
curl -v --http1.1 "https://httpbin.org/get?lang=en&page=2"

Here's the output, trimmed and annotated:

* Connected to httpbin.org port 443          <- '*' = curl telling you what it's doing
> GET /get?lang=en&page=2 HTTP/1.1           <- '>' = sent by you: REQUEST LINE
> Host: httpbin.org                          <- request headers
> User-Agent: curl/8.x
> Accept: */*
>                                            <- empty line: end of request headers (no body)
< HTTP/1.1 200 OK                            <- '<' = received: STATUS LINE
< Content-Type: application/json             <- response headers
< Content-Length: 270
<                                            <- empty line: headers done, body follows
{
  "args": { "lang": "en", "page": "2" },     <- RESPONSE BODY
  "headers": { "Host": "httpbin.org", ... },
  "url": "https://httpbin.org/get?lang=en&page=2"
}

The rule for reading it:

• > lines are what curl sent
• < lines are what curl received
• * lines are curl's own notes (DNS, connection, TLS)
• Lines with no prefix are the body

Gotcha: HTTP/2 looks different. Leave out --http1.1 and curl will usually use HTTP/2 over HTTPS. Then you'll see < HTTP/2 200 with no reason phrase, and header names in lowercase (content-type:). HTTP/2 dropped the reason phrase and requires lowercase field names on the wire. The meaning is the same either way, and header names are case-insensitive, so your code should never depend on their capitalisation.

The same call from Python

# pip install requests
import requests

r = requests.get("https://httpbin.org/get", params={"lang": "en", "page": 2})

print(r.request.method, r.request.url)    # GET https://httpbin.org/get?lang=en&page=2
print(r.status_code, r.reason)            # 200 OK   (from the status line)
print(r.headers["content-type"])          # application/json  (lookup ignores case)
print(r.json()["args"])                   # {'lang': 'en', 'page': '2'}  (the body, parsed)

A library hides the raw text, but the same parts are still there: method, URL, status, headers and body. Notice that page came back as the string "2". Everything in a query string is text.

Taking a URL apart

Look at this URL:

https://httpbin.org/get?lang=en&page=2#top
└─┬─┘   └────┬────┘└┬─┘└──────┬──────┘└┬─┘
scheme      host   path   query string  fragment

• Scheme (https): which protocol to use. With https the default port is 443, and with http it's 80. You can write a port explicitly, as in localhost:8000.
• Host (httpbin.org): which server to contact. curl sends it in the Host header.
• Path (/get): which thing on that server you want.
• Query string (lang=en&page=2): everything after ?, written as key=value pairs joined by &.
• Fragment (#top): only the client uses it. It is never sent to the server. Look at the curl request line again and you'll see no fragment in it.

You can check this with Python's standard library:

from urllib.parse import urlsplit
u = urlsplit("https://httpbin.org/get?lang=en&page=2#top")
print(u.scheme, u.netloc, u.path, u.query, u.fragment)
# https httpbin.org /get lang=en&page=2 top

HTTP is stateless, and why that shapes APIs

RFC 9110 calls HTTP a stateless protocol: each request can be understood on its own. The server isn't required to remember your previous request. Think of a call centre where every call reaches a different agent who has no notes about you. You have to give your reference number every time.

This has practical consequences for API design:

• Every request carries its own context. Credentials, the page you want and the filters you've chosen all travel in each request, usually as headers or the query string. Asking for page 3 means sending page=3, not "next".
• Any server can answer. No conversation has to be stored, so a load balancer can send request 1 to server A and request 2 to server B. This is a big reason HTTP APIs scale well.
• "State" is something the client sends back. Cookies and tokens don't make HTTP stateful. They're the client handing over its reference number again with each request.

Trade-off: requests get a little bigger because they repeat context. In return, servers become interchangeable and failures are much easier to reason about. Production systems take that deal almost every time.

Try it yourself

curl -v --http1.1 "https://httpbin.org/get?colour=blue#ignored" -H "X-Demo: hello"

Check your result: your > lines should contain GET /get?colour=blue HTTP/1.1 with no #ignored, plus X-Demo: hello. The status line should read HTTP/1.1 200 OK. The JSON body should show "colour": "blue" under args and your X-Demo header under headers.

Key takeaways

• A request is a request line (method, target, version), then headers, then an optional body.
• A response is a status line (version, status code, and in HTTP/1.1 a reason phrase), then headers, then a body.
• An empty line separates the headers from the body in both.
• In curl -v output, > means sent, < means received and * is curl's own commentary.
• A URL breaks into scheme, host, path, query string and fragment. The fragment never reaches the server.
• HTTP is stateless, so every request must carry everything the server needs. That's what lets any server handle any request.