HTTP Recon Toolkit

Almost every API bug is found and proven with three tools: curl to send exactly the request you want, jq to read the JSON that comes back, and HTTPie when you want the same thing with less typing. Master these and you can talk to any HTTP API precisely and repeatably.

Everything below is meant to run against in-scope targets or your own test infrastructure. Point examples at your own dev store, a local app, or a legal lab while you learn the flags.

curl: the deep dive

curl sends one HTTP request and prints the response. Its power is that you control every byte — method, headers, body, cookies — with nothing added behind your back. That precision is exactly what security testing needs.

The examples use https://api.example.com as a stand-in. Swap in an in-scope host.

Choosing the method: -X

By default curl sends GET. -X (or --request) sets the method.

curl -X GET  https://api.example.com/products
curl -X POST https://api.example.com/products
curl -X PUT  https://api.example.com/products/42
curl -X DELETE https://api.example.com/products/42
curl -X OPTIONS -i https://api.example.com/products   # see allowed methods

Caution

-X only changes the method string. If you also pass a body with -d, curl already switches to POST for you — adding -X POST is redundant, and forcing -X GET while sending a body produces a request most servers won’t expect. Set the method deliberately, not by habit.

Headers: -H

-H adds one request header. Repeat it for as many as you need.

curl https://api.example.com/products \
  -H 'Accept: application/json' \
  -H 'Authorization: Bearer eyJhbGci...' \
  -H 'X-Requested-With: XMLHttpRequest'

To remove a default header, give it no value with a trailing semicolon; to send an empty header, give it a colon and nothing:

curl https://api.example.com/ -H 'User-Agent;'   # drop the User-Agent entirely

Request bodies: -d, --data-raw, --json

-d (--data) sends a body and implies POST with Content-Type: application/x-www-form-urlencoded.

curl https://api.example.com/login \
  -d 'email=me@example.com' \
  -d 'password=hunter2'
# body sent: email=me@example.com&password=hunter2

Two gotchas with -d:

• It strips newlines and, if the value starts with @, it treats the rest as a filename to read. Use --data-raw when you want the literal string sent verbatim (including a leading @).
• It does not URL-encode for you. Use --data-urlencode when a value contains &, =, spaces, or other special characters.

curl https://api.example.com/search --data-urlencode 'q=shoes & socks'
curl https://api.example.com/webhook --data-raw '@not-a-file-just-text'

For JSON, either set the header yourself, or use --json (curl 7.82+), which sets Content-Type: application/json and Accept: application/json and sends the body untouched:

# Explicit, works on any curl version:
curl -X POST https://api.example.com/products \
  -H 'Content-Type: application/json' \
  -d '{"title":"Test","price":"9.99"}'

# Shorthand (curl 7.82+):
curl --json '{"title":"Test","price":"9.99"}' https://api.example.com/products

# Read the JSON body from a file or stdin:
curl --json @payload.json https://api.example.com/products
echo '{"title":"Test"}' | curl --json @- https://api.example.com/products

Cookies: -b and -c

-b (--cookie) sends cookies; -c (--cookie-jar) saves the cookies the server sets. Use them together to carry a session across requests.

# Log in and save the session cookies to a jar:
curl -c jar.txt -X POST https://app.example.com/login \
  -d 'email=me@example.com' -d 'password=hunter2'

# Reuse the saved session on a later request:
curl -b jar.txt https://app.example.com/account

# Send cookies inline without a file:
curl -b 'session=abc123; theme=dark' https://app.example.com/account

Authentication: -u

-u user:pass sends HTTP Basic auth (base64 of user:pass in the Authorization header). Leave the password off to be prompted so it stays out of your shell history.

curl -u apikey:x https://api.example.com/admin     # many APIs use the key as the username
curl -u admin https://api.example.com/admin        # prompts for the password

Bearer tokens and API keys are just headers — send them with -H (or the --oauth2-bearer <token> shorthand for Authorization: Bearer):

curl --oauth2-bearer 'eyJhbGci...' https://api.example.com/me
curl -H 'X-Api-Key: 9f8c...'       https://api.example.com/me

Seeing what happened: -i, -v, -s

• -i (--include) prints the response headers followed by the body — essential for reading status codes and Set-Cookie, Location, rate-limit headers, etc.
• -I (--head) sends a HEAD request and prints only the response headers.
• -v (--verbose) prints the whole exchange: > lines are what you sent, < lines are what came back, plus the TLS handshake. This is your primary debugging view.
• -s (--silent) hides the progress meter — pair with -S (--show-error) so real errors still surface. This is the combo you pipe into jq.

curl -i  https://api.example.com/products          # headers + body
curl -v  https://api.example.com/products          # full exchange, for debugging
curl -sS https://api.example.com/products | jq .   # clean output for a pipeline

Tip

Use -v while figuring a request out, then switch to -sS once it works and you want to pipe the body somewhere. For even deeper inspection, --trace-ascii trace.txt dumps the full raw request and response to a file.

Following redirects: -L

curl does not follow 3xx redirects by default — it prints the redirect response and stops. -L (--location) makes it follow. This matters in security testing: whether you follow the redirect changes what you observe.

curl -i  https://example.com/old-path    # shows the 301 and the Location header
curl -iL https://example.com/old-path    # follows it and shows the final response

Caution

By default -L will resend credentials to the redirect target, even a different host. Use --location-trusted only when you intend that; otherwise be aware that a redirect off your in-scope host could send your Authorization header somewhere out of scope. Watch the Location header.

Saving output: -o

-o file writes the body to a file; -O saves it under its remote filename. Combine with -J to honor a Content-Disposition filename.

curl -o response.json https://api.example.com/products
curl -O https://cdn.example.com/app.js       # saves as app.js

Overriding DNS: --resolve

--resolve host:port:address pins a hostname to an IP for this request only, without editing /etc/hosts. Useful to hit a specific origin behind a load balancer, or test a host header while keeping TLS SNI correct.

# Send the request to 203.0.113.10 but keep the Host and TLS cert for api.example.com:
curl --resolve api.example.com:443:203.0.113.10 https://api.example.com/health

The related --connect-to host:port:connecthost:connectport is more flexible when host and SNI need to differ.

Going through a proxy: -x

-x (--proxy) routes the request through a proxy — this is how you send curl traffic through Burp or ZAP so it is logged and replayable.

# Route through a local intercepting proxy; -k accepts the proxy's TLS cert.
curl -x http://127.0.0.1:8080 -k https://api.example.com/products

Danger

-k (--insecure) disables TLS certificate verification. It is fine for talking to your local proxy or a lab with a self-signed cert. Never use it as a habit against real targets — you lose the guarantee that you’re talking to who you think you are.

Uploading files: -F (and raw uploads)

-F (--form) sends multipart/form-data, the format HTML file uploads use. A field whose value starts with @ uploads that file; type= and filename= let you control the declared content type and name — handy for testing upload validation.

# Upload a file as the "avatar" field, plus a normal text field:
curl -F 'avatar=@cat.png' -F 'caption=my cat' https://app.example.com/upload

# Lie about the content type to test server-side validation (in scope only):
curl -F 'avatar=@shell.php;type=image/png;filename=cat.png' https://app.example.com/upload

To send a file as the raw request body (not multipart) — e.g. a JSON or XML API — use --data-binary @file:

curl -X POST https://api.example.com/import \
  -H 'Content-Type: application/xml' \
  --data-binary @data.xml

Timing and metrics: -w

-w (--write-out) prints selected variables after the transfer — perfect for measuring latency (useful when probing for timing differences) and capturing the status code without parsing the body.

curl -s -o /dev/null \
  -w 'code=%{http_code} time=%{time_total}s size=%{size_download}B\n' \
  https://api.example.com/products
# => code=200 time=0.183s size=4213B

Useful -w variables: %{http_code}, %{time_total}, %{time_namelookup}, %{time_connect}, %{time_appconnect} (TLS), %{time_starttransfer} (time to first byte), %{size_download}, %{num_redirects}, %{url_effective}, %{content_type}, %{remote_ip}. Newer curl (7.70+) also supports %{json} to emit all of them as one JSON object:

curl -s -o /dev/null -w '%{json}' https://api.example.com/products | jq '.http_code, .time_total'

A full request, annotated

curl -sS -i \
  -x http://127.0.0.1:8080 -k \
  -X POST \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer eyJhbGci...' \
  --data-raw '{"title":"IDOR test","vendor":"me"}' \
  -w '\n--- %{http_code} in %{time_total}s ---\n' \
  https://api.example.com/products

HTTPie: the friendlier client

HTTPie (http / https on the command line) is a higher-level client that assumes JSON, colorizes output, and has a terse request syntax. It is great for quick exploration; drop back to curl when you need byte-level control.

Its argument syntax uses different separators for different parts of the request:

Syntax	Meaning	Example
key=value	JSON string field (in the body)	title=Shoes
key:=value	Raw JSON field (number, bool, array, object)	price:=9.99 active:=true
key==value	URL query-string parameter	page==2
Header:value	Request header	Authorization:'Bearer x'
key@file	Upload a file field	avatar@cat.png

# GET with a query param and a header (method defaults to GET):
http https://api.example.com/products page==2 Accept:application/json

# POST JSON (POST is inferred because there are body fields):
http POST https://api.example.com/products title=Test price:=9.99 active:=true
# sends: {"title":"Test","price":9.99,"active":true}

Flags worth knowing:

• -v / --verbose prints the full request and response (like curl’s -v but readable).
• --offline builds and prints the request without sending it — great for confirming exactly what you’d send.
• -a user:pass sets Basic auth; -A bearer -a <token> sets a bearer token; -A is --auth-type.
• --form / -f switches the body to form-encoding instead of JSON.
• --follow follows redirects (off by default, like curl).
• --proxy=http:http://127.0.0.1:8080 routes through your proxy; --verify=no skips cert checks.
• --session=name persists cookies and headers across calls under a named session.

http --offline POST https://api.example.com/products title=Test price:=9.99
http -A bearer -a 'eyJhbGci...' https://api.example.com/me
http --proxy=http:http://127.0.0.1:8080 --verify=no https://api.example.com/products

jq: slicing JSON

APIs answer in JSON, and jq is a filter language for it. You pipe JSON in, give it a program, and it prints the transformed result.

echo '{"id":42,"name":"Widget","tags":["a","b"]}' | jq .

The essentials:

data='{"products":[{"id":1,"title":"A","price":10,"active":true},
                    {"id":2,"title":"B","price":20,"active":false}]}'

echo "$data" | jq '.products'              # the array
echo "$data" | jq '.products[]'            # each element, one per line
echo "$data" | jq '.products[0].title'     # "A"
echo "$data" | jq '.products | length'     # 2
echo "$data" | jq 'keys'                   # object keys, sorted
echo "$data" | jq '.products[].id'         # 1 then 2 — pull one field from each

-r (raw output) strips the JSON quotes — use it whenever you feed the result into another command (like a bash loop over IDs). -c prints compact one-line JSON.

echo "$data" | jq -r '.products[].id'      # 1\n2  (no quotes)
echo "$data" | jq -c '.products[]'         # {"id":1,...} one per line

Reshaping and filtering:

# Keep only active products, and only their id + title:
echo "$data" | jq '.products[] | select(.active == true) | {id, title}'

# Build a new object / rename fields:
echo "$data" | jq '.products[] | {productId: .id, label: .title}'

# Pass a shell variable in safely:
name="Widget"
echo "$data" | jq --arg n "$name" '.products[] | select(.title == $n)'

curl + jq together

This pairing is the workhorse of API testing — send a request, and immediately dig into the response.

# List the IDs the API returns for your account:
curl -sS https://api.example.com/orders -H 'Authorization: Bearer eyJhbGci...' \
  | jq -r '.orders[].id'

# Count results and show the first record:
curl -sS https://api.example.com/orders -H 'Authorization: Bearer eyJhbGci...' \
  | jq '{count: (.orders | length), first: .orders[0]}'

# Extract only error messages to study how the API fails:
curl -sS https://api.example.com/products/999999 \
  | jq '.errors // .error // .message'

Capturing and replaying requests

A verified finding needs an exact, repeatable request. Two habits make that painless:

Copy as cURL from the browser or proxy. In your browser’s DevTools Network tab (right-click a request → Copy → Copy as cURL), or in Burp/ZAP, you get a full curl command with the real headers and cookies. Paste it into your terminal, then start editing one thing at a time — change an ID, drop a header, flip a method.

Save requests to files and diff them. Keep the exact commands that produced a finding so you can rerun and hand them over verbatim.

# Save a working baseline request as a runnable script:
cat > req-baseline.sh <<'EOF'
curl -sS https://api.example.com/orders/1001 \
  -H 'Authorization: Bearer eyJhbGci...' \
  -H 'Accept: application/json'
EOF
chmod +x req-baseline.sh

# Copy it, change the ID, and diff the two responses:
sed 's#/orders/1001#/orders/1002#' req-baseline.sh > req-other.sh
diff <(bash req-baseline.sh) <(bash req-other.sh)

Tip

Route these replays through your proxy (-x http://127.0.0.1:8080) so every attempt is logged. When it’s time to report, a proxy history plus your saved scripts give you clean, timestamped evidence.

Reading responses methodically

Don’t just eyeball the body. For every response, read it in this order — it’s where the signal is:

1. Status code. 200 vs 201 vs 204; 301/302 (where does it go?); 401 (unauthenticated) vs 403 (authenticated but forbidden) — that distinction is central to authorization testing; 429 (rate limited); 500 (you may have broken something server-side). curl -o /dev/null -w '%{http_code}\n' gets just the code.
2. Headers. Set-Cookie, Location, WWW-Authenticate, Content-Type (does it match the body?), CORS headers (Access-Control-Allow-Origin), rate-limit headers (X-RateLimit-Remaining, Retry-After), and any framework or version leakage.
3. Timing. A consistently slower response can reveal a real code path (e.g. a valid vs invalid username). -w '%{time_total}' measures it; repeat several times before trusting a difference.
4. Body. Now read the JSON with jq. Look for fields you shouldn’t see, verbose error messages, stack traces, internal IDs, and differences between authorized and unauthorized responses.

# One-liner status probe across a few endpoints:
for p in /products /orders /users /admin; do
  code=$(curl -s -o /dev/null -w '%{http_code}' "https://api.example.com$p")
  printf '%s -> %s\n' "$p" "$code"
done

Key points

• curl gives you byte-level control: -X method, -H headers, -d/--data-raw/--json bodies, -b/-c cookies, -u auth, -i/-v/-s visibility, -L redirects, --resolve DNS pinning, -x proxy, -F uploads, -w metrics.
• Know the -d gotchas: it strips newlines, treats leading @ as a file, and doesn’t URL-encode — reach for --data-raw and --data-urlencode.
• HTTPie is the fast, JSON-first alternative; --offline shows exactly what you’d send.
• jq turns JSON into answers; -r for raw values you feed into loops, select/{...} to filter and reshape.
• Read responses in order — status, headers, timing, body — and route everything through a proxy so it’s logged and replayable.