SSRF: The Server Fetches What You Say

The shop has a link preview: a visitor sends POST /preview with a url, the server fetches that address with requests.get, and the answer holds the first 100 characters. The catch is who fetches: the shop's server, not the visitor.

The attack: the supplier's private API listens on 127.0.0.1:8001, the shop's own machine, so nobody on the internet can reach it. But the shop's server can, and it fetches whatever a visitor names. This is Server-Side Request Forgery (SSRF). What the attacker gets: the supplier's private price list, just by sending {"url": "http://127.0.0.1:8001/prices"}.

Refusing bad addresses one by one doesn't work: there is always one more (localhost, 127.0.0.2, [::1], …). Allow a short list of good sites instead, and compare the real host. urlparse, from urllib.parse, splits an address into its parts:

parts = urlparse("https://shop.example:8443/help?page=2")
parts.scheme     # "https"
parts.hostname   # "shop.example"
parts.port       # 8443

Compare parts.hostname, not the start of the text: https://blog.example.com@127.0.0.1:8001/prices starts like the blog, but what comes before an @ is a login name, and the host is 127.0.0.1. Redirects are covered already: the starting code fetches with allow_redirects=False, so an allowed site can't send the server on to the supplier.

Your code must answer:

POST /preview  {"url": "https://blog.example.com/mug-review"}
  →  200  {"preview": "<h1>Our favourite mug</h1><p>The Corner Shop mug keeps tea hot for an hour.</p>", "status": 200}
POST /preview  {"url": "http://127.0.0.1:8001/prices"}
  →  400  {"error": "Only https links can be previewed"}
POST /preview  {"url": "https://127.0.0.1:8001/prices"}
  →  400  {"error": "This site can't be previewed"}
POST /preview  {"url": "https://blog.example.com@127.0.0.1:8001/prices"}
  →  400  {"error": "This site can't be previewed"}
POST /preview  {"url": "http://blog.example.com/mug-review"}
  →  400  {"error": "Only https links can be previewed"}
POST /preview  {"url": "file:///etc/passwd"}
  →  400  {"error": "Only https links can be previewed"}
POST /preview  {"url": "https://blog.example.com/old-post"}
  →  200  {"preview": "", "status": 302}

Your task: add a list PREVIEW_HOSTS with "blog.example.com" and "news.example.com". In preview, split the url with urlparse. Answer 400 with "Only https links can be previewed" when the scheme isn't https, and 400 with "This site can't be previewed" when the hostname isn't on the list.

Assignment
Follow the Coding Tutorial to let /preview fetch only https pages from the two allowed sites.

Hint
Check the scheme first, then the hostname, and do both before requests.get, so a refused address is never fetched. not in tells you a value is missing from a list.