X-Forwarded-For

Software development can be a bit of a roller-coaster. Certain portions are simple, and others are incredibly complex - and that's before you get to the subjective areas. IP address detection is one which sits squarely in the middle. To some, it just happens, and they don't need to care about how it works, because it does always work. But that's only because someone else has put in the time and effort to ensure it's implemented correctly and securely.

IP address detection is something people often get wrong - with potentially dangerous consequences depending on your use case. Fortunately, once you understand how it works, it's incredibly simple to get right, and often simpler than other implementations out there.

#What is X-Forwarded-For?

If you're working on a project locally, your browser will likely be talking to the web server directly. When this happens, your framework can easily tell the client's IP, as it's the device connecting to the socket directly, and thus where your OS will send the response.

However in production, it's likely you'll have (often more than one) proxy in the middle, whether it be a load balancer, CDN, or something more exotic. In this case, the OS will see the reverse proxy as the device sending it requests, rather than your browser.

Since "Proxy B" is the device the server's OS sees as connecting, that's the IP address it will report as the "Client"'s, despite that not being accurate. In these cases, a side-channel is needed to convey the correct IP address, which won't be overwritten by something in the middle.

X-Forwarded-For is a HTTP header which can be used to determine the IP address of the client. More specifically, X-Forwarded-For contains not only the IP address of the client, but also of every proxy in between.

#What does it look like

Here's an example of a X-Forwarded-For header:

X-Forwarded-For: 203.0.113.1, 198.51.100.101

It's simply a comma-separated list of IP addresses. It can be read from right to left, getting further away from the web server itself, with the left most being the address of the client (usually, more on that later). If we received this header, in combination with 198.51.100.102 being the connecting IP (which I'll refer to as RemoteAddr), we'd interpret it as:

Whenever a proxy receives a request, it looks at the X-Forwarded-For header. Whatever the value is, the proxy adds the IP address that connected to it onto the end, and passes it down the chain. The "Server", being the final destination, simply reads the values from the header and extracts the client's IP address from it.

This nicely illustrates the issues with relying on the connecting address (RemoteAddr), and the benefits of X-Forwarded-For. In each link in the chain, the previous data for RemoteAddr is lost. Only "Proxy A" saw the correct value - everyone else got an incorrect value. By using X-Forwarded-For, connection information is correctly passed through.

#Misconfiguration

Most proxies will handle X-Forwarded-For automatically - there's rarely a need to configure X-Forwarded-For if all you're doing is passing it through (and thus not acting on it). If you need to do something with the value (ie determine the client's IP), it's more important you configure it properly.

IP detection in itself may not sound important, but it's often used in a number of very important areas, such as Authentication, rate limiting, Geolocation and audit trails. In these cases, using the wrong IP address can go from a bug to a problem.

The biggest gotcha with X-Forwarded-For is verification. If not configured correctly, there's nothing prevent a malicious actor from posing as a different IP address. With the connecting address, if you try and spoof it you'll never get a reply, and given HTTPS (more specifically TLS) requires a 2-way handshake, you probably wouldn't get as far as the server. Setting arbitrary headers on the other hand is trivial, and in fact a great feature of HTTP.

Let's take the above example, but try and trick the system to appear as a different IP. What if the client sent an X-Forwarded-For themselves?

Each proxy is doing exactly what it should: Taking the connecting IP address and adding it to X-Forwarded-For. The problem is, the initial value was malicious. By passing a value for itself, the client has changed the left-most address to be a malicious value (203.0.113.2), which differs from its actual IP address (203.0.113.1). The same trick is possible, and can be harder to detect, if the client has a way of talking to "Proxy B" directly, without going via "Proxy A", as then even the number of IP addresses would be correct.

#Protections

Naturally, being able to circumvent any IP detection measures this easily is bad. Fortunately, protecting against them is just as simple. There are a few different techniques to go about it, and you may need to employ multiple depending on your hosting environment.

#Trusted Routes

The first is to ensure traffic can only come via routes you expect. This way, you can ensure those routes are configured correctly and everything else is blocked. If you need multiple proxies in the chain, make sure they're all required.

Here, the user can only access the server via both "Proxy A" and "Proxy B". Access directly to "Proxy B" or the server itself is blocked. Exactly how this works will depend on your environment - it could be an IP restriction, or a firewall, or even Basic Auth.

Doing this ensures a malicious user can't insert themselves into the chain and spoof the header's value - pretending to be "Proxy A" in a way.

If this isn't possible, you can also configure the proxy to only trust the X-Forwarded-For header from certain addresses. For example, "Proxy B" might be internet-accessible, but it only trusts existing X-Forwarded-For values from "Proxy A" and ignores any other.

#Discard values at the edge

When a proxy receives traffic with a header, it adds its own data to what's already there. For intermediary proxies, this is ok, as we expect a proxy before it to include its own value. However for the outer-most proxy, this probably isn't what you want.

The outer-most proxy shouldn't receive traffic proxied to it - it's the outer-most. You can therefore configure this proxy to ignore any existing X-Forwarded-For values, and just include the connecting client's IP. If a client, malicious or otherwise, provides an X-Forwarded-For header, it gets silently ignored.

If the intermediary proxies are internet-accessible, similar protections can be added. This time, instead of blanket discarding X-Forwarded-For headers, it can discard values based on the connecting address. For example, "Proxy B" might be internet-accessible, but it only trusts existing X-Forwarded-For values from "Proxy A" and ignores any other.

#Ignore invalid values

And of course, you don't need to discard all values. At all stages in the chain, it's a good idea to ignore any invalid-looking header, knowing the only way they could have got there is a malicious actor putting them there. Perhaps the header contains an IP address 666.666.666.666 or 127.0.0.1. I suspect the proxy didn't receive a request from itself or from the devil an invalid IP address. In those cases, it's a good idea to ignore them, regardless of where they came from.

Depending on how security conscious you are, "ignored" isn't the only option - you can also block the requests outright. Any invalid-looking request is almost certainly not legitimate, so blocking it prevents any further potential security issues.

#Alternatives

X-Forwarded-For is the standard, but it's not the only proxy-compatible way of getting the true IP address from a reverse proxy - Some use their own completely separate standard. Cloudflare for example uses uses CF-Connecting-IP, whilst CloudFront uses CloudFront-Viewer-Address. You might think that requires custom logic depending on which CDN you're using, but you don't - that's the value of a standard. Both Cloudflare and CloudFront (and I strongly suspect most others) follow the X-Forwarded-For spec, and send the header correctly and securely by default. So long as you can be sure it's only them sending you traffic, there's no custom behaviour necessary. In some cases, these custom headers only contain a single IP address in the header, making it almost impossible to detect tampering with the header alone.

X-Forwarded-For is just a part of a number of helpful X-Forwarded-* headers in the standard. X-Forwarded-Host ensure the Host header is passed through reverse proxies unmodified, whilst X-Forwarded-Proto can be used to ensure the client's connection was secure, even if others in the chain aren't.

If you're developing an application, chances are your infrastructure is already sending you these headers. If you're using a framework, it's even likely it's already considering these headers when getting a client's IP (or in fDjango's case, it delegates it entirely to you). Because there's a well defined, relatively simple standard, you can learn the intricacies of X-Forwarded-* once, and apply them no matter your deployment environment and no matter your use case.