Fixing HTTP 502 errors of Services Behind an Application Load Balancer

tl;dr If a load balancer occasionally returns 502 while the backend looks perfectly healthy, check whether the backend’s keep-alive timeout is shorter than the load balancer’s idle timeout. If it is, the backend closes connections the load balancer still thinks are open.

This post investigates random 502 Bad Gateway errors a monitoring job got. Even if it is written in relation to AWS, the solution can be generalized to every client-server-architecture.

For a monitoring job some 502 Bad Gateway errors have been logged during health checks:

INFO HttpCheckTask - Health check failed. Status code: 502 Url: http://internal-service/health

In the logs of the monitored service was nothing unusual in its logs around the same timestamps: no restarts and no errors. Whatever was returning the 502 wasn’t the application.

The problem

To dig deeper, the request flow should be verified:

Monitoring Client --> Load Balancer --> Target Backend

The Monitoring Client sends a GET /health request to the Load Balancer. The Load Balancer delegates the request to the Target Backend.

The Load Balancer was in this case an Application Load Balancer in AWS.

Note: Instead of the Load Balancer it could also be HAProxy or any comparable service.

As the application logs were clean, the access log of the load balancer was checked.

An example log of the failure looks like this:

http 2026-07-07T00:22:09.724313Z app/my-service/... 10.0.1.10:58052 10.0.2.20:8080 0.000 0.000 -1 502 - 203 277 "GET http://internal-service/health HTTP/1.1" ...

Some of the fields are truncated for better readability. The full structure of the log is documented as:

type time elb client:port target:port request_processing_time target_processing_time response_processing_time elb_status_code target_status_code received_bytes sent_bytes "request" "user_agent"

The AWS Application Load Balancer writes an elb_status_code and a target_status_code for every request. This is exactly the piece of information needed to figure out who’s responsible for the error:

The relevant fields, in order, are response_processing_time, elb_status_code and target_status_code. Fetching them from the log above their values are:

response_processing_time = -1
elb_status_code          = 502
target_status_code       = -

The AWS documentation states:

If the elb_status_code is 502 and the target_status_code is -, then your load balancer is the source of the HTTP 502 errors. If the elb_status_code is 502 and the target_status_code is also 502, then your target is the source of the errors.

(See the AWS troubleshooting guide for ALB 502 errors.)

The conclusion from the guide is that the Target Backend never got a chance to respond with anything. The Load Balancer produced the 502 itself. This explains why the application logs were clean.

Checking the root-cause: keep-alive vs. idle timeout

The two terms describe timeouts owned by opposite ends of the same connection:

• Keep-alive timeout is set by the Target Backend (which is the application server). It’s how long the backend will keep a connection open, waiting for another request, before it closes the socket.

• Idle timeout is set by the Load Balancer (or any proxy in front of the backend). It defines how long the load balancer will keep a connection open without traffic before it gives up on it. And also how long it will reuse a pooled connection to a backend without re-checking that the backend hasn’t already closed it.

Both define how long a connection can idle, but from different ends at the same TCP connection. That’s what makes a mismatch dangerous: the backend closing its timer doesn’t tell the load balancer anything. The load balancer only finds out when it tries to use the connection and it’s already gone.

See also the AWS troubleshooting guide:

The target closed the connection with a TCP RST or a TCP FIN while the load balancer had an outstanding request to the target. This usually occurs when the duration of the keep-alive timeout for the target is shorter than the idle timeout value of the load balancer.

A load balancer keeps a pool of connections open to its backends to avoid the overhead of a new TCP handshake per request.

If the Target Backend's keep-alive timeout is shorter (or equally long) as the Load Balancer's idle timeout, the backend can close a connection at almost the same moment the load balancer decides to reuse it for a new request.

The request goes out on a socket the backend has already torn down, and the load balancer has no way to know that in advance. This leads to 502 failures.

As this is a race condition, it can happen randomly and look like an occasional health-check issue instead of badly configured parameters.

Verifying it

If HTTPS is not enabled in the internal network netcat can be used to check and verify the actual timeouts values:

echo -e "GET /health HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\n\r\n" | nc example.com 8080

time nc example.com 8080

For SSL use:

time (printf "GET / HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\n\r\n" | openssl s_client -connect example.com:443 -quiet -ign_eof)

The connection should close after the expected value.

If it hangs open past that, the configured keep-alive value either isn’t applied or isn’t the one which is expected.

This can be seen as the baseline for testing it after doing changes.

The chain of timeouts

Like written above every hop between the requester and backend interacts with each other:

Monitoring Client --> Load Balancer --> Target Backend

To prevent the 502 Bad Gateway issues timeouts must increase as they go down to the backend.

This means: The Load Balancer needs an idle timeout which is smaller than the keep-alive timeout of the Target Backend and the timeout of the Monitoring Client should be also smaller.

If the Load Balancer has a smaller timeout than the Target Backend, it closes the connection first.

For example:

The fix

Raise the Target backend keep-alive timeout so that it’s larger than the Load Balancers idle timeout, for every hop in the chain.

In the AWS docs it’s stated as follow:

We also recommend that you configure the idle timeout of your application to be larger than the idle timeout configured for the load balancer. Otherwise, if the application closes the TCP connection to the load balancer ungracefully, the load balancer might send a request to the application before it receives the packet indicating that the connection is closed. If this is the case, then the load balancer sends an HTTP 502 Bad Gateway error to the client.

Most application servers expose have this as a configurable setting e.g. server.tomcat.keep-alive-timeout on Tomcat or keepalive_timeout on nginx.

It’s important to align here the Load Balancer timeout with the Target Backend.

When the fix is done, rerun the test mentioned above and verify that the timeouts are used as expected.

Conclusion

A 502 with a healthy-looking backend is a hint that the timeouts are wrongly configured.

This post showed how to investigate this issue and which solutions are available.

Check every hop between the client and the backend, check each one’s idle or keep-alive timeout and make sure the values increase the closer you get to the application.