Mastering Load Balancer and Proxy Troubleshooting Techniques

When troubleshooting proxies, load balancers, ADCs, and Layer 4 intermediaries, packet captures can reveal where a connection is failing and which side of the path is affected.
A useful troubleshooting sequence is:

1. Identify the symptom
2. Match the packet pattern
3. Determine the proxy or forwarding role
4. Check the affected side
5. Validate the likely cause
6. Confirm the fix with a new capture

The most important idea is:
Do not diagnose from one packet alone.
Always correlate:

• TCP handshake
• Sequence numbers
• ACK behavior
• Retransmissions
• TLS handshake
• RST source
• Window behavior
• Timing
• Proxy logs
• Backend health
• Routing

—

TCP Connection Establishment Problems

Start with the TCP three-way handshake.
A normal handshake is:
Client → SYN
Server/Proxy → SYN-ACK
Client → ACK
If this sequence does not complete, application troubleshooting should wait until the TCP problem is understood.

SYN Retransmissions — No Connection Established

A common pattern is:
Client → SYN
No SYN-ACK arrives.
The client retries:
Client → SYN
Client → SYN
This usually means the client did not receive a valid response to its SYN.
Example
Suppose a client tries to connect to:
10.10.10.100:443
Expected:
192.168.10.25 → 10.10.10.100 TCP SYN
10.10.10.100 → 192.168.10.25 TCP SYN-ACK
Instead, Wireshark shows:

192.168.10.25 → 10.10.10.100 TCP SYN
192.168.10.25 → 10.10.10.100 TCP SYN Retransmission
192.168.10.25 → 10.10.10.100 TCP SYN Retransmission

What It Means

The TCP handshake is not completing.
Possible causes include:

• ACL or firewall drop
• Incorrect route
• Incorrect VIP
• Service not listening
• Proxy unavailable
• Load balancer not accepting the connection
• SYN flood protection
• Return-path problem
• Backend issue in some L4 designs

Recommended Checks

Step 1 — Verify the destination
Confirm:

• Destination IP
• Destination port
• VIP
• DNS resolution

Step 2 — Check routing

ip route get 10.10.10.100

Step 3 — Check reachability

ping -c 4 10.10.10.100

Step 4 — Check the TCP port

nc -vz 10.10.10.100 443

Proxy-Specific Interpretation

Full Proxy
Ask:

• Did the SYN reach the proxy?
• Did the proxy reply with SYN-ACK?
• Is the client-side virtual service listening?
• Is the proxy dropping before the TCP handshake completes?

L4 / Flow-Oriented Device
Ask:

• Did the SYN reach the intermediary?
• Was it forwarded toward the selected backend?
• Did the backend reply?
• Did the return SYN-ACK follow the correct path?

SYN-ACK Seen but Final ACK Missing

Another pattern is:
Client → SYN
Proxy/Server → SYN-ACK
No final ACK arrives.

What It Means

The responder transmitted a SYN-ACK, but the final client ACK was not observed.
Possible causes include:

• Return traffic never reached the client
• Client-side firewall
• Asymmetric routing
• NAT issue
• Client TCP stack problem
• Packet capture missing the ACK

—

Connection Established but Application Does Not Progress

Sometimes the TCP handshake completes successfully, but no meaningful application data follows.

Handshake Completes but No Payload

Normal TCP establishment is visible:
SYN → SYN-ACK → ACK
but then the session becomes idle.

Possible Causes

• Client application waiting
• TLS handshake not initiated
• Server application stalled
• Proxy waiting on backend
• Backend connection failure
• Authentication delay
• Application timeout

Step-by-Step Check

Step 1 — Confirm TCP is established
Look for the complete handshake.
Step 2 — Check for client payload
Does the client send:

• TLS ClientHello?
• HTTP request?
• Application protocol data?

Step 3 — Check the opposite direction
Does the proxy or server respond?
Step 4 — Check timing
Large pauses can indicate:

• Backend delay
• Application processing
• DNS dependency
• Authentication
• Proxy queueing

—

TLS and SSL Troubleshooting

For HTTPS applications, a healthy TCP handshake does not guarantee a healthy TLS handshake.
A simplified TLS flow is:
ClientHello → ServerHello → Certificate → Key Exchange / Finished
The exact message sequence depends on the TLS version and configuration.

Repeated TLS ClientHello

A common symptom is the client sending a ClientHello repeatedly or reconnecting and sending new ClientHello messages.

What It May Indicate

Possible causes include:

• No response from TLS terminator
• TLS version incompatibility
• Cipher mismatch
• Certificate or trust problem
• mTLS configuration problem
• Proxy TLS inspection issue
• Resource exhaustion
• Backend TLS failure
• Middlebox interference

Recommended TLS Checks

Step 1 — Identify where TLS terminates
Determine whether TLS terminates at:

• Full Proxy
• Reverse proxy
• WAF
• Backend server
• Another security device

Step 2 — Inspect the ClientHello
Look for:

• TLS versions
• SNI
• Cipher suites
• Extensions
• ALPN

Step 3 — Look for ServerHello
If no ServerHello is returned, investigate the TLS endpoint.
Step 4 — Test with OpenSSL

openssl s_client -connect example.com:443 -servername example.com

Example — SNI Problem

Suppose the proxy hosts several HTTPS applications on one VIP.
The client connects to the correct IP but sends the wrong SNI hostname.
Possible result:

• Wrong certificate
• TLS alert
• Default virtual host
• Connection termination

ServerHello Never Returns

If ClientHello reaches the TLS endpoint but ServerHello does not return, check:

• TLS profile
• Supported TLS versions
• Cipher overlap
• Certificate configuration
• Private key
• Proxy CPU/resource condition
• TLS connection limits
• Backend dependency
• mTLS requirements

—

Retransmissions, Duplicate ACKs and Flow Control

Retransmissions are among the most useful packet indicators, but they must be interpreted according to the capture point and proxy architecture.

Client-Side Retransmissions

Suppose a Full Proxy has two independently managed TCP connections:
Client ↔ Proxy
and
Proxy ↔ Backend
If retransmissions appear only on the client-side connection, the problem may be isolated to the client-facing path.
Possible causes include:

• Client WAN loss
• Wi-Fi loss
• Internet congestion
• Client-side path issue
• Proxy-to-client return-path problem

Server-Side Retransmissions

If retransmissions appear only between the proxy and backend, investigate:

• Data-center network
• Backend interface
• Backend congestion
• Server overload
• Server-side firewall
• Proxy backend interface

Why Full Proxy Helps Isolation

A classic Full Proxy separates TCP state.
Therefore, a loss event on one side does not necessarily appear as the exact same TCP sequence/retransmission event on the other side.

Duplicate ACKs

Duplicate ACKs commonly appear when the receiver has received later TCP data but is still waiting for missing sequence data.
A typical sequence is:

1. Segment is lost.
2. Later segments arrive.
3. Receiver continues acknowledging the last contiguous sequence.
4. Duplicate ACKs are sent.
5. Sender may perform Fast Retransmission.

What to Check

• Which direction contains the duplicate ACKs?
• Which sequence number is missing?
• Is SACK present?
• Does a retransmission follow?
• Does throughput recover afterward?

Zero Window

A TCP Zero Window means a receiver is advertising that it currently has no receive-window space available.
This is flow control.
Possible causes include:

• Application not reading data fast enough
• Receive buffer pressure
• Proxy process slowdown
• Backend application slowdown
• Resource exhaustion

Example
Suppose the proxy sends:

TCP Window: 0

The sender must pause normal data transmission until the receive window opens again.

Recommended Checks

• Which device advertised Zero Window?
• Is the proxy or server CPU high?
• Is application processing slow?
• Are buffers exhausted?
• Are large responses accumulating?
• Is the client consuming data slowly?

—

Connection Resets, FIN and Timeouts

TCP RST packets are important because they indicate an abrupt connection termination.

Proxy Sends RST to Client

If the proxy sends the RST, investigate the proxy-side reason.
Possible causes include:

• Idle timeout
• Policy enforcement
• Invalid protocol
• Backend unavailable
• Connection limit
• TLS failure
• WAF action
• Administrative reset
• Failover/session-state issue

Step-by-Step Check

Step 1 — Identify the RST source IP
Determine which device actually sent it.
Step 2 — Check the packet immediately before the RST
Look for:

• TLS alert
• HTTP error
• Timeout gap
• Retransmissions
• Zero Window
• Backend failure

Step 3 — Correlate proxy logs
Check:

• Virtual service logs
• WAF logs
• TLS logs
• Backend health
• Connection timeout

Backend Sends RST to Proxy

Possible causes include:

• Application reset
• Backend service restart
• Server-side timeout
• Server resource issue
• Local firewall
• Application rejection

FIN vs RST

FIN
Usually represents an orderly TCP close.
A common graceful close includes:
FIN → ACK → FIN → ACK
RST
Represents an abrupt reset.
Possible reasons include:

• Application error
• Policy enforcement
• Invalid state
• Timeout
• Port closed
• Resource issue

Idle Timeout

Proxies often maintain idle connection timers.
If the application remains inactive longer than the configured timeout, the proxy may close the session.
Possible symptoms:

• Connection works initially
• Session remains idle
• Next application request fails
• RST or new TCP handshake appears

Example
Application idle period:
600 seconds
Proxy idle timeout:
300 seconds
The proxy may close the connection before the application expects it to remain available.

Recommended Checks

Compare:

• Proxy idle timeout
• Application keepalive interval
• TCP keepalive
• Backend timeout
• Client timeout
• Load balancer connection timeout

—

Identifying Proxy Behavior from Packet Captures

Packet captures can provide clues about proxy architecture, but avoid using a single field as absolute proof.

Classic Full Proxy Indicators

Possible indicators include:

• Separate client-side and server-side TCP connections
• Different TCP sequence spaces
• Different source/destination socket pairs
• Proxy independently ACKing client data
• Independent client-side and backend-side retransmissions
• Different MSS/window behavior
• TLS terminating at the proxy

Flow-Oriented / L4 Indicators

Possible indicators include:

• More direct preservation of original flow characteristics
• Traffic forwarded based on connection state
• No application-layer termination
• TLS remains end-to-end
• TCP recovery behavior follows the forwarded flow

Best Verification Method

Use simultaneous captures at:

1. Client side
2. Proxy ingress
3. Proxy egress
4. Backend side

Then compare:

• 5-tuple
• Sequence numbers
• ACKs
• MSS
• Window size
• TLS endpoints
• Retransmissions
• Timing

This gives much stronger evidence than inspecting one capture location.
—

Quick Symptom-to-Root-Cause Guide

—

Golden Troubleshooting Workflow

Use this order to avoid jumping randomly through packets.

Step 1 — Check TCP Handshake

Confirm:
SYN → SYN-ACK → ACK
If this fails, solve the TCP connectivity problem first.

Step 2 — Check TLS Handshake

For HTTPS, determine whether:
ClientHello → ServerHello
progresses normally.

Step 3 — Check Application Data

Look for:

• HTTP request
• HTTP response
• Application protocol request
• Backend delay

Step 4 — Check Retransmission Scope

Determine whether loss appears on:

• Client side
• Server side
• Both sides
• One specific network segment

Step 5 — Identify RST Source

Determine exactly which endpoint reset the session.

Step 6 — Check Window and Flow Control

Look for:

• Zero Window
• Window Full
• Persist behavior
• Slow receiver

Step 7 — Check Timers

Compare:

• Client timeout
• Proxy idle timeout
• Backend timeout
• Application timeout
• TCP keepalive

Step 8 — Check Proxy Resources

Review:

• CPU
• Memory
• Connection count
• TLS TPS
• Buffer usage
• Backend health
• Error counters

Step 9 — Compare Multiple Capture Points

Capture on both sides of the suspected device.

Step 10 — Verify the Fix

After changing configuration:

1. Reproduce the test.
2. Capture again.
3. Compare packet behavior.
4. Confirm the user symptom is resolved.

—

Summary

Proxy troubleshooting becomes much easier when TCP behavior is analyzed in a consistent order.
Start with:
TCP handshake
Then check:
TLS handshake
Then:
Application traffic
After that, correlate:

• Retransmissions
• Duplicate ACKs
• Window behavior
• RST source
• Timeouts
• Proxy resources
• Backend health

The most important lessons are:

• Repeated SYN packets indicate an incomplete TCP handshake.
• A successful TCP handshake does not prove TLS or the application is healthy.
• Client-side and server-side retransmissions can help isolate loss in a Full Proxy architecture.
• Duplicate ACKs indicate TCP sequence recovery behavior but do not automatically identify the physical loss point.
• Zero Window indicates receiver-side flow control, not packet loss.
• Always identify who sent a RST before assigning root cause.
• FIN normally indicates an orderly close; RST indicates abrupt termination.
• Idle timeout mismatches are common in long-lived applications.
• Proxy architecture should be inferred using multiple indicators rather than a single packet field.
• Multi-point captures provide much stronger evidence than a single trace.

—

Useful Links and References

SanchitGurukul Static Resources

Official External References