- If TLS Handshake fails, it may not work and the web page is not displayed.
- There are two mobile phones, both of which use a SIM line and access a web page, one of them succeeds and one of them fails
- If successful, Server Hello, Change Cipher Spec, Encrypted Handshake Messge are sent together in one response. ACK is only once.
- If i can't see it completely, you suspected the firewall settings, but one is still visible.
- As seen from tcpdump (&wireshark), it seems that TCP retransmission occurs in the following cases, and as a result, the browser times out. (Excerpt from wireshark)
- At TCP Out-Of-Order, Wireshark shows (reassembly error, protocol TCP: New fragment overlaps old data).
|2||HAProxy||Internet||TCP||443->xxx SYN, ACK|
|4||HAProxy||HOST ||TCP||yyy->30397 SYN|
|5||HOST ||HAProxy||TCP||30397->yyy SYN, ACK|
|6||HAProxy||HOST ||TCP||yyy->30397 ACK|
|9||HAProxy||HOST ||TLS1.2||Client Hello|
|10||HOST ||HAProxy||TCP||30397->yyy ACK|
|11||HOST ||HAProxy||TLS1.2||Server Hello|
|12||HOST ||HAProxy||TLS1.2||Certificate (TCP segment of a reassembled PDU)|
|13||HOST ||HAProxy||TLS1.2||Server Key Exchange, Server Hello Done|
|14||HAProxy||HOST ||TCP||yyy->30397 ACK|
|15||HAProxy||HOST ||TCP||yyy->30397 ACK|
|16||HAProxy||HOST ||TCP||yyy->30397 ACK|
|18||HAProxy||Internet||TLS1.2||Certificate (TCP segment of a reassembled PDU)|
|19||HAProxy||Internet||TLS1.2||Server Key Exchange, Server Hello Done|
|20||HAProxy||Internet||TCP||[TCP Out-Of-Order] 443->xxx ACK|
|21||HAProxy||Internet||TCP||[TCP Out-Of-Order] 443->xxx ACK|
|22||HAProxy||Internet||TCP||[TCP Retransmission] 443->xxx ACK|
... Retransmission continuesQuestion
- Where is the cause?
- What else should I try
- What should I fix?
Currently, I am creating a verification environment by setting up a server at home.Kubernetes
- Works with Host 1-3
- Receiving HTTPS request at Port 30397 (provisional).
- Receiving HTTP request on Port 30402 (provisional).
- Issuing a wildcard certificate with NGINX ingress Controller&cert-manager (www, *)
- L7 balancing
- TCP Mode
- 80, 443 Listen
- 80->30402 (provisional) Forward
- 443->30397 (provisional) Forward
- Runs on Raspberry-pi with Docker-compose
- L4 balancing on HOST 1-3
- The router has a virtual server function, so 80, 443 Open
- 80->HAProxy (80) Forward
- 443->HAProxy (443) Forward
(Internet) ↕ [Router] ↕ [Raspberry Pi] ↕ ↕ ↕ [Host1] [Host2] [Host3]
- Raspberry Pi: HAProxy (TCP Mode)
- Host [1-3]: Kubernetes Worker (Master, but I'm not skipping it from Haproxy)
Although I am not familiar with the L3 level story and Network, I am not familiar with the mechanism of TCP and TLS.
Answer # 1
In such a case, it is quite difficult to isolate the situation with tcpdump, so it is often early to isolate the problem source by angry checking the operation in order from the minimum configuration.
Access from mobile 1 and 2 on one host (repeated by the number of hosts)
Access from mobile devices 1 and 2 with one host + HAproxy (repeated by the number of hosts)
Access from mobile 1 and 2 with host + HAProxy of 1 + 2
1 + 3 host + HAProxy access from mobile 1 and 2
Access from mobile 1 and 2 with 2 + 3 host + HAProxySince the pattern can be covered in about
, I think that it is better to simply check while checking the web server access log, error log, HAProxy access log, and error log on the host.Below, answer by intuition
Termination of HAProxy&HTTPS is done on the host side (interpreted as if it is different, please point out)
So I think that it is highly likely that the problem is between host or host-HAProxy.
HAProxy is StickyCookie and balancing to the same host for each client->Expect a case where a client accessing a bad host always fails.
If I verify,
Check the cookie by clearing the cookie on the PC browser and checking if the situation cannot be reproduced
Monitor the access log and packet on the host side, and check that the same host is accessed when accessing with mobile 1 and mobile 2 respectively.
If 1 or 2 is positive, review the nginx setting of the bad host&review the setting balancing to the bad host with HAProxy balancingFirst of all, check with
. . .
Answer # 2
Mtu size correction for WAN-LAN routerKeyword