The curl command is not working correctly
-
Hello,
I have made the following configuration and am routing traffic through the GRE tunnel.
Traffic seems to be flowing correctly, but when I use the curl command to download files, the local pfsense sometimes the webgui slows down or crashes and curl doesn't work correctly. Sometimes it works well, sometimes it doesn't. Pfsense has 8 vCores and 8 Gb of ram and when analyzing the graphics it doesn't seem to be a problem with either cpu or ram.
As we can see, when executing the following command, curl became "choked" and does not move forward.
sudo curl -L https://github.com/pelican-dev/panel/releases/latest/download/panel.tar.gz | sudo tar -xzv
-
Where is the 185.113.141.217 client you are testing from? Behind the 'local' pfSense?
Are you NATing traffic anywhere? Outbound? Port forwards?
Do you have VIPs on the remote pfSense?
This looks almost identical to a problem posted a few months back and the issues are the same. That public subnet exists on the remote pfSense WAN so you can't use it anywhere else.
Steve
Edit: Still the same issue you were seeing here?
And here?
-
Yes, 185.113.141.217 is from behind the local pfsense.
pfSense Remote Settings:
pfSense Local Settings:
No, here the problem is that the network does not work.
Now the network is working "almost normally", but sometimes the network seems to be slow. When downloading the same file, sometimes it is quick, other times it takes some time.
In local pfsense I am using a lan simulated network of the real network and I am only routing traffic from hosts to the real network.
-
@s_serra Im having the same trouble and dont really know where the problem is comming from... Hope @stephenw10 can give us a hand
-
The network traffic seems to be ok but there seems to be something slowing down the network.
Server doing traceroute to 1.1.1.1:
My pc doing traceroute to 185.113.141.217:
I installed apache2 on the server http://185.113.141.217/ and sometimes the page loads slowly and other times it doesn't even load.
-
Mmm, OK. There is quite a lot of discussion on the other threads about this. You are trying to make a single layer2 subnet work across a layer3 routed network and that shouldn't work. The fact it works at all is surprising.
Do you see traffic blocked in the firewall log when trying to make a TCP connection?
-
In the remote pfsense I disabled the firewall completely in the settings, leaving only the virtual ips and static routes.
In the local pfsense I see Default deny rule IPv4 (1000000103) with the protocols TCP:SA and TCP:A I believe that this is not what is making the network "slow".
You are trying to make a single layer2 subnet work across a layer3 routed network and that shouldn't work.
I didn't quite understand what you said, is the GRE tunnel not suitable for routing traffic?
Thank you for your willingness to answer my questions.
-
Yes a GRE tunnel is suitable for routing traffic (layer 3). But you should be bridging this (layer 2) because you have IPs from the same subnet at both ends of the tunnel.
If those blocked SYN-ACKs are from servers you are trying to connect to then they are being blocked as out of state. That could be because the state timed out or because there's some asymmetry.
-
I ran the curl ifconfig.me command on host 185.113.141.217 and waited to receive the response and did a pcap on the GRE interface on both pfsenses.
pfLocal:
12:16:10.937750 IP (tos 0x0, ttl 64, id 11639, offset 0, flags [DF], proto TCP (6), length 60) 185.113.141.217.55370 > 34.160.111.145.80: Flags [S], cksum 0xb8d6 (correct), seq 3855994060, win 32120, options [mss 1460,sackOK,TS val 3802647119 ecr 0,nop,wscale 7], length 0 12:16:10.956111 IP (tos 0x0, ttl 123, id 0, offset 0, flags [DF], proto TCP (6), length 60) 34.160.111.145.80 > 185.113.141.217.55370: Flags [S.], cksum 0xdbfa (correct), seq 727513950, ack 3855994061, win 65535, options [mss 1412,sackOK,TS val 1821034284 ecr 3802647119,nop,wscale 8], length 0 12:16:10.956313 IP (tos 0x0, ttl 64, id 11640, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.55370 > 34.160.111.145.80: Flags [.], cksum 0x098b (correct), ack 1, win 251, options [nop,nop,TS val 3802647137 ecr 1821034284], length 0 12:16:10.956381 IP (tos 0x0, ttl 64, id 11641, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0x9a39 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802647137 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:16:11.180755 IP (tos 0x0, ttl 64, id 11642, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0x9958 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802647362 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:16:11.404858 IP (tos 0x0, ttl 64, id 11643, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0x9878 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802647586 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:16:11.844862 IP (tos 0x0, ttl 64, id 11644, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0x96c0 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802648026 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:16:12.732746 IP (tos 0x0, ttl 64, id 11645, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0x9348 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802648914 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:16:13.314007 IP (tos 0x28, ttl 246, id 12051, offset 0, flags [none], proto TCP (6), length 44) 193.163.125.234.42661 > 185.113.141.217.448: Flags [S], cksum 0x0105 (correct), seq 414324265, win 14600, options [mss 1460], length 0 12:16:13.314354 IP (tos 0x28, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.448 > 193.163.125.234.42661: Flags [R.], cksum 0x51b6 (correct), seq 0, ack 414324266, win 0, length 0 12:16:14.524883 IP (tos 0x0, ttl 64, id 11646, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0x8c48 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802650706 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:16:18.044837 IP (tos 0x0, ttl 64, id 11647, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0x7e88 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802654226 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:16:25.468775 IP (tos 0x0, ttl 64, id 11648, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0x6188 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802661650 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:16:39.560884 IP (tos 0x2a,ECT(0), ttl 120, id 10434, offset 0, flags [DF], proto TCP (6), length 52) 193.201.9.156.42259 > 185.113.141.217.22: Flags [SEW], cksum 0x0b6b (correct), seq 426471840, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0 12:16:39.561145 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.22 > 193.201.9.156.42259: Flags [S.], cksum 0xe578 (correct), seq 3329229497, ack 426471841, win 32120, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0 12:16:39.643100 IP (tos 0x28, ttl 120, id 10435, offset 0, flags [DF], proto TCP (6), length 40) 193.201.9.156.42259 > 185.113.141.217.22: Flags [.], cksum 0xa2bf (correct), ack 1, win 260, length 0 12:16:39.656640 IP (tos 0x0, ttl 63, id 33073, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:16:39.804859 IP (tos 0x0, ttl 64, id 11649, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0x2988 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802675986 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:16:39.940682 IP (tos 0x0, ttl 63, id 33074, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:16:40.228757 IP (tos 0x0, ttl 63, id 33075, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:16:40.828629 IP (tos 0x0, ttl 63, id 33076, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:16:41.411031 IP (tos 0x0, ttl 249, id 54321, offset 0, flags [none], proto TCP (6), length 44) 185.224.128.83.42362 > 185.113.141.217.4719: Flags [S], cksum 0x5eb8 (correct), seq 3641648886, win 65535, options [mss 1460], length 0 12:16:41.411451 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.4719 > 185.224.128.83.42362: Flags [R.], cksum 0x7661 (correct), seq 0, ack 3641648887, win 0, length 0 12:16:41.980740 IP (tos 0x0, ttl 63, id 33077, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:16:44.284696 IP (tos 0x0, ttl 63, id 33078, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:16:49.020693 IP (tos 0x0, ttl 63, id 33079, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:16:54.648261 IP (tos 0x28, ttl 120, id 10446, offset 0, flags [DF], proto TCP (6), length 40) 193.201.9.156.42259 > 185.113.141.217.22: Flags [R.], cksum 0x9f7b (correct), seq 1053, ack 41, win 0, length 0 12:16:58.236738 IP (tos 0x0, ttl 63, id 33080, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:17:07.964840 IP (tos 0x0, ttl 64, id 11650, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0xbb87 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802704146 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:17:07.984675 IP (tos 0x0, ttl 123, id 3227, offset 0, flags [none], proto TCP (6), length 52) 34.160.111.145.80 > 185.113.141.217.55370: Flags [.], cksum 0x4bc4 (correct), ack 76, win 256, options [nop,nop,TS val 1821091312 ecr 3802704146], length 0 12:17:08.249803 IP (tos 0x0, ttl 241, id 54321, offset 0, flags [none], proto TCP (6), length 44) 194.165.17.13.39416 > 185.113.141.217.8081: Flags [S], cksum 0x5b0b (correct), seq 2762851302, win 65535, options [mss 1460], length 0 12:17:08.250084 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.8081 > 194.165.17.13.39416: Flags [R.], cksum 0x72b4 (correct), seq 0, ack 2762851303, win 0, length 0 12:17:12.328637 IP (tos 0x28, ttl 235, id 54321, offset 0, flags [none], proto TCP (6), length 44) 172.169.109.202.44786 > 185.113.141.217.3001: Flags [S], cksum 0x88ef (correct), seq 1886290526, win 65535, options [mss 1460], length 0 12:17:12.328959 IP (tos 0x28, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.3001 > 172.169.109.202.44786: Flags [R.], cksum 0xa098 (correct), seq 0, ack 1886290527, win 0, length 0 12:17:16.668717 IP (tos 0x0, ttl 63, id 33081, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:17:18.146861 IP (tos 0x0, ttl 248, id 53539, offset 0, flags [DF], proto TCP (6), length 44) 185.113.172.205.23971 > 185.113.141.217.23: Flags [S], cksum 0x68df (correct), seq 1277140699, win 14600, options [mss 1460], length 0 12:17:18.147185 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.23 > 185.113.172.205.23971: Flags [R.], cksum 0xb990 (correct), seq 0, ack 1277140700, win 0, length 0 (.... character limit) 12:18:05.207962 IP (tos 0x0, ttl 124, id 3237, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0x4658 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821148535 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:18:05.208348 IP (tos 0x0, ttl 64, id 11651, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.55370 > 34.160.111.145.80: Flags [.], cksum 0x8c00 (correct), ack 167, win 250, options [nop,nop,TS val 3802761389 ecr 1821148535], length 0 12:18:05.208619 IP (tos 0x0, ttl 64, id 11652, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.55370 > 34.160.111.145.80: Flags [F.], cksum 0x8bfe (correct), seq 76, ack 167, win 250, options [nop,nop,TS val 3802761390 ecr 1821148535], length 0 12:18:05.230896 IP (tos 0x0, ttl 124, id 3238, offset 0, flags [none], proto TCP (6), length 52) 34.160.111.145.80 > 185.113.141.217.55370: Flags [F.], cksum 0x8be0 (correct), seq 167, ack 77, win 256, options [nop,nop,TS val 1821148558 ecr 3802761390], length 0 12:18:05.231231 IP (tos 0x0, ttl 64, id 11653, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.55370 > 34.160.111.145.80: Flags [.], cksum 0x8bd0 (correct), ack 168, win 250, options [nop,nop,TS val 3802761412 ecr 1821148558], length 0
pfRemote:
12:16:11.163031 IP (tos 0x0, ttl 64, id 11639, offset 0, flags [DF], proto TCP (6), length 60) 185.113.141.217.55370 > 34.160.111.145.80: Flags [S], cksum 0xb8d6 (correct), seq 3855994060, win 32120, options [mss 1460,sackOK,TS val 3802647119 ecr 0,nop,wscale 7], length 0 12:16:11.172588 IP (tos 0x0, ttl 123, id 0, offset 0, flags [DF], proto TCP (6), length 60) 34.160.111.145.80 > 185.113.141.217.55370: Flags [S.], cksum 0xdbfa (correct), seq 727513950, ack 3855994061, win 65535, options [mss 1412,sackOK,TS val 1821034284 ecr 3802647119,nop,wscale 8], length 0 12:16:11.181410 IP (tos 0x0, ttl 64, id 11640, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.55370 > 34.160.111.145.80: Flags [.], cksum 0x098b (correct), ack 1, win 251, options [nop,nop,TS val 3802647137 ecr 1821034284], length 0 12:16:13.530387 IP (tos 0x28, ttl 246, id 12051, offset 0, flags [none], proto TCP (6), length 44) 193.163.125.234.42661 > 185.113.141.217.448: Flags [S], cksum 0x0105 (correct), seq 414324265, win 14600, options [mss 1460], length 0 12:16:13.539339 IP (tos 0x28, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.448 > 193.163.125.234.42661: Flags [R.], cksum 0x51b6 (correct), seq 0, ack 414324266, win 0, length 0 12:16:39.777421 IP (tos 0x2a,ECT(0), ttl 120, id 10434, offset 0, flags [DF], proto TCP (6), length 52) 193.201.9.156.42259 > 185.113.141.217.22: Flags [SEW], cksum 0x0b6b (correct), seq 426471840, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0 12:16:39.786202 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.22 > 193.201.9.156.42259: Flags [S.], cksum 0xe578 (correct), seq 3329229497, ack 426471841, win 32120, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0 12:16:39.859587 IP (tos 0x28, ttl 120, id 10435, offset 0, flags [DF], proto TCP (6), length 40) 193.201.9.156.42259 > 185.113.141.217.22: Flags [.], cksum 0xa2bf (correct), ack 1, win 260, length 0 12:16:39.861362 IP (tos 0x28, ttl 120, id 10436, offset 0, flags [DF], proto TCP (6), length 52) 193.201.9.156.42259 > 185.113.141.217.22: Flags [P.], cksum 0x5056 (correct), seq 1:13, ack 1, win 260, length 12: SSH: SSH-2.0-Go 12:16:39.882224 IP (tos 0x0, ttl 63, id 33073, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:16:39.955672 IP (tos 0x28, ttl 120, id 10437, offset 0, flags [DF], proto TCP (6), length 1080) 193.201.9.156.42259 > 185.113.141.217.22: Flags [P.], cksum 0xda66 (correct), seq 13:1053, ack 41, win 260, length 1040 12:16:40.202311 IP (tos 0x28, ttl 120, id 10438, offset 0, flags [DF], proto TCP (6), length 1092) 193.201.9.156.42259 > 185.113.141.217.22: Flags [P.], cksum 0x8811 (correct), seq 1:1053, ack 41, win 260, length 1052: SSH: SSH-2.0-Go 12:16:40.502997 IP (tos 0x28, ttl 120, id 10439, offset 0, flags [DF], proto TCP (6), length 1092) 193.201.9.156.42259 > 185.113.141.217.22: Flags [P.], cksum 0x8811 (correct), seq 1:1053, ack 41, win 260, length 1052: SSH: SSH-2.0-Go 12:16:41.203486 IP (tos 0x28, ttl 120, id 10440, offset 0, flags [DF], proto TCP (6), length 1092) 193.201.9.156.42259 > 185.113.141.217.22: Flags [P.], cksum 0x8811 (correct), seq 1:1053, ack 41, win 260, length 1052: SSH: SSH-2.0-Go 12:16:41.627427 IP (tos 0x0, ttl 249, id 54321, offset 0, flags [none], proto TCP (6), length 44) 185.224.128.83.42362 > 185.113.141.217.4719: Flags [S], cksum 0x5eb8 (correct), seq 3641648886, win 65535, options [mss 1460], length 0 12:16:41.636596 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.4719 > 185.224.128.83.42362: Flags [R.], cksum 0x7661 (correct), seq 0, ack 3641648887, win 0, length 0 12:16:42.503431 IP (tos 0x28, ttl 120, id 10441, offset 0, flags [none], proto TCP (6), length 576) 193.201.9.156.42259 > 185.113.141.217.22: Flags [P.], cksum 0x5f96 (correct), seq 1:537, ack 41, win 260, length 536: SSH: SSH-2.0-Go 12:16:43.803694 IP (tos 0x28, ttl 120, id 10442, offset 0, flags [none], proto TCP (6), length 576) 193.201.9.156.42259 > 185.113.141.217.22: Flags [P.], cksum 0x5f96 (correct), seq 1:537, ack 41, win 260, length 536: SSH: SSH-2.0-Go 12:16:45.112316 IP (tos 0x28, ttl 120, id 10443, offset 0, flags [DF], proto TCP (6), length 1092) 193.201.9.156.42259 > 185.113.141.217.22: Flags [P.], cksum 0x8811 (correct), seq 1:1053, ack 41, win 260, length 1052: SSH: SSH-2.0-Go 12:16:47.666899 IP (tos 0x28, ttl 120, id 10444, offset 0, flags [DF], proto TCP (6), length 1092) 193.201.9.156.42259 > 185.113.141.217.22: Flags [P.], cksum 0x8811 (correct), seq 1:1053, ack 41, win 260, length 1052: SSH: SSH-2.0-Go 12:16:52.498047 IP (tos 0x28, ttl 120, id 10445, offset 0, flags [DF], proto TCP (6), length 1092) 193.201.9.156.42259 > 185.113.141.217.22: Flags [P.], cksum 0x8811 (correct), seq 1:1053, ack 41, win 260, length 1052: SSH: SSH-2.0-Go 12:16:54.862707 IP (tos 0x28, ttl 120, id 10446, offset 0, flags [DF], proto TCP (6), length 40) 193.201.9.156.42259 > 185.113.141.217.22: Flags [R.], cksum 0x9f7b (correct), seq 1053, ack 41, win 0, length 0 12:16:58.465310 IP (tos 0x0, ttl 63, id 33080, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 193.201.9.156.42259: Flags [P.], cksum 0x3dc7 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:17:08.191361 IP (tos 0x0, ttl 64, id 11650, offset 0, flags [DF], proto TCP (6), length 127) 185.113.141.217.55370 > 34.160.111.145.80: Flags [P.], cksum 0xbb87 (correct), seq 1:76, ack 1, win 251, options [nop,nop,TS val 3802704146 ecr 1821034284], length 75: HTTP, length: 75 GET / HTTP/1.1 Host: ifconfig.me User-Agent: curl/7.88.1 Accept: */* 12:17:08.200880 IP (tos 0x0, ttl 123, id 3227, offset 0, flags [none], proto TCP (6), length 52) 34.160.111.145.80 > 185.113.141.217.55370: Flags [.], cksum 0x4bc4 (correct), ack 76, win 256, options [nop,nop,TS val 1821091312 ecr 3802704146], length 0 12:17:08.305527 IP (tos 0x0, ttl 124, id 3228, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0x2578 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821091416 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:17:08.466047 IP (tos 0x0, ttl 241, id 54321, offset 0, flags [none], proto TCP (6), length 44) 194.165.17.13.39416 > 185.113.141.217.8081: Flags [S], cksum 0x5b0b (correct), seq 2762851302, win 65535, options [mss 1460], length 0 12:17:08.475626 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.8081 > 194.165.17.13.39416: Flags [R.], cksum 0x72b4 (correct), seq 0, ack 2762851303, win 0, length 0 12:17:08.529572 IP (tos 0x0, ttl 124, id 3229, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0x2498 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821091640 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:17:08.752096 IP (tos 0x0, ttl 124, id 3230, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0x23b8 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821091864 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:17:09.193495 IP (tos 0x0, ttl 124, id 3231, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0x2200 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821092304 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:17:10.129861 IP (tos 0x0, ttl 124, id 3232, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0x1e58 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821093240 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:17:11.919110 IP (tos 0x0, ttl 124, id 3233, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0x1759 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821095031 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:17:12.545103 IP (tos 0x28, ttl 235, id 54321, offset 0, flags [none], proto TCP (6), length 44) 172.169.109.202.44786 > 185.113.141.217.3001: Flags [S], cksum 0x88ef (correct), seq 1886290526, win 65535, options [mss 1460], length 0 12:17:12.554043 IP (tos 0x28, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.3001 > 172.169.109.202.44786: Flags [R.], cksum 0xa098 (correct), seq 0, ack 1886290527, win 0, length 0 12:17:15.439113 IP (tos 0x0, ttl 124, id 3234, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0x0999 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821098551 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:17:18.363108 IP (tos 0x0, ttl 248, id 53539, offset 0, flags [DF], proto TCP (6), length 44) 185.113.172.205.23971 > 185.113.141.217.23: Flags [S], cksum 0x68df (correct), seq 1277140699, win 14600, options [mss 1460], length 0 12:17:18.372330 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.23 > 185.113.172.205.23971: Flags [R.], cksum 0xb990 (correct), seq 0, ack 1277140700, win 0, length 0 12:17:22.928396 IP (tos 0x0, ttl 124, id 3235, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0xec57 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821106040 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:17:26.647722 IP (tos 0x28, ttl 248, id 54321, offset 0, flags [none], proto TCP (6), length 44) 45.145.42.234.58282 > 185.113.141.217.8080: Flags [S], cksum 0x9f2e (correct), seq 1529281232, win 65535, options [mss 1460], length 0 12:17:26.656490 IP (tos 0x28, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.8080 > 45.145.42.234.58282: Flags [R.], cksum 0xb6d7 (correct), seq 0, ack 1529281233, win 0, length 0 12:17:29.788925 IP (tos 0x28, ttl 40, id 62464, offset 0, flags [none], proto TCP (6), length 44) 203.223.51.162.21679 > 185.113.141.217.23: Flags [S], cksum 0x985a (correct), seq 3111226841, win 7406, options [mss 1460], length 0 12:17:29.798509 IP (tos 0x28, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.23 > 203.223.51.162.21679: Flags [R.], cksum 0xccf1 (correct), seq 0, ack 3111226842, win 0, length 0 12:17:30.213813 IP (tos 0x28, ttl 47, id 59210, offset 0, flags [none], proto TCP (6), length 44) 211.194.231.72.44223 > 185.113.141.217.22: Flags [S], cksum 0x7812 (correct), seq 1369511709, win 3014, options [mss 536], length 0 12:17:30.222903 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 44) 185.113.141.217.22 > 211.194.231.72.44223: Flags [S.], cksum 0x6df9 (correct), seq 4154367258, ack 1369511710, win 32120, options [mss 1460], length 0 12:17:30.505421 IP (tos 0x28, ttl 47, id 0, offset 0, flags [DF], proto TCP (6), length 40) 211.194.231.72.44223 > 185.113.141.217.22: Flags [R], cksum 0x97f5 (correct), seq 1369511710, win 0, length 0 12:17:30.540339 IP (tos 0x28, ttl 47, id 33754, offset 0, flags [DF], proto TCP (6), length 52) 211.194.231.72.63915 > 185.113.141.217.22: Flags [S], cksum 0xd9a4 (correct), seq 1931668778, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0 12:17:30.549036 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.22 > 211.194.231.72.63915: Flags [S.], cksum 0x5e77 (correct), seq 232532175, ack 1931668779, win 32120, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0 12:17:30.825784 IP (tos 0x28, ttl 47, id 33755, offset 0, flags [DF], proto TCP (6), length 40) 211.194.231.72.63915 > 185.113.141.217.22: Flags [.], cksum 0x1bdd (correct), ack 1, win 229, length 0 12:17:30.848773 IP (tos 0x0, ttl 63, id 24759, offset 0, flags [DF], proto TCP (6), length 80) 185.113.141.217.22 > 211.194.231.72.63915: Flags [P.], cksum 0xb6c5 (correct), seq 1:41, ack 1, win 251, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:17:31.125966 IP (tos 0x28, ttl 47, id 33756, offset 0, flags [DF], proto TCP (6), length 40) 211.194.231.72.63915 > 185.113.141.217.22: Flags [.], cksum 0x1bb5 (correct), ack 41, win 229, length 0 12:17:31.229436 IP (tos 0x28, ttl 47, id 33757, offset 0, flags [DF], proto TCP (6), length 80) 211.194.231.72.63915 > 185.113.141.217.22: Flags [P.], cksum 0xb6b3 (correct), seq 1:41, ack 41, win 229, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:17:34.230439 IP (tos 0x28, ttl 47, id 33758, offset 0, flags [DF], proto TCP (6), length 80) 211.194.231.72.63915 > 185.113.141.217.22: Flags [P.], cksum 0xb6b3 (correct), seq 1:41, ack 41, win 229, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:17:35.818941 IP (tos 0x28, ttl 246, id 27645, offset 0, flags [none], proto TCP (6), length 44) 193.163.125.221.49596 > 185.113.141.217.4096: Flags [S], cksum 0xbf29 (correct), seq 2551557974, win 14600, options [mss 1460], length 0 12:17:35.830934 IP (tos 0x28, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.4096 > 193.163.125.221.49596: Flags [R.], cksum 0x0fdb (correct), seq 0, ack 2551557975, win 0, length 0 12:17:37.264637 IP (tos 0x0, ttl 124, id 3236, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0xb458 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821120375 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:17:38.042463 IP (tos 0x0, ttl 113, id 2004, offset 0, flags [DF], proto TCP (6), length 52) 103.157.94.5.61598 > 185.113.141.217.445: Flags [S], cksum 0xfca0 (correct), seq 2331625522, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 12:17:38.054693 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.445 > 103.157.94.5.61598: Flags [R.], cksum 0x5d5a (correct), seq 0, ack 2331625523, win 0, length 0 12:17:38.774860 IP (tos 0x0, ttl 113, id 2036, offset 0, flags [DF], proto TCP (6), length 52) 103.157.94.5.61598 > 185.113.141.217.445: Flags [S], cksum 0xfca0 (correct), seq 2331625522, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 12:17:38.783671 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.445 > 103.157.94.5.61598: Flags [R.], cksum 0x5d5a (correct), seq 0, ack 1, win 0, length 0 12:17:40.250868 IP (tos 0x28, ttl 47, id 33759, offset 0, flags [DF], proto TCP (6), length 80) 211.194.231.72.63915 > 185.113.141.217.22: Flags [P.], cksum 0xb6b3 (correct), seq 1:41, ack 41, win 229, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:17:44.594731 IP (tos 0x0, ttl 52, id 1994, offset 0, flags [none], proto TCP (6), length 60) 167.94.145.90.4529 > 185.113.141.217.18812: Flags [S], cksum 0x60b1 (correct), seq 530888548, win 42340, options [mss 1460,sackOK,TS val 1722951931 ecr 0,nop,wscale 10], length 0 12:17:44.603838 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.18812 > 167.94.145.90.4529: Flags [R.], cksum 0xfd95 (correct), seq 0, ack 530888549, win 0, length 0 12:17:51.461048 IP (tos 0x0, ttl 247, id 17872, offset 0, flags [none], proto TCP (6), length 44) 193.163.125.247.59346 > 185.113.141.217.1035: Flags [S], cksum 0x947d (correct), seq 1695019733, win 14600, options [mss 1460], length 0 12:17:51.469886 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40) 185.113.141.217.1035 > 193.163.125.247.59346: Flags [R.], cksum 0xe52e (correct), seq 0, ack 1695019734, win 0, length 0 12:17:52.290516 IP (tos 0x28, ttl 47, id 33760, offset 0, flags [DF], proto TCP (6), length 80) 211.194.231.72.63915 > 185.113.141.217.22: Flags [P.], cksum 0xb6b3 (correct), seq 1:41, ack 41, win 229, length 40: SSH: SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3 12:17:57.722352 IP (tos 0x28, ttl 47, id 33761, offset 0, flags [DF], proto TCP (6), length 352) 211.194.231.72.63915 > 185.113.141.217.22: Flags [FP.], cksum 0xe587 (correct), seq 41:353, ack 41, win 229, length 312 12:17:57.740082 IP (tos 0x0, ttl 63, id 24765, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.22 > 211.194.231.72.63915: Flags [.], cksum 0x2362 (correct), ack 1, win 251, options [nop,nop,sack 1 {41:354}], length 0 12:18:05.424504 IP (tos 0x0, ttl 124, id 3237, offset 0, flags [none], proto TCP (6), length 218) 34.160.111.145.80 > 185.113.141.217.55370: Flags [P.], cksum 0x4658 (correct), seq 1:167, ack 76, win 256, options [nop,nop,TS val 1821148535 ecr 3802704146], length 166: HTTP, length: 166 HTTP/1.1 200 OK date: Thu, 08 Aug 2024 12:17:08 GMT content-type: text/plain Content-Length: 15 access-control-allow-origin: * via: 1.1 google 185.113.141.217 [|http] 12:18:05.433606 IP (tos 0x0, ttl 64, id 11651, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.55370 > 34.160.111.145.80: Flags [.], cksum 0x8c00 (correct), ack 167, win 250, options [nop,nop,TS val 3802761389 ecr 1821148535], length 0 12:18:05.436211 IP (tos 0x0, ttl 64, id 11652, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.55370 > 34.160.111.145.80: Flags [F.], cksum 0x8bfe (correct), seq 76, ack 167, win 250, options [nop,nop,TS val 3802761390 ecr 1821148535], length 0 12:18:05.447577 IP (tos 0x0, ttl 124, id 3238, offset 0, flags [none], proto TCP (6), length 52) 34.160.111.145.80 > 185.113.141.217.55370: Flags [F.], cksum 0x8be0 (correct), seq 167, ack 77, win 256, options [nop,nop,TS val 1821148558 ecr 3802761390], length 0 12:18:05.456507 IP (tos 0x0, ttl 64, id 11653, offset 0, flags [DF], proto TCP (6), length 52) 185.113.141.217.55370 > 34.160.111.145.80: Flags [.], cksum 0x8bd0 (correct), ack 168, win 250, options [nop,nop,TS val 3802761412 ecr 1821148558], length 0
The information seems to take a long time to reach each end of the tunnel.
What can I use for layer 2 to make this work correctly?
Thank you for your availability once again. -
The best option here is if the provider routes a subnet to you via some other IP address. Then you could just route that over the GRE correctly. However I think we discussed that and it's not an option?
Otherwise I would bridge the connection using OpenVPN TAP at both ends. The you will have a single layer 2 between both sites and all public IPs can reach the ISP gateway directly.
-
I have already configured OpenVPN in tap mode. Now I don't know what to do xD. On the LAN interface on the local pfsense side, can I put the ip address 185.113.141.1/24? I don't know if it will make sense.
Thanks
-
I created a bridge between the WAN and OpenVPN on the remote side and a bridge between the LAN (I removed the LAN IP) and OpenVPN on the local side. Everything seems to be working perfectly.
A big thank you for the help.
-
Nice! Yup bridges are needed at both ends as you added.
-
@stephenw10 Hello! I did the same setup as @s_serra and for some reason with that config my network is pretty slow, i usually have 200 download and now went to 50, any idea why?
-
How are you testing? From where? What WAN bandwidths do you have at each end of the tunnel?
-
@stephenw10 On each pf wan I allways have more than 500/500 and i executed an iperf of the vm behind the pf to the local pf and got around 3gbps
-
What latency do you have over the tunnel?
Try an iperf test between the two pfSense instances directly. Try to determine where the throttling is actually happening.
-
@stephenw10
Iperf between both pf's without going through the tunnel:
Local Pf logs (This pf is on a vm inside the proxmox)
SpeedTest on a VM with the tunnel working:
-
Do you see the same results in both directions?
That's a lot of variation in the result, even outside the tunnel.
How much traffic is running through that local pfSense? How much RAM does it have?
You can increase the state table size in Sys > Adv > Firewall+NAT but exhausting it usually implies some very high use. You may need to reduce the state timeouts so the table is pruned more frequently.
-
There's the iperf of the other direction:
The only traffic is from speedtest, im not running anything else and the pf has 8GB Ram and 8 Cores
Pflocal:
Pfremote: