2.3.2\. Multiwan: Cannot access some web-sites
-
Hi all,
Pfsense 2.3.2 (updated from 2.2.4) on Dell PE1800 server with 4 Eth cards. One of them for Internet access.
Intel em0 with 5 Vlans (3 uplink ISP - Uznet 213+Uznet27+SITA151 and 2 subnets Uznet9 behind Uznet213 + SITA155 behind SITA151)
Have to use all 3 lines because Uznet27 has 30Mbps link, Uznet 213 only 2Mbps world + 10Mbps domestic, and SITA151 only 2Mbps, but without restrictions (Uznet ISP blocks some web-sites and services we need)
For those blocked sites I use IP aliaces, then add alias in static routing with gateway SITA151_GW. Also point sites in DNS Resolver so every time users get same IP and last in LAN rules add special one (source LAN net, destination SITA_Web, gateway SITA151_GW) at the top.
762/530.38 MiB IPv4 * DMZ net * SITA WEB * SITA151GW none ALLOW blocked sites to SITA
So far this works like a charm, but Last 2-3 days cannot open 2 web-sites:
1. ng.ru (188.40.89.58) full blocked by UzNET
2. inosmi.ru (178.248.232.60) partially blocked by UzNETWhen try access to ng.ru get 403 forbidden
When try access to inosmi.ru even over Uznet some time it works, but mostly nothing displayedtraceroute from any user PC show that SITA151_GW is used.
Diagnostics/States indicate strange behavior: some time UzNET27_GW is used when accessing inosmi.ru.At the same time, if put SITA151 IP and SITA151_GW on laptop and connect right to CISCO router, can access both sites. Also sites opens freely with SITA155 subnet IP (routed by pfsense to SITA151).
What is the problem?
More info
states for ng.ru (403 forbidden)LAN1 tcp 192.168.1.164:30286 -> 188.40.89.58:80 ESTABLISHED:ESTABLISHED 20 / 13 6 KiB / 3 KiB SITA151 tcp x.x.151.183:28775 (192.168.1.164:30286) -> 188.40.89.58:80 ESTABLISHED:ESTABLISHED 20 / 13 6 KiB / 3 KiB ``` **states for inosmi.ru**
LAN1 tcp 192.168.1.110:54401 -> 178.248.232.60:80 CLOSED:SYN_SENT 3 / 0 152 B / 0 B
SITA151 tcp x.x.151.183:45974 (192.168.1.110:54401) -> 178.248.232.60:80 SYN_SENT:CLOSED 3 / 0 152 B / 0 B
LAN1 tcp 192.168.1.110:54403 -> 178.248.232.60:80 CLOSED:SYN_SENT 3 / 0 152 B / 0 B
SITA151 tcp x.x.151.183:58359 (192.168.1.110:54403) -> 178.248.232.60:80 SYN_SENT:CLOSED 3 / 0 152 B / 0 B
LAN1 tcp 192.168.1.110:54407 -> 178.248.232.60:80 CLOSED:SYN_SENT 3 / 0 152 B / 0 B
SITA151 tcp x.x.151.183:22841 (192.168.1.110:54407) -> 178.248.232.60:80 SYN_SENT:CLOSED 3 / 0 152 B / 0 B**packet capture on LAN for ng.ru**
11:26:25.037718 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 5623, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30239 > 188.40.89.58.80: Flags , cksum 0x924c (correct), seq 4291653341, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:26:25.051331 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 5628, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30241 > 188.40.89.58.80: Flags , cksum 0x801e (correct), seq 1736776018, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:26:25.169843 00:0a:5e:08:71:06 > 98:90:96:ad:d0:ae, ethertype IPv4 (0x0800), length 66: (tos 0x40, ttl 54, id 9662, offset 0, flags , proto TCP (6), length 52)
188.40.89.58.80 > 192.168.1.164.30239: Flags [S.], cksum 0xd98d (correct), seq 239702357, ack 4291653342, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 9], length 011:26:25.170286 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 128, id 5632, offset 0, flags , proto TCP (6), length 40)
192.168.1.164.30239 > 188.40.89.58.80: Flags [.], cksum 0x1341 (correct), seq 1, ack 1, win 16425, length 011:26:25.171017 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 5633, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30243 > 188.40.89.58.80: Flags , cksum 0x18ad (correct), seq 2030989112, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:26:25.183831 00:0a:5e:08:71:06 > 98:90:96:ad:d0:ae, ethertype IPv4 (0x0800), length 66: (tos 0x40, ttl 54, id 51180, offset 0, flags , proto TCP (6), length 52)
188.40.89.58.80 > 192.168.1.164.30241: Flags [S.], cksum 0xfd3a (correct), seq 3001530075, ack 1736776019, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 9], length 011:26:25.184035 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 128, id 5634, offset 0, flags , proto TCP (6), length 40)
192.168.1.164.30241 > 188.40.89.58.80: Flags [.], cksum 0x36ee (correct), seq 1, ack 1, win 16425, length 011:26:25.185792 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 525: (tos 0x0, ttl 128, id 5635, offset 0, flags , proto TCP (6), length 511)
192.168.1.164.30241 > 188.40.89.58.80: Flags [P.], cksum 0xd7e2 (correct), seq 1:472, ack 1, win 16425, length 47111:26:25.311653 00:0a:5e:08:71:06 > 98:90:96:ad:d0:ae, ethertype IPv4 (0x0800), length 66: (tos 0x40, ttl 54, id 13342, offset 0, flags , proto TCP (6), length 52)
188.40.89.58.80 > 192.168.1.164.30243: Flags [S.], cksum 0x5fa0 (correct), seq 3700867925, ack 2030989113, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 9], length 011:26:25.311907 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 128, id 5636, offset 0, flags , proto TCP (6), length 40)
192.168.1.164.30243 > 188.40.89.58.80: Flags [.], cksum 0x9953 (correct), seq 1, ack 1, win 16425, length 011:26:25.325500 00:0a:5e:08:71:06 > 98:90:96:ad:d0:ae, ethertype IPv4 (0x0800), length 54: (tos 0x40, ttl 54, id 24132, offset 0, flags , proto TCP (6), length 40)
188.40.89.58.80 > 192.168.1.164.30241: Flags [.], cksum 0x7521 (correct), seq 1, ack 472, win 31, length 011:26:25.326994 00:0a:5e:08:71:06 > 98:90:96:ad:d0:ae, ethertype IPv4 (0x0800), length 380: (tos 0x40, ttl 54, id 12721, offset 0, flags , proto TCP (6), length 366)
188.40.89.58.80 > 192.168.1.164.30241: Flags [P.], cksum 0x5418 (correct), seq 1:327, ack 472, win 31, length 32611:26:25.527244 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 128, id 5637, offset 0, flags , proto TCP (6), length 40)
192.168.1.164.30241 > 188.40.89.58.80: Flags [.], cksum 0x3423 (correct), seq 472, ack 327, win 16343, length 0**packet capture on SITA151 for ng.ru**
11:27:57.056526 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 48868, offset 0, flags , proto TCP (6), length 52)
x.x.151.183.18524 > 188.40.89.58.80: Flags , cksum 0x3acc (correct), seq 451899, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:27:57.188907 f8:72:ea:68:82:00 > 00:15:c5:88:1e:83, ethertype IPv4 (0x0800), length 66: (tos 0x40, ttl 55, id 0, offset 0, flags , proto TCP (6), length 52)
188.40.89.58.80 > x.x.151.183.18524: Flags [S.], cksum 0xd976 (correct), seq 3924385355, ack 451900, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 9], length 011:27:57.189427 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 128, id 63634, offset 0, flags , proto TCP (6), length 40)
x.x.151.183.18524 > 188.40.89.58.80: Flags [.], cksum 0x132a (correct), seq 1, ack 1, win 16425, length 011:27:57.194542 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 551: (tos 0x0, ttl 128, id 53518, offset 0, flags , proto TCP (6), length 537)
x.x.151.183.18524 > 188.40.89.58.80: Flags [P.], cksum 0x4907 (correct), seq 1:498, ack 1, win 16425, length 49711:27:57.331339 f8:72:ea:68:82:00 > 00:15:c5:88:1e:83, ethertype IPv4 (0x0800), length 60: (tos 0x40, ttl 55, id 12592, offset 0, flags , proto TCP (6), length 40)
188.40.89.58.80 > x.x.151.183.18524: Flags [.], cksum 0x5143 (correct), seq 1, ack 498, win 31, length 011:27:57.334447 f8:72:ea:68:82:00 > 00:15:c5:88:1e:83, ethertype IPv4 (0x0800), length 380: (tos 0x40, ttl 55, id 12593, offset 0, flags , proto TCP (6), length 366)
188.40.89.58.80 > x.x.151.183.18524: Flags [P.], cksum 0x2e3b (correct), seq 1:327, ack 498, win 31, length 32611:27:57.535233 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 128, id 4416, offset 0, flags , proto TCP (6), length 40)
x.x.151.183.18524 > 188.40.89.58.80: Flags [.], cksum 0x1045 (correct), seq 498, ack 327, win 16343, length 0**packet capture on LAN for inosmi.ru**
11:33:11.271975 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 6261, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30311 > 178.248.232.60.80: Flags , cksum 0x8108 (correct), seq 1594464971, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:33:11.522529 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 6279, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30313 > 178.248.232.60.80: Flags , cksum 0x49d0 (correct), seq 2309924700, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:33:14.272012 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 6306, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30311 > 178.248.232.60.80: Flags , cksum 0x8108 (correct), seq 1594464971, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:33:14.516047 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 6307, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30313 > 178.248.232.60.80: Flags , cksum 0x49d0 (correct), seq 2309924700, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:33:20.266430 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 6311,
offset 0, flags , proto TCP (6), length 48)
192.168.1.164.30311 > 178.248.232.60.80: Flags , cksum 0x9511 (correct), seq 1594464971, win 8192, options [mss 1460,nop,nop,sackOK], length 011:33:20.516339 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 6312, offset 0, flags , proto TCP (6), length 48)
192.168.1.164.30313 > 178.248.232.60.80: Flags , cksum 0x5dd9 (correct), seq 2309924700, win 8192, options [mss 1460,nop,nop,sackOK], length 011:33:32.271890 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 6322, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30316 > 178.248.232.60.80: Flags , cksum 0x40da (correct), seq 1316285321, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:33:32.523542 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 6323, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30318 > 178.248.232.60.80: Flags , cksum 0x4f89 (correct), seq 2369101335, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:33:35.273376 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 6327, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30316 > 178.248.232.60.80: Flags , cksum 0x40da (correct), seq 1316285321, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:33:35.523316 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 6328, offset 0, flags , proto TCP (6), length 52)
192.168.1.164.30318 > 178.248.232.60.80: Flags , cksum 0x4f89 (correct), seq 2369101335, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:33:41.276039 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 6332, offset 0, flags , proto TCP (6), length 48)
192.168.1.164.30316 > 178.248.232.60.80: Flags , cksum 0x54e3 (correct), seq 1316285321, win 8192, options [mss 1460,nop,nop,sackOK], length 011:33:41.516843 98:90:96:ad:d0:ae > 00:0a:5e:08:71:06, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 6333, offset 0, flags , proto TCP (6), length 48)
192.168.1.164.30318 > 178.248.232.60.80: Flags , cksum 0x6392 (correct), seq 2369101335, win 8192, options [mss 1460,nop,nop,sackOK], length 0**packet capture on SITA151 for ng.ru**
11:36:09.201736 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 21188, offset 0, flags , proto TCP (6), length 52)
x.x.151.183.6805 > 178.248.232.60.80: Flags , cksum 0x0956 (correct), seq 4010397602, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:36:09.453032 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 20063, offset 0, flags , proto TCP (6), length 52)
x.x.151.183.31524 > 178.248.232.60.80: Flags , cksum 0x2c8a (correct), seq 961413531, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:36:12.208786 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 39327, offset 0, flags , proto TCP (6), length 52)
x.x.151.183.6805 > 178.248.232.60.80: Flags , cksum 0x0956 (correct), seq 4010397602, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:36:12.458782 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 8321, offset 0, flags , proto TCP (6), length 52)
x.x.151.183.31524 > 178.248.232.60.80: Flags , cksum 0x2c8a (correct), seq 961413531, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:36:16.625279 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 48509, offset 0, flags , proto TCP (6), length 52)
x.x.151.183.25498 > 178.248.232.60.80: Flags , cksum 0xba85 (correct), seq 2543856855, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:36:16.875155 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 36204, offset 0, flags , proto TCP (6), length 52)
x.x.151.183.30899 > 178.248.232.60.80: Flags , cksum 0xa9a1 (correct), seq 4276470108, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:36:18.204186 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 52843, offset 0, flags , proto TCP (6), length 48)
x.x.151.183.6805 > 178.248.232.60.80: Flags , cksum 0x1d5f (correct), seq 4010397602, win 8192, options [mss 1460,nop,nop,sackOK], length 011:36:18.453287 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 63430, offset 0, flags , proto TCP (6), length 48)
x.x.151.183.31524 > 178.248.232.60.80: Flags , cksum 0x4093 (correct), seq 961413531, win 8192, options [mss 1460,nop,nop,sackOK], length 011:36:19.625366 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 26386, offset 0, flags , proto TCP (6), length 52)
x.x.151.183.25498 > 178.248.232.60.80: Flags , cksum 0xba85 (correct), seq 2543856855, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:36:19.874360 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 59660, offset 0, flags , proto TCP (6), length 52)
x.x.151.183.30899 > 178.248.232.60.80: Flags , cksum 0xa9a1 (correct), seq 4276470108, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 011:36:25.627676 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 50339, offset 0, flags , proto TCP (6), length 48)
x.x.151.183.25498 > 178.248.232.60.80: Flags , cksum 0xce8e (correct), seq 2543856855, win 8192, options [mss 1460,nop,nop,sackOK], length 011:36:25.874599 00:15:c5:88:1e:83 > f8:72:ea:68:82:00, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 10667, offset 0, flags , proto TCP (6), length 48)
x.x.151.183.30899 > 178.248.232.60.80: Flags , cksum 0xbdaa (correct), seq 4276470108, win 8192, options [mss 1460,nop,nop,sackOK], length 0 -
More pics