One external IP is being (wrongly) routed to OpenVPN
-
I had originally posted this question in "General Questions" (http://forum.pfsense.org/index.php/topic,21658.0.html), but after some investigation I think it might be more of an OpenVPN question.
One of my clients was suddenly unable to access a certain website (www.palmettogba.com) on the Monday after Xmas. They had used it every day for months; my other clients in the building (all of whom are also using pfSense!) could still reach it; pinging or tracerouting directly from the pfSense box failed, but if I configured my laptop with the office's static IP and plugged directly into the T1, I could reach it.
Cutting to the chase: I ran tcpdump on the LAN interface in one shell session, and pinged the numeric IP of the Palmetto site in another shell session. I received the following in the tcpdump:
10:46:29.182937 arp who-has 216.251.231.64 tell 192.168.33.1 10:46:30.183472 arp who-has 216.251.231.64 tell 192.168.33.1
Meanwhile, each ping simply timed out with no response.
192.168.33.0/24 was the Address Pool I had reserved for the "road warrior" OpenVPN tunnel. As an experiment, I changed the Address Pool to 192.168.35.0/24, and tried ping/tcpdump again. This time, the response was:
11:13:18.106462 arp who-has 216.251.231.64 tell 192.168.35.1 11:13:19.107454 arp who-has 216.251.231.64 tell 192.168.35.1
I then tried disabling the tunnel, and suddenly I was able to ping/surf to Palmetto! However, as soon as I re-enabled the tunnel, Palmetto disappeared again.
So my question is: where is this set? I've looked through all the configuration files/locations I can think of, and don't see anything that specifically routes 216.251.231.64 through OpenVPN; by the way, 216.254.231.63 IS reachable (but I don't need it.) And how/why would this have changed spontaneously over the weekend? I'm the only person who has (or wants) authorized access, and I don't see any evidence of other tampering; I don't think this is a hack.
Before anyone asks - yes, this is persistent across reboots.
If you'd like me to post any specific files/screenshots, let me know. Thank you!
-
The output of "netstat -rn" before and during an OpenVPN connection would be nice, as well as the output of "ipconfig -a" before and during.
It may also help to have that same (or equivalent output) from the client machine. If it's windows, this would be the output of "route print" and also "ipconfig /all"
-
The output of "netstat -rn" before and during an OpenVPN connection would be nice, as well as the output of "ipconfig -a" before and during.
It may also help to have that same (or equivalent output) from the client machine. If it's windows, this would be the output of "route print" and also "ipconfig /all"
Here ya go… I'm afraid I don't think you'll find this very helpful, though...
(I've taken the liberty of obfuscating the client's public IP info.)
Before connecting:# netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default OurPublicIP UGS 0 811675 fxp1 OurPublicNet/29 link#2 UC 0 0 fxp1 OurPublicGW 00:a0:c8:41:75:f3 UHLW 2 9190 fxp1 860 127.0.0.1 127.0.0.1 UH 0 0 lo0 192.168.35.0&0xc0a82302 link#9 UC 0 0 tap0 192.168.254.0/24 link#1 UC 0 0 fxp0 192.168.254.5 00:10:5a:62:f0:a9 UHLW 1 207 fxp0 209 192.168.254.18 00:11:11:97:e8:44 UHLW 1 46270 fxp0 1176 192.168.254.32 00:0d:56:9b:4e:fc UHLW 1 10776 fxp0 1184 192.168.254.48 00:15:f2:92:40:cc UHLW 1 1607 fxp0 1037 192.168.254.49 00:15:f2:92:41:3b UHLW 1 427 fxp0 1173 192.168.254.50 00:15:f2:92:41:4b UHLW 1 897 fxp0 1087 192.168.254.101 00:15:f2:92:d0:60 UHLW 1 124 fxp0 1131 192.168.254.103 00:15:f2:92:3d:d7 UHLW 1 2480 fxp0 726 192.168.254.112 00:15:f2:92:d0:78 UHLW 1 10343 fxp0 1182 192.168.254.113 00:15:f2:92:41:23 UHLW 1 210 fxp0 1122 192.168.254.114 00:15:f2:92:d0:7a UHLW 1 269024 fxp0 993 192.168.254.116 00:0d:56:9b:4f:ee UHLW 1 31277 fxp0 996 192.168.254.132 00:26:18:30:13:8f UHLW 1 67652 fxp0 867 192.168.254.172 00:13:20:96:c5:33 UHLW 1 36753 fxp0 1157 192.168.254.193 00:80:77:ed:c4:79 UHLW 1 0 fxp0 1039 192.168.254.254 00:50:8b:68:d6:8a UHLW 1 96982 lo0 Internet6: Destination Gateway Flags Netif Expire ::1 ::1 UHL lo0 fe80::%fxp0/64 link#1 UC fxp0 fe80::250:8bff:fe68:d68a%fxp0 00:50:8b:68:d6:8a UHL lo0 fe80::%fxp1/64 link#2 UC fxp1 fe80::250:8bff:fe68:d68b%fxp1 00:50:8b:68:d6:8b UHL lo0 fe80::%lo0/64 fe80::1%lo0 U lo0 fe80::1%lo0 link#4 UHL lo0 fe80::2bd:e1ff:fe25:0%tap0 00:bd:e1:25:00:00 UHL lo0 ff01:1::/32 link#1 UC fxp0 ff01:2::/32 link#2 UC fxp1 ff01:4::/32 ::1 UC lo0 ff01:9::/32 link#9 UC tap0 ff02::%fxp0/32 link#1 UC fxp0 ff02::%fxp1/32 link#2 UC fxp1 ff02::%lo0/32 ::1 UC lo0 ff02::%tap0/32 link#9 UC tap0
After connecting:
# netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default OurPublicIP UGS 0 811953 fxp1 OurPublicNet/29 link#2 UC 0 0 fxp1 OurPublicGW 00:a0:c8:41:75:f3 UHLW 2 9200 fxp1 703 127.0.0.1 127.0.0.1 UH 0 0 lo0 192.168.35.0&0xc0a82302 link#9 UC 0 0 tap0 192.168.254.0/24 link#1 UC 0 0 fxp0 192.168.254.5 00:10:5a:62:f0:a9 UHLW 1 207 fxp0 1078 192.168.254.18 00:11:11:97:e8:44 UHLW 1 46270 fxp0 1019 192.168.254.32 00:0d:56:9b:4e:fc UHLW 1 10776 fxp0 1199 192.168.254.48 00:15:f2:92:40:cc UHLW 1 1607 fxp0 880 192.168.254.49 00:15:f2:92:41:3b UHLW 1 429 fxp0 1145 192.168.254.50 00:15:f2:92:41:4b UHLW 1 897 fxp0 930 192.168.254.101 00:15:f2:92:d0:60 UHLW 1 124 fxp0 974 192.168.254.103 00:15:f2:92:3d:d7 UHLW 1 2480 fxp0 569 192.168.254.112 00:15:f2:92:d0:78 UHLW 1 10343 fxp0 1053 192.168.254.113 00:15:f2:92:41:23 UHLW 1 210 fxp0 1195 192.168.254.114 00:15:f2:92:d0:7a UHLW 1 269024 fxp0 1168 192.168.254.116 00:0d:56:9b:4f:ee UHLW 1 31277 fxp0 1142 192.168.254.132 00:26:18:30:13:8f UHLW 1 67652 fxp0 710 192.168.254.150 00:ff:f0:09:36:d9 UHLW 1 1 fxp0 1194 192.168.254.172 00:13:20:96:c5:33 UHLW 1 36753 fxp0 1000 192.168.254.193 00:80:77:ed:c4:79 UHLW 1 0 fxp0 882 192.168.254.254 00:50:8b:68:d6:8a UHLW 1 96982 lo0 Internet6: Destination Gateway Flags Netif Expire ::1 ::1 UHL lo0 fe80::%fxp0/64 link#1 UC fxp0 fe80::250:8bff:fe68:d68a%fxp0 00:50:8b:68:d6:8a UHL lo0 fe80::%fxp1/64 link#2 UC fxp1 fe80::250:8bff:fe68:d68b%fxp1 00:50:8b:68:d6:8b UHL lo0 fe80::%lo0/64 fe80::1%lo0 U lo0 fe80::1%lo0 link#4 UHL lo0 fe80::2bd:e1ff:fe25:0%tap0 00:bd:e1:25:00:00 UHL lo0 ff01:1::/32 link#1 UC fxp0 ff01:2::/32 link#2 UC fxp1 ff01:4::/32 ::1 UC lo0 ff01:9::/32 link#9 UC tap0 ff02::%fxp0/32 link#1 UC fxp0 ff02::%fxp1/32 link#2 UC fxp1 ff02::%lo0/32 ::1 UC lo0 ff02::%tap0/32 link#9 UC tap0
Windows settings, after connecting:
C:\Users\Marc>route print =========================================================================== Interface List 16...00 ff ec f4 7e 6f ......TAP-Win32 Adapter V9 #2 15...00 ff f0 09 36 d9 ......TAP-Win32 Adapter V9 12...00 24 2b d3 13 13 ......Atheros AR5007 802.11b/g WiFi Adapter 11...00 23 5a 37 ed ee ......Realtek RTL8102E/RTL8103E Family PCI-E Fast Ethern et NIC (NDIS 6.20) 21...08 00 27 00 54 4b ......VirtualBox Host-Only Ethernet Adapter 1...........................Software Loopback Interface 1 14...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter 13...00 00 00 00 00 00 00 e0 Teredo Tunneling Pseudo-Interface 24...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2 25...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #3 23...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #4 22...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #5 =========================================================================== IPv4 Route Table =========================================================================== Active Routes: Network Destination Netmask Gateway Interface Metric 0.0.0.0 0.0.0.0 192.168.0.1 192.168.0.144 25 127.0.0.0 255.0.0.0 On-link 127.0.0.1 306 127.0.0.1 255.255.255.255 On-link 127.0.0.1 306 127.255.255.255 255.255.255.255 On-link 127.0.0.1 306 192.168.0.0 255.255.255.0 On-link 192.168.0.144 281 192.168.0.144 255.255.255.255 On-link 192.168.0.144 281 192.168.0.255 255.255.255.255 On-link 192.168.0.144 281 192.168.56.0 255.255.255.0 On-link 192.168.56.1 276 192.168.56.1 255.255.255.255 On-link 192.168.56.1 276 192.168.56.255 255.255.255.255 On-link 192.168.56.1 276 192.168.254.0 255.255.255.0 On-link 192.168.254.150 286 192.168.254.0 255.255.255.0 192.168.254.254 192.168.254.150 30 192.168.254.150 255.255.255.255 On-link 192.168.254.150 286 192.168.254.255 255.255.255.255 On-link 192.168.254.150 286 224.0.0.0 240.0.0.0 On-link 127.0.0.1 306 224.0.0.0 240.0.0.0 On-link 192.168.56.1 276 224.0.0.0 240.0.0.0 On-link 192.168.0.144 281 224.0.0.0 240.0.0.0 On-link 192.168.254.150 286 255.255.255.255 255.255.255.255 On-link 127.0.0.1 306 255.255.255.255 255.255.255.255 On-link 192.168.56.1 276 255.255.255.255 255.255.255.255 On-link 192.168.0.144 281 255.255.255.255 255.255.255.255 On-link 192.168.254.150 286 =========================================================================== Persistent Routes: None IPv6 Route Table =========================================================================== Active Routes: If Metric Network Destination Gateway 1 306 ::1/128 On-link 21 276 fe80::/64 On-link 15 286 fe80::/64 On-link 15 286 fe80::c8d3:8856:771e:c54/128 On-link 21 276 fe80::dd17:67ce:1898:494e/128 On-link 1 306 ff00::/8 On-link 21 276 ff00::/8 On-link 15 286 ff00::/8 On-link =========================================================================== Persistent Routes: None C:\Users\Marc>ipconfig /all Windows IP Configuration Host Name . . . . . . . . . . . . : Marc-HP Primary Dns Suffix . . . . . . . : Node Type . . . . . . . . . . . . : Mixed IP Routing Enabled. . . . . . . . : No WINS Proxy Enabled. . . . . . . . : No DNS Suffix Search List. . . . . . : lbms.local Ethernet adapter OpenVPN-CCMG: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : TAP-Win32 Adapter V9 #2 Physical Address. . . . . . . . . : 00-FF-EC-F4-7E-6F DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Ethernet adapter OpenVPN: Connection-specific DNS Suffix . : Description . . . . . . . . . . . : TAP-Win32 Adapter V9 Physical Address. . . . . . . . . : 00-FF-F0-09-36-D9 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::c8d3:8856:771e:c54%15(Preferred) IPv4 Address. . . . . . . . . . . : 192.168.254.150(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Lease Obtained. . . . . . . . . . : Thursday, December 31, 2009 8:07:43 PM Lease Expires . . . . . . . . . . : Friday, December 31, 2010 8:07:42 PM Default Gateway . . . . . . . . . : DHCP Server . . . . . . . . . . . : 192.168.254.0 DHCPv6 IAID . . . . . . . . . . . : 402718704 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-92-34-EA-00-23-5A-37-ED-EE DNS Servers . . . . . . . . . . . : 192.168.254.254 NetBIOS over Tcpip. . . . . . . . : Enabled Wireless LAN adapter Wireless Network Connection: Connection-specific DNS Suffix . : lbms.local Description . . . . . . . . . . . : Atheros AR5007 802.11b/g WiFi Adapter Physical Address. . . . . . . . . : 00-24-2B-D3-13-13 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes IPv4 Address. . . . . . . . . . . : 192.168.0.144(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Lease Obtained. . . . . . . . . . : Thursday, December 31, 2009 7:32:50 PM Lease Expires . . . . . . . . . . : Thursday, December 31, 2009 9:32:50 PM Default Gateway . . . . . . . . . : 192.168.0.1 DHCP Server . . . . . . . . . . . : 192.168.0.1 DNS Servers . . . . . . . . . . . : 192.168.0.1 208.67.222.222 NetBIOS over Tcpip. . . . . . . . : Enabled Ethernet adapter Local Area Connection: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Realtek RTL8102E/RTL8103E Family PCI-E Fa st Ethernet NIC (NDIS 6.20) Physical Address. . . . . . . . . : 00-23-5A-37-ED-EE DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Ethernet adapter Local Area Connection 2: Connection-specific DNS Suffix . : Description . . . . . . . . . . . : VirtualBox Host-Only Ethernet Adapter Physical Address. . . . . . . . . : 08-00-27-00-54-4B DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::dd17:67ce:1898:494e%21(Preferred) IPv4 Address. . . . . . . . . . . : 192.168.56.1(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : DHCPv6 IAID . . . . . . . . . . . : 705167399 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-92-34-EA-00-23-5A-37-ED-EE DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1 NetBIOS over Tcpip. . . . . . . . : Enabled Tunnel adapter isatap.lbms.local: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : lbms.local Description . . . . . . . . . . . : Microsoft ISATAP Adapter Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Tunnel adapter Teredo Tunneling Pseudo-Interface: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Tunnel adapter isatap.{ECF47E6F-D4B5-4F85-AE89-A70F744115A6}: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2 Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Tunnel adapter isatap.{6AFFB67E-9A26-47F7-A49F-EA070A9B7F01}: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Microsoft ISATAP Adapter #3 Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Tunnel adapter isatap.{F00936D9-FBAD-4401-9B43-4848C8B3BA98}: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Microsoft ISATAP Adapter #4 Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Tunnel adapter isatap.{DBBED88B-A4B8-4D6E-98AF-E01EBC1C62BB}: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Microsoft ISATAP Adapter #5 Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes
-
I should give a little extra background: I only discovered the OpenVPN tie-in after barking up many wrong trees. I don't understand the connection at all…
The client is a doctor's office, and Palmetto is the Medicare carrier for our region, so the office accesses the website every day. This has always worked, up until Monday morning. Also, I set up an OpenVPN road-warrior tunnel at the same time that I installed pfSense (about six months ago), and there hadn't been any previous problems. Suddenly, on Monday the office could not reach the Palmetto site; besides being unable to browse the site, I could not ping from the client machines, from the pfSense shell (via Putty), or from Diagnostics/Ping in the WebGUI. I have several other clients in the same medical building who also use pfSense, and I was able to ping Palmetto from their pfSense boxes, so initially I blamed the ISP. The ISP's tech support instructed me to bypass the router and plug my laptop directly into the T1… and imagine how foolish I felt when it worked! However, I still couldn't find anything in the pfSense configuration that would cause such a result - all packets destined for Palmetto seemed to vanish into thin air. (And ONLY Palmetto - all other sites seem to be just fine, and even an IP address that's off-by-one from the Palmetto address works.)
Finally this morning I logged in via Putty from home, and got the result that I posted earlier. It hadn't even crossed my mind earlier that OpenVPN might be involved, as this seemed to be a strictly internal issue (and since I hadn't made any changes recently.)
It makes no difference whether anyone is connected via OpenVPN or not; if the tunnel is enabled, "arp who-has Palmetto" gets the answer "OpenVPN has it" - and presumably, the packet then gets sent through OpenVPN and into a black hole (especially if no-one is connected at the time!) If I disable the tunnel, packets intended for Palmetto are routed through the WAN interface as they should be, and the ping succeeds. I could delete the tunnel and re-create it, but I would rather not (unless I was sure that it would fix the problem) because I dread the thought of re-creating and re-distributing certificates...In any case, I think I've looked in all the usual places. I was hoping that someone would have some ideas/experience of UNusual places to look?
-
I am confused by the client arping for an address not on its subnet and getting a reply. This may be some openvpn oddity, though. Have you looked at the openvpn config (/conf/config.xml) to see if that address shows up?
-
Yes, the contents of the OpenVPN client (on windows) config and the server on pfsense (/var/etc/openvpn_*.conf) might also help shed some light on the situation.
-
Yes, the contents of the OpenVPN client (on windows) config and the server on pfsense (/var/etc/openvpn_*.conf) might also help shed some light on the situation.
Windows config file (tz.ovpn):
client dev tap dev-node OpenVPN proto udp remote OurPublicIP 1194 resolv-retry infinite nobind persist-key persist-tun mute-replay-warnings ca tz-ca.crt cert tz-marc-hp.crt key tz-marc-hp.key ns-cert-type server cipher BF-CBC comp-lzo verb 3
cat openvpn_server0.conf
writepid /var/run/openvpn_server0.pid #user nobody #group nobody daemon keepalive 10 60 ping-timer-rem persist-tun persist-key dev tun proto udp cipher BF-CBC up /etc/rc.filter_configure down /etc/rc.filter_configure tls-server ifconfig 192.168.35.1 192.168.35.2 push "route 192.168.254.0 255.255.255.0" lport 1194 push "dhcp-option DNS 192.168.254.254" push "dhcp-option NBT 4" ca /var/etc/openvpn_server0.ca cert /var/etc/openvpn_server0.cert key /var/etc/openvpn_server0.key dh /var/etc/openvpn_server0.dh comp-lzo dev tap0 server-bridge 192.168.254.254 255.255.255.0 192.168.254.150 192.168.254.160
-
I am confused by the client arping for an address not on its subnet and getting a reply. This may be some openvpn oddity, though.
I'm not sure what you mean by "the client" in this context. I was pinging from the pfSense shell… I do wonder where the reply came from. Is there any switch I can use with tcpdump (or another tool) to find out?
Have you looked at the openvpn config (/conf/config.xml) to see if that address shows up?
Yes I have, and no it doesn't. In fact, even the first octet doesn't appear anywhere in the file. I can post it if you'd like, but it would obviously require some obfuscation.
I think the answer lies in the fact that somebody somewhere on the network is answering that ARP who-has, and if I can find out who it is (and stop it) the problem will be solved. Perhaps I'm in for a day of good old-fashioned unplugging and re-plugging… I'm hoping for a more modern answer, though.
-
sorry misread who was doing the arp. you could try 'tcpdump -lnvvv' instead. what i think must be going on is that the local IP (192.168.x.x) would not be arping for the palmetto address normally, much less getting a reply, so this must be coming to/from the tunnel. What does 'clog /var/log/openvpn.log' show?
-
Why are you using "dev tap" on the client? That is probably your problem. And why the server-bridge line on the server side?
iirc, tap devices are more for bridging than routing. You probably want that to be tun.
If you are trying to bridge, you want that to be tap on both sides, but I'm not familiar with a bridged setup.
-
Good point about tap vs tun.
-
Why are you using "dev tap" on the client? That is probably your problem. And why the server-bridge line on the server side?
iirc, tap devices are more for bridging than routing. You probably want that to be tun.
If you are trying to bridge, you want that to be tap on both sides, but I'm not familiar with a bridged setup.
I'm using tap intentionally because I do use bridging. (It's not necessary at this office, but I have been using a standard configuration across all of my installations - much less hassle that way!) One of my offices has a permanent (or, at least, we WISH it were permanent) IPSec tunnel to a hosting company. The hosting company's road-warrior IPSec VPN (from a company that rhymes with Crisco) is much less reliable than the site-to-site, so I decided to have my road warriors connect to the office network and access the hosted network via bridging. It's been very stable and very easy to add/remove clients, as opposed to dealing with the hosting people… Once I got that set up and working, I standardized my OpenVPN setup; this particular office doesn't currently need bridging, but it's been working smoothly.
I can change the OpenVPN setup to use tun instead; again, I'd prefer not to unless I'm sure it will fix this problem, because it means modifying all the client machines (most of which are at the doctors'/employees' houses.) I have this same setup in nearly a dozen offices, running for over two years now, and this is the first time anything like this has come up. Perhaps this is a hidden danger of using bridging, in which case I should expect to get hit with it at my other sites sooner or later; meanwhile, I'd like to get to the bottom of why it happened here.
- It's persistent across reboots.
- It doesn't matter whether any OpenVPN clients are connected or not.
- As far as I can tell, it's only this one IP address that's affected. (Google.com and forum.pfsense.org, for instance, work just fine.)
These things make me think that pfSense has stored this (bad) routing information somewhere. At this point it's more of an intellectual curiosity to me to find out where. Should I perhaps ask on a FreeBSD forum?
-
sorry misread who was doing the arp. you could try 'tcpdump -lnvvv' instead. what i think must be going on is that the local IP (192.168.x.x) would not be arping for the palmetto address normally, much less getting a reply, so this must be coming to/from the tunnel. What does 'clog /var/log/openvpn.log' show?
# clog /var/log/openvpn.log Jan 1 22:55:25 pfsense openvpn[345]: OpenVPN 2.0.6 i386-portbld-freebsd7.2 [SSL] [LZO] built on Dec 4 2009 Jan 1 22:55:25 pfsense openvpn[345]: WARNING: file '/var/etc/openvpn_server0.key' is group or others accessible Jan 1 22:55:25 pfsense openvpn[345]: WARNING: Since you are using --dev tap, the second argument to --ifconfig must be a netmask, for example something like 255.255.255.0\. (silence this warning with --ifconfig-nowarn) Jan 1 22:55:25 pfsense openvpn[345]: TUN/TAP device /dev/tap0 opened Jan 1 22:55:25 pfsense openvpn[345]: /sbin/ifconfig tap0 192.168.35.1 netmask 192.168.35.2 mtu 1500 up Jan 1 22:55:25 pfsense openvpn[345]: /etc/rc.filter_configure tap0 1500 1574 192.168.35.1 192.168.35.2 init Jan 1 22:55:25 pfsense openvpn[352]: UDPv4 link local (bound): [undef]:1194 Jan 1 22:55:25 pfsense openvpn[352]: UDPv4 link remote: [undef] Jan 1 22:55:25 pfsense openvpn[352]: Initialization Sequence Completed
-
Just thought I'd post the eventual solution, in case anyone else ever has the same problem. I added a static route:
Interface Network Gateway Description
WAN 216.251.231.64/32 (our gateway) Palmetto
- in other words, I added an explicit rule to reinforce what should be happening anyway. And now it works. What caused the original problem, I don't know…