ovpnc with subint, no arp replies for not directly connected
-
I'm attempting to route traffic from a RHEL node via a bonded subinterface bond0.20/192.168.20.100 (that's VLAN tagging packets) via an openvpn tunnel established via the client at the pfsense box.
My use case is that on the RHEL box, there will be docker container for which I'll use macvlan.
I've configured a subint on the pfsense box igb1.20/192.168.20.1, and configured the NAT, the rules, and the gateway all according to pfsense documentation.
I can ping the ip address of the subint on the pfsense that's facing the subint on the RHEL box (using
ping -Ito source from a specific interface on the RHEL box). I can see the hits in the firewall logs as well with pcap within pfsense. However, when I ping an internet based IP via the RHEL subint, I see arp traffic ("who has 4.2.2.2"), but the pfsense box isn't sending "is-at" replies.I've referred to the follow:
https://helpdesk.privateinternetaccess.com/guides/routers/pfsense/pfsense-2-6-0-openvpn-setup
https://docs.netgate.com/pfsense/en/latest/recipes/openvpn-s2s-route-internet-traffic.html
https://docs.netgate.com/pfsense/en/latest/recipes/openvpn-s2s-tls.htmlHere is the routing table on the pfsense box with the "Don't add or remove routes automatically" within the ovpn client config checked:
Routing tables Internet: Destination Gateway Flags Netif Expire default xxx.xxx.xxx.1 UGS igb0 <-- set to use WAN as default gateway 4.2.2.2 10.20.112.1 UGHS ovpnc1 <-- used for monitoring of the ovpnc tunnel 10.20.112.0/24 link#7 U ovpnc1 10.20.112.168 link#4 UHS lo0 zzz.zzz.zzz.75 link#1 UHS igb0 zzz.zzz.yyy.76 link#1 UHS igb0 127.0.0.1 link#4 UH lo0 xxx.xxx.xxx.0/22 link#1 U igb0 xxx.xxx.xxx.2 link#4 UHS lo0 192.168.1.0/24 link#2 U igb1 192.168.1.1 link#4 UHS lo0 192.168.20.0/24 link#8 U igb1.20 192.168.20.1 link#4 UHS lo0And unchecked:
Routing tables Internet: Destination Gateway Flags Netif Expire 0.0.0.0/1 10.10.112.1 UGS ovpnc1 <-- must be dynamically pushed into route table from ovpn server default xxx.xxx.xxx.1 UGS igb0 <-- set to use WAN as default gateway 4.2.2.2 10.10.112.1 UGHS ovpnc1 <-- used for monitoring of the ovpnc tunnel 10.10.112.0/24 link#7 U ovpnc1 10.10.112.107 link#4 UHS lo0 qqq.qqq.qqq.qqq xxx.xxx.xxx.1 UGHS igb0 zzz.zzz.zzz.75 link#1 UHS igb0 zzz.zzz.yyy.76 link#1 UHS igb0 127.0.0.1 link#4 UH lo0 128.0.0.0/1 10.10.112.1 UGS ovpnc1 xxx.xxx.xxx.0/22 link#1 U igb0 xxx.xxx.xxx.2 link#4 UHS lo0 192.168.1.0/24 link#2 U igb1 192.168.1.1 link#4 UHS lo0 192.168.20.0/24 link#8 U igb1.20 192.168.20.1 link#4 UHS lo0Even with this prioritized route, I do not receive arp replies from the pfsense box.
I believe my challenge is related to the use of subints/vlan tagging and the non-functioning of policy routes as per the Firewall Rules?
I'm at a loss. Any guidance is appreciated.
Thanks!
-
@mbrownnyc said in ovpnc with subint, no arp replies for not directly connected:
However, when I ping an internet based IP via the RHEL subint, I see arp traffic ("who has 4.2.2.2"), but the pfsense box isn't sending "is-at" replies.
pfSense is not responsible to do this. The device tries to access an IP outside of its subnet, so it should not do an ARP request for it, but send the packets to its default gateway.
So I guess, there is something wrong in the network configuration.
-
@viragomann sorry, it arps for the DNS name. I'll post the pcap output later today. So... it's client DNS?
-
@mbrownnyc
Not clear, what you mean with "DNS name".
Anyway ARP requests are only done for IPs within the same network segment. ARP is a layer 2 protocol, not meant to pass a router. -
@viragomann it appears that the client is performing DNS lookups within tcpdump.
On the client (192.168.1.100) I'm running the following:
ping -I bond0.20 google.comOn the client, I'm also running tcpdump:
ash-4.4# tcpdump -i 6 -vvveXX arp tcpdump: listening on bond0.20, link-type EN10MB (Ethernet), snapshot length 262144 bytes 21:55:27.548736 00:11:32:ef:c5:21 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has lga25s78-in-f14.1e100.net tell 192.168.20.100, length 28 0x0000: ffff ffff ffff 0011 32ef c521 0806 0001 ........2..!.... 0x0010: 0800 0604 0001 0011 32ef c521 c0a8 1464 ........2..!...d 0x0020: 0000 0000 0000 8efb 23ae ........#.On the pfsense instance, I run the Packet Capture Diagnostics Tool, bound to the subinet igb1.20:
02:53:38.918946 ARP, Request who-has 4.2.2.2 tell 192.168.20.100, length 46 02:53:39.920937 ARP, Request who-has 4.2.2.2 tell 192.168.20.100, length 46 02:53:40.923218 ARP, Request who-has 4.2.2.2 tell 192.168.20.100, length 46 02:53:42.919292 ARP, Request who-has 4.2.2.2 tell 192.168.20.100, length 46 02:53:43.921172 ARP, Request who-has 4.2.2.2 tell 192.168.20.100, length 46 02:53:44.923179 ARP, Request who-has 4.2.2.2 tell 192.168.20.100, length 46Testing this further, I assume my client isn't sending packets to the gateway at all... Does that sound accurate?
-
Arg!!! this is a client routing "issue"
With route in local route table on RHEL host to the gateway of subint:
ash-4.4# ip route add 9.9.9.9/32 via 192.168.20.1 ash-4.4# ip route get 9.9.9.9 from 192.168.20.100 9.9.9.9 from 192.168.20.100 via 192.168.20.1 dev bond0.20 cache ash-4.4# ping -I bond0.20 9.9.9.9 PING 9.9.9.9 (9.9.9.9) from 192.168.20.100 bond0.20: 56(84) bytes of data. 64 bytes from 9.9.9.9: icmp_seq=1 ttl=50 time=19.7 ms 64 bytes from 9.9.9.9: icmp_seq=2 ttl=50 time=17.9 ms 64 bytes from 9.9.9.9: icmp_seq=3 ttl=50 time=21.0 ms ^C --- 9.9.9.9 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 5ms rtt min/avg/max/mdev = 17.911/19.525/21.007/1.277 mswithout route in local route table on RHEL host to the gateway on the subint:
ash-4.4# ip route delete 9.9.9.9/32 via 192.168.20.1 ash-4.4# ip route get 9.9.9.9 from 192.168.20.100 9.9.9.9 from 192.168.20.100 via 192.168.1.1 dev bond0.1 cache ash-4.4# ping -I bond0.20 9.9.9.9 PING 9.9.9.9 (9.9.9.9) from 192.168.20.100 bond0.20: 56(84) bytes of data. ^C --- 9.9.9.9 ping statistics --- 2 packets transmitted, 0 received, 100% packet loss, time 1msNow to tshoot in that direction.
Thanks!