Static route through an interface with no IP configuration
-
I've set up a site-to-site VPN using WireGuard.
[Site A] 10.10.0.1/24 [Site B] 10.20.0.1/24They're connected together through a VPS to get around CGNAT/NAT issues, but for the sake of example assume the sites have public IPs and connect directly.
And out of habit I allocated a VPN subnet as well,
10.30.0.0/24and assigned each site an IP, then set up a static routes between the two. In short, in order to reach10.20.0.0/24from Site A, route through Site B's VPN IP10.30.0.2.But then I stepped back and realized: "I shouldn't need a VPN subnet. I should be able to throw packets destined to
10.20.0.0/24down the interface and Site B should route." However to make a static route there has to be a gateway, and that gateway can only be on the subnet of a configured interface. So I can't route down a interface without a configuration, or specify Site B's LAN IP10.20.0.1as the gateway from Site A if the interface uses the VPN network10.30.0.1/24.Is there a way to get pfSense to route down a interface with no IP (static, DHCP) configuration?
Edit:
I also tried leaving it blank, resulting in a
dynamicgateway, but that didn't work with the silence I got when checkingtcpdumpon the tunnel from Site A when a client on Site A was trying to ping Site B, even though the routing table suggested it should be going down the VPN link. Not sure whatdynamicmeans here. -
@jarrodsfarrell said in Static route through an interface with no IP configuration:
However to make a static route there has to be a gateway, and that gateway can only be on the subnet of a configured interface.
I don't think so. Here you can do this. You will fail though when it comes to NAT or Reply-to.
That WG interface has no counterpart but it works -
@jarrodsfarrell said in Static route through an interface with no IP configuration:
But then I stepped back and realized: "I shouldn't need a VPN subnet.
Why would you think that?
What problem/issue are you trying to solve? Just create a tunnel network.. You could use link local address space if you wanted 169.254.x.x etc..
Not sure how you think this would work.. Why would the other end know to process this traffic. Your not sending traffic to its IP, your not sending it to its mac.. Even if the traffic got to the end - why would the vpn interface even look to process the traffic?
-
Mmm, you probably could make this work but pfSense is not setup to do so. I don't believe you can route that without interface IPs currently. For a point-to-point link for example you don't need addressing to pass the traffic.
-
@stephenw10 but again what problem is there to solve, what advantage? Just create a tunnel network and there you go. Spinning cycles trying solve a problem that doesn't exist if you ask me.
-
@johnpoz said in Static route through an interface with no IP configuration:
Just create a tunnel network and there you go
The funny thing is, you can even do that for S2S if the other site doesn't, like fritzbox or Lancom. It will work regardless and makes pfSense happy.
-
@johnpoz said in Static route through an interface with no IP configuration:
stephenw10 but again what problem is there to solve, what advantage?
Indeed there is none. I was just agreeing that it's technically not required. I imagine the webgui could probably be reworked to allow such a setup. But that's unlikely to happen because there's little to be gained.
-
@stephenw10 Thanks for the replies, and that is what I was probably wanting to know in the end if pfSense even supported what I was wanting to do with addressless routing. Implementing it through the webgui is not a hurry for me and what we have now is fine, but it's a nice to have to chuck onto the list.
@johnpoz And yeah I agree it's not really a problem to solve since relying on a VPN subnet to route through is perfectly fine, but it was still something in the back of my head that bugged me because I knew I shouldn't need to do routing like this.
I've been doing some networking with
iproute2and WireGuard in a NixOS install along with network namespaces, resulting in some of its routing done by IP matching then sending it down a interface with no deeper logic.#> ip -n rmt address 1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: host@if8: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ae:48:d2:2a:82:60 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet6 fe80::ac48:d2ff:fe2a:8260/64 scope link proto kernel_ll valid_lft forever preferred_lft forever 4: rmt: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000 link/noneLot of addresses reachable through the
rmtlink, but addressing the host goes down thehostlink.#> ip -n rmt route default dev rmt scope link 10.10.10.0/24 dev rmt scope link 10.20.20.0/24 dev rmt scope link 10.30.30.1 dev rmt scope link … 10.30.30.100 dev host scope linkrmt@inf2here is aveththat connects tohost@if8inside of thermtnamespace.# Note to self: properly disable ipv6 autoconf for this inteface #> ip address … 8: rmt@if2: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ae:48:d2:2a:82:60 brd ff:ff:ff:ff:ff:ff link-netns rmt inet 10.30.30.100/32 scope global rmt valid_lft forever preferred_lft forever inet6 fe80::ac48:d2ff:fe2a:8260/64 scope link proto kernel_ll valid_lft forever preferred_lft foreverThe host itself hosts some services, but only for use within the VPN network
#> ip route … 10.30.30.0/24 dev rmt scope linkAll the extra configuration to make this work:
# Create a veth between the host and namespace ip link add rmt type veth peer host netns rmt # Disable auto configuration sysctl -w net.ipv6.conf.rmt.accept_ra=0 sysctl -w net.ipv6.conf.rmt.autoconf=0 ip netns exec rmt sysctl -w net.ipv6.conf.host.autoconf=0 ip netns exec rmt sysctl -w net.ipv6.conf.host.accept_ra=0 # Disable ARP ip link set rmt arp off ip -n rmt link set host arp off # Setting the same mac address since IP Forwarding depends on mac addresses # matching the incoming interface. # Without ARP the other end sets the source and destination as the same so # IP Forwarding never acts, so to get around it just use the same mac address # on both ends of the veth. MACADDR=`ip link show rmt | grep link/ether | awk '{print $2}' # Because of a bug (?), the mac address changes during boot so set it again ip link set rmt address $MACADDR ip -n rmt link set host address $MACADDR # Alternatively could leave ARP on and proxy ARP to appease IP Forwarding, # but makes the link too noisy for my liking. #ip netns exec rmt sysctl -w net.ipv4.conf.host.proxy_arp=1 # Bring the veth up ip -n rmt link set host up ip link set rmt up # Enable forwarding ip netns exec rmt sysctl -w net.ipv4.conf.rmt.forwarding=1 ip netns exec rmt sysctl -w net.ipv4.conf.host.forwarding=1So I had it in my head that pfSense potentially supported what I had in mind given the Linux/BSD lineage and it being a valid way to route packets. In retrospect I should've talked about it since some would get what I was wanting to do and where I was coming from.
-
@jarrodsfarrell Like I said, you already can do that on pfSense...
-
@Bob.Dig Oh, sorry. I got focused in the other conversations, but I will have to experiment with dynamic routing option you suggested because I did stumble into doing it but didn't work last I tried,
@Bob.Dig said in Static route through an interface with no IP configuration:
@jarrodsfarrell said in Static route through an interface with no IP configuration:
However to make a static route there has to be a gateway, and that gateway can only be on the subnet of a configured interface.
I don't think so. Here you can do this. You will fail though when it comes to NAT or Reply-to.
That WG interface has no counterpart but it worksThough the warning about NAT might be why it's not working since I did configure Site A with a NAT so clients behind it appear from Site A's gateway, since I'm okay with Site A indiscriminately accessing Site B but not the other way around.
But revisiting the thought I could just use firewall rules to get to the same goal with the added benefit of devices in Site B knowing which device from Site A is interacting with it, along with just opening access to devices behind Site A as needed. Still needs as much statefulness as NAT anyways.
Given I'm pro-IPv6, I'm wondering why I'm voluntarily trying to do NAT in the first place and dealing with the pain it brings.
I will be visiting Site B today—I just happened to wake up middle of the night when you sent the post—so I can break things without fear of being locked out remotely.
-
@jarrodsfarrell said in Static route through an interface with no IP configuration:
since I did configure Site A with a NAT so clients behind it appear from Site A's gateway
But that is contrary to having a "Static route through an interface with no IP configuration".
You can't do NAT if you don't have an IP to begin with... -
@Bob.Dig I agree it does sound contrary and could use a better title, but I was NATing as Site A's LAN IP. i.e
10.10.0.20 wants 10.20.0.10 10.10.0.20 routes through 10.10.0.1 as it is its gateway 10.10.0.1 does NAT, turning source 10.10.0.20 to 10.10.0.1 10.10.0.1 routes through tun_wg0 for 10.20.0.0/24 to reach 10.20.0.10 10.20.0.10 receives packets from 10.10.0.1, with real address actually 10.10.0.20But again I think I'm going to scrap using NAT in this case since it brings extra complexity that isn't needed.
-
So I mentioned I could break things without fear. The environment seems to have already beat me to it.
Probably something to do with the storms we had which adds some concern.Going to get a bit more sleep in before I drive out and investigate. I'll report back if I get anywhere.
-
@jarrodsfarrell well you do you - not sure why your playing with something that is rock solid and just use a tunnel network ;)
What does not using a tunnel bring to the table other than a setup that is not a normal setup? I just do not see the point is all.
-
Well if you treat everything as P-t-P like, say, PPPoE you can do some things that would fail in pure routing. Like having a gateway outside the interface subnet. Or having the same gateway on multiple interfaces.
But....is that actually a good thing? -
Turns out the DSL modem for the site did not reconnect by itself, so I guess I'm nudging the ISP about why it didn't. It's part of the PoE switch so I could cause a power cycle on ping fail, but I'd prefer not because it belongs in the bin of "hacks I should not be doing." Anyways.
@Bob.Dig Happy to report that using the dynamic option for the gateway on both sides is working nicely, watching
tcpdumpacross every hop to see both sides chatting using their real IPs. Had to adjust the Wireguard configuration on the VPN server to include the whole Site A subnet instead of just the gateway which makes sense, otherwise the VPN server would drop those packets since those would be outside the allowed IPs for the peer—side-effect of the old NAT setup I was trying.I did try putting a outbound NAT rule on Site A for Site B's subnet to route via its LAN IP, but it seems to ignore it and use the VPN IP it has instead when watching
tcpdump. Though as said I'm abandoning NAT so it's a non-issue, though weird behavior. -
Oh, removing the IP configuration from both sides of the VPN link and removing the VPN subnet's existence on the VPN server still works too. So the original question can be considered answered I guess.
-
So there's a side-effect to this if Site A's pfSense itself wants to talk to Site B, and I can see the situation pfSense is in.
Site A knows that to reach 10.20.0.0/24 it goes down the tunnel, and this works fine when my laptop at Site A wants to talk to Site B since Site A's pfSense will be given the packet and route it as per its routing table. But Site A's pfSense is doing DNS resolving and is configured to query Site B for its domain...and it doesn't know which which IP to send as—it only knows how to route.
I did try to recreate the situation using
iprouteand network namespaces for the sake of demonstration (and because I think it's neat and cool,) but oddly it picks the correct source IP. Neat?# The namespaces ip netns add net_a ip netns add net_b ip netns add router # The veths connecting between the namespaces ip -n router link add net_a type veth peer router netns net_a ip -n router link add net_b type veth peer router netns net_b # Bring them all up for routing ip -n router link set net_a up ip -n router link set net_b up ip -n net_a link set router up ip -n net_b link set router up # Add addresses ip -n router address add 10.10.0.1/24 dev net_a ip -n net_a address add 10.10.0.2/24 dev router ip -n net_b address add 10.20.0.1/24 dev router # Routes ip -n router route add 10.20.0.0/24 dev net_b # Getting slightly ahead of ourselves.. ip -n net_b route add 10.10.0.0/24 dev router src 10.20.0.1 # And ape a typical LAN to WAN ip -n net_a route add default via 10.10.0.1 dev router src 10.10.0.2 ip -n router link set lo up ip -n router route add default via 127.0.0.1 dev lo src 127.0.0.2 # Easier to do than fake a point to point link; # disabling arp and setting the same mac on each side of the veth ip netns exec router sysctl -w -w net.ipv4.conf.net_b.proxy_arp=1 # Enable forwarding (naturally) ip netns exec router sysctl -w -w net.ipv4.conf.net_a.forwarding=1 ip netns exec router sysctl -w -w net.ipv4.conf.net_b.forwarding=1#> ip netns exec net_b tcpdump tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on router, link-type EN10MB (Ethernet), snapshot length 262144 bytes 14:12:24.732059 IP 10.10.0.1 > 10.20.0.1: ICMP echo request, id 20347, seq 1, length 64 14:12:24.732092 IP 10.20.0.1 > 10.10.0.1: ICMP echo reply, id 20347, seq 1, length 64#> ip netns exec router ping 10.20.0.1 PING 10.20.0.1 (10.20.0.1) 56(84) bytes of data. 64 bytes from 10.20.0.1: icmp_seq=1 ttl=64 time=0.061 ms???
# Cleanup ip netns delete net_a ip netns delete net_b ip netns delete routerI put back the tunnel network then configured the resolver to use the VPN IP, so pfSense has an IP to use to reach the other instance (and vice-versa.) I guess this is what @stephenw10 hinted about:
@stephenw10 said in Static route through an interface with no IP configuration:
Or having the same gateway on multiple interfaces.
But....is that actually a good thing?Still I'm curious if I can coerce Site A's pfSense that if it wants to talk to Site B it should use it's LAN IP to do so.
-
Hmm, yeah that could be tricky. Without gateways on the interfaces you don't get repy-to/route-to for traffic passed there so it relies exclusively on static routing.
I would expect Unbound to reply using the IP it received the traffic on but I'm not sure I'vc ever tested that.
-
@stephenw10 said in Static route through an interface with no IP configuration:
I would expect Unbound to reply using the IP it received the traffic on but I'm not sure I'vc ever tested that.
We're not getting to that point since Site A's pfSense just doesn't know which IP to send as when told to reach Site B. From memory, attempting to ping from Site A's pfSense to Site B would fallback to
0.0.0.0as the source IP, so Unbound is probably doing the same thing when it's trying to resolve. If it wasn't dropped by WireGuard (0.0.0.0is not in the allowed subnets,) the other side wouldn't be able to route back a reply anyhow.But that does explain why I'm still able to login from my laptop from Site A to Site B's pfSense—it was contacted on it's LAN address so reasonably reply using it which is enough for routing to deliver.
Realizing I missed the point I wanted to make with the demonstration which helps get across what I'm trying to do in pfSense: if the demonstration was as broken as I wanted with
routeralone being unable to contactnet_b, I could've fixed the problem withip -n router route add 10.20.0.0/24 dev net_b src 10.10.0.1sincesrc 10.10.0.1would've hinted which IP to send as when reaching10.20.0.0/24outside of handling other packets passing through fromnet_a.Going into Static Routing comes to mind, but as we discovered it does not expect you to give a source IP outside of the outgoing interface subnet even if it would route correctly.
Privacy Policy · Cookie Policy