pfsense 2.5.2-Release (amd64) - routing table default gateway entry (erratically) missing
-
I am in the process of building a vanilla pfsense 2.5.2-Release (amd64) install for a friend on non Netgate hardware (hereinafter named "TEST"). I am using a Jetway JBC313 N3160 - its hardware I have successfully used for pfsense in the past and also use for this current release. However with this TEST build I have run into an issue with routing - namely a default IPv4 gateway is not always set post TEST pfsense reboot. Note IPv6 on TEST has been DISABLED.
I wonder if there is any additional monitoring I can enable or specific log files I can review to try and identify why a default gateway is not always set post restart ?
Note I have attempted a number of fresh vanilla (re) builds of TEST but the problem repeats. As this is being used in a home environment its an ongoing irritation rather than a dire problem!
Current setup.
As its a TEST build its currently incorporated into my local home network thus;
TEST WAN - DHCP associated IP is allocated by my home "production" pfsense box. This results in the TEST WAN Address being allocated an IP within the range 192.168.123.0/24. Note as regards the TEST WAN interface on the vanilla build I have unticked (ie disabled) the "Block Private Networks and loop back addresses"
TEST LAN - has been left as the standard pfsense 192.168.1.0/24 range. Ditto DHCP.
No upstream/forwarding DNS server has been set - I want the TEST pfsense install to act as a fully resolving DNS Server (via Unbound)
So far so good - I can browse the internet normally from a client attached to the TEST pfsense network (192.168.1.0/24)
My next step is to configure an outbound OpenVPN Client on TEST pfsense to an external commercial VPN provider. This successfully connects.
In this regard I have configured a static route (System/Routing/Static Routes to ensure that traffic destined to the inbound gateway IP address of the VPN provider is always routed via the TEST WAN interface, not default gateway TEST VPN interface (catch 22 otherwise!)
I wish to route all my TEST LAN outbound traffic out via this VPN interface not the default TEST WAN interface (above static route excepted)
Within Firewall/NAT/Outbound (have selected option "Manual Outbound NAT Rule Generation") I have DISABLED the existing auto created LAN to WAN NAT rule (192.168.1.0/24 -> WAN Address. Within same console I have CREATED A new NAT rule 192.168.1.0/24 -> VPN Interface of Commercial VPN Provider.
Within System/Routing/Gateways I have set the default gateway to be the Interface associated with the outbound TEST OpenVPN Client ie NOT WAN address
As regards DNS (Services/DNS Resolver/General Settings) I have set Outgoing Network Interfaces to Local host and TEST VPN Interface of Commercial VPN provider ie I am forcing all DNS queries to route via VPN interface NOT WAN.
Post configuration all seems to work as anticipated. All traffic is routed via VPN, address resolution normal etc. No DNS leaks
However if I reboot or halt system, upon restart I erratically lose routing. Sometimes it operates normally and on other occasions not. There is no consistency as sometimes I can reboot multiple times and it works fine, but then post a fresh restart if fails. When it fails viewing Diagnostics/Routes shows under the iPV4 Routes an ABSENCE of any "default" route under destination.
Expected behaviour is a "default" route showing the Internal IP address allocated by my Commercial VPN provider (10.55.40.x)
I can easily resolve the issue by going back to System/Routing (where VPN Interface is still correctly shown as default gateway) and clicking Save and then Apply Changes at which point the missing default gateway entry reappears within Diagnostics/Routes and normal internet access resumes.
I should add that I currently use this type of setup (with same VPN provider) on two identical hardware/software pfsense boxes without issue. The only differences I can think of are (1) they have been upgraded from early revisions of pfsense to 2.5.2 and (2) their outbound WAN interfaces use PPP0E rather than DHCP
Any suggestions appreciated!
Cheers
Alan
-
@farrina said in pfsense 2.5.2-Release (amd64) - routing table default gateway entry (erratically) missing:
Within System/Routing/Gateways I have set the default gateway to be the Interface associated with the outbound TEST OpenVPN Client ie NOT WAN address
You should not set a static route to a VPN gateway and above all not the default route!
Most VPN providers push the default route to the clients anyway. If yours doesn't you can enter "0.0.0.0/0" respectively "::/0" at "Remote Networks" to set it, when the connection is up.
-
A rather belated response on my part to thank you for your answer to my question.
Part of the reason for the delay was ongoing testing and wider learning on my part. Factoring in your configuration guidance I have ultimately achieved what I was attempting to do.
The only other "gotcha" (which took me some time to identify) was DNS clients not resolving post pfsense reboot. This appears to be an issue with Unbound and fixed by restarting the Unbound service post reboot. Ultimately I have automated this by way of a CRON job @reboot
Thanks again for your help
-
@farrina
I corrected the v4 route above, typed it wrong before.The only other "gotcha" (which took me some time to identify) was DNS clients not resolving post pfsense reboot.
This may happen, when the default route is pointing to the VPN gateway, before the gateway is reachable at all.
Therefore it's not a good idea to set it static. When doing it in the VPN settings, the OpenVPN client can resolve the servers name before, then he is establishing the connection and changing the route. And all works as it should. -
Thanks for your further interest in my circumstances.
Just to clarify, following your advice (hopefully I have fully understood) I made the following changes
System/Routing/Gateways
Changed default IPv4 gateway to automatic (IPv6 is none)
System/Routing/Static Routes
Removed static routes
System/Routing/Gateway Groups
Created a Gateway Group comprising the two outbound VPN Clients (set as Tier1/Tier2). Routing out via this GG is configured via LAN Firewall Rules (previously I thought this was undertaken by way of the above gateway setting, but I now recognise this seems to be only appropriate to change this where there are two physical WAN connections.
You may recall that I have configured Unbound to only use the 2 outbound VPN's for DNS resolution (looking to avoid any possibility of a DNS leak). To negate the possibility of a catch 22 (DNS waiting for VPN to come up/VPN Client seeking DNS response to Gateway query) I have hard coded the IP address of the VPN Gateways into the OpenVPN clients.
Not withstanding the above I still seemed to be afflicted by a loss of local client DNS resolution post pfsense reboot.
This does seem to be an know issue (possibly with Unbound) and whilst a bit of a "bodge" seems to be easily resolved by restarting Unbound reboot using CRON.
At the risk of "teaching you how to suck an egg", I found this discussion here helpful
https://www.reddit.com/r/PFSENSE/comments/lxu3yg/workaround_unbound_restart_at_reboot_using_cron/
Thanks again for your help.