Some clients can ping lan some can't.
-
@kiokoman Yes this seems to be exactly the same issue. I get ERROR: FreeBSD route add command failed: external program exited with error status:1
Is there another way to fix this rather than using an external script? I'm running a new version of code 2.4.4-RELEASE-p3.
-
i realy don't know, i was searching for a solution for you and i found that post
@jimp do you already have/do you need a bug report for this ? or are you aware of a possible cause and solution ? -
I doubt there is a bug here. Probably something in the setup/config but please don't ping me or other devs directly for this kind of thing.
-
sorry I'm probably putting too much enthusiasm into it :)
-
@kiokoman Some more info.
If I restart openvpn the following routes are added automatically with out any connections to vpn.
10.200.4.1 link#26 UHS lo0
10.200.4.2 link#26 UH ovpns2When the first client that connects it gets 10.200.4.2. Which is the only one that works because the route is already there.
When the second client connects it gets 10.200.4.3. The route add fails on the pfsense. If I manually add the route using route add -host 10.200.4.2 10.200.4.1. The client still can't connect to internal systems. Using packet capture I can see the traffic going inbound only on the openvpn interface. On the Lan when I add the route I can see both the sent packet and the return packet. It seems that the lan isn't routing the packet back to the openvpn interface.
The first client can ping it self and the vpn gateway 10.200.4.1.
The second client can't ping it self or the vpn gateway.Is there a way to see why the route add is failing? It just says returned a status of 1.
-
if i understand it right ..
route add -host 10.200.4.3 -interface ovpns2
before connecting the second cllient -
@kiokoman Hehe I was just googleing that. That works. I can now ping internal hosts. Oddly enough I can now ping myself 10.200.4.3 and the gateway 10.200.4.. This I find odd because that should be L2 from my host.
Is there some way to get more logs regarding why the automatic route add is failing?
-
increase Verbosity level option for openvpn
-
@kiokoman More Info.
Interesting I get the following.
Sep 23 16:43:52 openvpn 19893 ERROR: FreeBSD route add command failed: external program exited with error status: 1
Sep 23 16:43:52 openvpn 19893 /sbin/route add -net 10.200.4.0 10.200.4.2 255.255.255.0This makes sense because 10.200.4.2 exists (not sure why). This is created automatically when openvpn is re/started on the pfsense. I haven't been able to see the error when .3 is given out. The verbosity is high and I can only see 2000 entries. So finding the route is kind of tough.
-
restart openvpn connect the first client, clear the log
connect the second client and see what is logged -
@kiokoman Setup a syslog server and turned up logging to 11. Found some interesting things.
When the server openvpn service is started I see the following entries.
ifconfig_local = '10.200.4.1'
ifconfig_remote_netmask = '255.255.255.0'
route_script = '[UNDEF]'
route_default_gateway = '10.200.4.2'
'route_default_metric = 0
route_noexec = DISABLED
route_delay = 0
route_delay_window = 30
route_delay_defined = DISABLED
route_nopull = DISABLED
route_gateway_via_dhcp = DISABLED
allow_pull_fqdn = DISABLEDserver_network = 10.200.4.0
server_netmask = 255.255.255.0
push_entry = 'route-gateway 10.200.4.1'
push_entry = 'topology subnet'ifconfig_pool_defined = ENABLED
ifconfig_pool_start = 10.200.4.2
ifconfig_pool_end = 10.200.4.253
ifconfig_pool_netmask = 255.255.255.0
ifconfig_pool_persist_filename = '[UNDEF]'
ifconfig_pool_persist_refresh_freq = 600
ifconfig_ipv6_pool_defined = DISABLED
ifconfig_ipv6_pool_base = ::
ifconfig_ipv6_pool_netbits = 0/sbin/ifconfig ovpns2 10.200.4.1 10.200.4.2 mtu 1500 netmask 255.255.255.0 up
/sbin/route add -net 10.200.4.0 10.200.4.2 255.255.255.0
IFCONFIG POOL: base=10.200.4.2 size=252, ipv6=01st Client Connecting getting 10.200.0.2 (Note that is set as the default gateway of the server above)
MULTI_sva: pool returned IPv4=10.200.4.2, IPv6=(Not enabled)
MULTI: Learn: 10.200.4.2 -> user1/XXX.XXX.XXX.XXX:1194
MULTI: primary virtual IP for user1/XXX.XXX.XXX.XXX:1194: 10.200.4.2I don't see a route add on the server side at all. I grep for it and only find this for the client connect
user1/XXX.XXX.XXX.XXX:1194 SENT CONTROL [michael.carey]: 'PUSH_REPLY,route 172.18.0.0 255.255.248.0,route 198.18.0.0 255.255.0.0,route 192.168.4.0 255.255.252.0,route 10.50.0.0 255.255.254.0,route 10.200.0.0 255.255.252.0,dhcp-option DOMAIN xxxxxx.com,dhcp-option DNS 172.18.2.6,dhcp-option DNS 10.200.0.1route-gateway 10.200.4.1,topology subnet,ping 10,ping-restart 60,ifconfig 10.200.4.2 255.255.255.0,peer-id 0' (status=1)
2nd client connecting
MULTI_sva: pool returned IPv4=10.200.4.3, IPv6=(Not enabled)
MULTI: Learn: 10.200.4.3 -> user2/50.208.131.246:5238
MULTI: primary virtual IP for user2/50.208.131.246:5238: 10.200.4.3
Again no route on the server sideonly the push
user/XXX.XXX.XXX.XXX:5238 SENT CONTROL [mike]: 'PUSH_REPLY,route 172.18.0.0 255.255.248.0,route 198.18.0.0 255.255.0.0,route 192.168.4.0 255.255.252.0,route 10.50.0.0 255.255.254.0,route 10.200.0.0 255.255.252.0,dhcp-option DOMAIN xxxxx.com,dhcp-option DNS 172.18.2.6,dhcp-option DNS 10.200.0.1,route-gateway 10.200.4.1,topology subnet,ping 10,ping-restart 60,ifconfig 10.200.4.3 255.255.255.0,peer-id 1' (status=1)
-
Nothing related to "FreeBSD route add command failed: external program exited with error status: 1" ?
The first route already exist so the question is .. why the other are not created if it is needed? I will try to test it on my lab but i first need some hours of sleep -
@kiokoman That does happen when server starts and it runs /sbin/route add -net 10.200.4.0 10.200.4.2 255.255.255.0 only because it is already there. If I delete the route and start the server it doesn't happen. It doesn't happen when the client connects there are no route add commands in the log when the client connects. I figured this out yesterday. It seems like there should be. Is there a way to configure the pool to start with 10.200.4.3? I'm wondering if because it is giving out it's own default route to a client that is what is braking it. I'm not sure why it is taking .1 and .2 and using .2 as it's default route. It doesn't make sense to me. It is behind a carp interface that is on a clustered virtual IP so could it be taking 2 addresses because there are 2 addresses under it on the lan interface?
-
@kiokoman I figured out how to configure the pool. server 10.200.4.0 255.255.255.0 'nopool';ifconfig-pool 10.200.4.3 10.200.4.253. Unfortunately that didn't work. Now no client that connects can ping anything on the other side of the tunnel.
-
ok so that error is only spam on the log, i was able to configure some vm machine to test this but i had the time to connect only 1 client, the route was already there so the first client was working, this evening after work i will check what happen with the second client
-
@kiokoman I figured it out. For some reason I had a static route 10.200.4.0 pointing to the lan interface. I got rid of that and everything worked. I'm not sure why .2 worked at all.
-
glad you found out
did you restart pfsense to see if it is completely solved?
when was that route created ? any guess ? -
I can't restart because it is in production for other purposes. I did restart openvpn a couple times and it worked. I had to have added it since I set it up. It was a couple months ago so I don't remember when or why though. It's a good thing to know that behavior though in case someone else has the issue.
-
@careymichael I am having this same issue. When you said you had a static route pointed to the LAN interface, are you meaning in the firewall rules?
