OpenVPN TAP works, but cant access any services on the router

[seewolf]

Hello everyone.

I have multiple pfSense router around, and on two of them i have an OpenVPN tap server (I also have a tun servers ruining, but I prefer tap for some things) running so i can be drooped on the local network.
Last week, after doing absolute nothing (!!) on one of the routers, the tap server stooped working, kinda.

Let me explain, I can still logon with OpenVPN, I can reach all thing on the network, I can ping the router, but i cant access anything else running on the router. I cant logon to the web gui, the ssh is not working, I cant get data from the apcupsd server, nothing.. But I can ping the router.
The router can also ping the client machine on the tap server that has a local IP, but it can not ARPing it.

I verified all the settings, compared it to the exact same settings on the other router with the tap server, I tried to restore an older configuration file, I updated everything to pfSense 2.4.0, and I even crashed the whole machine and installed it from scratch, and then brought up an config file. Nothing helped.

The only time it seems to be working, is when I edit the LAN -> tap_interface bridge.
I edit the bridge so I deselect the tap interface, and then I edit it again by selecting both tap and lan. In those few seconds after adding both interfaces, I can see the webgui, I can access the apcupsd server, and I can reach the ssh from the client. But after a few seconds the thing stops working.

There arent any useful firewall logs, nothing is blocked, and system logs dont seem to indicate anything that could help me solve this.
Again, I can logon, I can do whatever I want on the LAN from the client (multiple clients), but I cant do anything on the router itself except ping it. And the same configuration on the other router is working fine.

Oh yea, when I logon to the tun server, its is fine. Only the tap server broke, and with that bizarre bridge thing that is happening, I suspect something is wrong with the bridging part of it all.

Does anyone have any ideas I can try? Or some experience with stuff like this?
I lost tree days (about 17 working hours) on this by now, and I cant find what is wrong.

Thank you.

[seewolf]

Update for future readers.

The problem seemed to be pfblockerng. Even tough I have the same package with the same configuration on the other router, after removing pfblockerng and reinstalling it (I left the pfblockerng config file on the disk), the tap server started working as it should. I dont know where was the problem, because I reinstalled the whole router witch reinstalled all of the packages of course, but it seems to be working now.

I will do some more testing, rebooting, pulling the ethernet cables out and similar, and if I dont find any issues, I will call it fixed.

[seewolf]

Another update.

The same problem returned.. In the morning everything was working fine, and this afternoon I again lost communication with the router. Same symptoms as before, now I am going to try reinstall pfblockerng and see if it will fix the problem again.
This is getting quite annoying.

[seewolf]

Another another update.

After removing pfblocker package, rebooting the router and then installing the pfblocker package with all the same settings, the openvpn tap server started working as it should.
Quite annoying, but it works again.. Now just to see for how long.

[simon.lock]

What version of pfsense are you using? I'm just wondering if your openvpn tap server is still working if you are now using 2.4.2? I've been struggling to get openvpn tap working on pfsense.

https://forum.pfsense.org/index.php?topic=142446.0

[seewolf]

hi simon.

i am currently on the latest 2.4.2-RELEASE version on all of my routers, and openvpn tap and tun servers are working as they should.
as i see in your thread, it seems that you have a problem bridging the LAN an TAP interfaces? that can be done under interfaces -> assignments -> bridges. you need to configure the LAN TAP bridge there.

[simon.lock]

Thanks for your reply. I've been struggling for many days now and there is no documentation available. I had done that already my interface from the openvpn tap server has been enabled and named "TAP" and is bridged with the LAN interface and the resulting bridge is called BR0. I have assigned BR0 as my Bridge Interface in the openvpn server configuration page. I have also specified Server Bridge DHCP Start and End IPs in the server configuration page and it is these IPs that do not appear in the generated server.conf file. This is where I have my doubts.

Still it's good to know that you have it working properly.

Did you need to set any additional Custom options?
Please could you share your firewall rules needed?
I've been trying to Force all client-generated IPv4 traffic through the tunnel. Have you been able to do this?

[seewolf]

i know there is a lack of documentation on getting tap running in pfsense.

the first time i made a tap server, i actually fallowed documentation on making a tun server, and then tweaking it to make it work as a tap server.

i can make some screenshots tomorrow so you can compare settings with yours.
i did not make any changes to the conf file, i just used the web gui to make all the necessary settings.

for tunneling trough the routers - ie using it as a vpn tunnel so i can exit to the internet trough that router, i have a separate tun server running in parallel with tap server.
every router has 2 openvpn servers, one tap for accessing local network while going to the internet not using a tunnel, and other one is tun for using a pfsense router as an exit node to the internet.

also, tap is not working on android, so that is a reason for making another tun server.

[johnpoz]

What is the point of the tap?  It doesn't work on ios either… I just do not get why anyone would need to run a tap?  What are you doing that requires layer 2?

[seewolf]

@simon.lock:

Thanks for your reply. I've been struggling for many days now and there is no documentation available. I had done that already my interface from the openvpn tap server has been enabled and named "TAP" and is bridged with the LAN interface and the resulting bridge is called BR0. I have assigned BR0 as my Bridge Interface in the openvpn server configuration page. I have also specified Server Bridge DHCP Start and End IPs in the server configuration page and it is these IPs that do not appear in the generated server.conf file. This is where I have my doubts.

Still it's good to know that you have it working properly.

Did you need to set any additional Custom options?
Please could you share your firewall rules needed?
I've been trying to Force all client-generated IPv4 traffic through the tunnel. Have you been able to do this?

hi simon.

here are some screenshots in the attachment of my tap configuration. hope it helps.
unfortunately i do not have a complete tutorial, i learned by trial and error, and after 3 days on a test system i learned how to make it work and then made all the necessary configurations on the main routers.
currently i use the dhcp of the vpn server (.240 - .249), but i plan to configure the server so every client gets the same ip every time.

also, this was a great help then, and if you need a tutorial on making users, certificates and everything else, you can watch this.
https://www.youtube.com/watch?v=xiy52Hn5bTc
i know that the turtorial is for tun mode, but many things are the same.

another useful info for new users - on the home router, where i do not have static ip, i use freemyip.com dynamic DNS.


[seewolf]

@johnpoz:

What is the point of the tap?  It doesn't work on ios either… I just do not get why anyone would need to run a tap?  What are you doing that requires layer 2?

dear johnpoz, actually one part of me was waiting for exactly this reply from you :).

in my case, i want all my clients to be on the same subnet (the vpn server just drops them on the same network), i wanted a 'real' (virtual) network adapter on my clients, and wanted broadcasts to work properly.

[johnpoz]

But Why??  You do understand that is not the most efficient connection.  it is pointless to broadcast traffic over a vpn unless there is specific actual requirement for such a connection and also it adds overhead to every packet..  What is your specific need for this layer 2 traffic?  What is it that is not working that you believe layer 2 is worth the overheard - that is my question..

https://community.openvpn.net/openvpn/wiki/BridgingAndRouting

TAP drawbacks

causes much more broadcast overhead on the VPN tunnel
    adds the overhead of Ethernet headers on all packets transported over the VPN tunnel
    scales poorly
    can not be used with Android or iOS devices

Let me guess your answer is because that is what you want - without any technical reason?  Lets lower the efficiency of the connection because I want to send noise across it so some machine shows up in windows network neighborhood?  Where is the technical need for the tap vs the tun is my ?  That damn curiosity cat of mine is a PITA sometimes ;)

[seewolf]

@johnpoz:

But Why??  You do understand that is not the most efficient connection.  it is pointless to broadcast traffic over a vpn unless there is specific actual requirement for such a connection and also it adds overhead to every packet..  What is your specific need for this layer 2 traffic?  What is it that is not working that you believe layer 2 is worth the overheard - that is my question..

https://community.openvpn.net/openvpn/wiki/BridgingAndRouting

TAP drawbacks

causes much more broadcast overhead on the VPN tunnel
    adds the overhead of Ethernet headers on all packets transported over the VPN tunnel
    scales poorly
    can not be used with Android or iOS devices

Let me guess your answer is because that is what you want - without any technical reason?  Lets lower the efficiency of the connection because I want to send noise across it so some machine shows up in windows network neighborhood?  Where is the technical need for the tap vs the tun is my ?  That damn curiosity cat of mine is a PITA sometimes ;)

i know it is not as efficient as a tun connection, but i am not the only one using it.

a few colleagues running linux machines, and other network connected laboratory test equipment (oscilloscopes, temperature controllers, measurement computers, etc), wanted a tap connection, so i made a tap server for them, and then after a while started using it on my own routers as my primary vpn connection to a remote network.
they wanted to have the same privileges on the network as any other locally connected user (this sentence would make a lot more sense if you saw in how much chaos our work network was till recently).

also, everything is connected over 100M or 1G fiber connection, so a little loss of efficiency is not a big deal here.

[simon.lock]

Thank you seewolf.

Our configurations are very incredibly similar. However, I've been assigning the created bridge interface for the "Bridge Interface" whereas you've assigned the LAN interface directly. Also I've been trying to Force all client-generated IPv4 traffic through the tunnel I'll do some testing and let you know. Thanks also for the YouTube video link. I also found that video useful when setting the tun server. In fact I was the last person to comment on that video 2 weeks ago suggestions a tap tutorial :-) Does the pfsense unbound DNS Resolver resolve hostnames on you LAN?

[seewolf]

@simon.lock:

Thank you seewolf.

Our configurations are very incredibly similar. However, I've been assigning the created bridge interface for the "Bridge Interface" whereas you've assigned the LAN interface directly. Also I've been trying to Force all client-generated IPv4 traffic through the tunnel I'll do some testing and let you know. Thanks also for the YouTube video link. I also found that video useful when setting the tun server. In fact I was the last person to comment on that video 2 weeks ago suggestions a tap tutorial :-) Does the pfsense unbound DNS Resolver resolve hostnames on you LAN?

no problem simon.

i use a tun server for redirecting / forcing all traffic from clients trough the router, that is (as johnpoz pointed out) more efficient. i use tap server when i connect to the internet trough a clients local connection, and then just want to access other computers, servers etc, on a remote network as if i have two (or more) network connections / adapters on the working client. that is why i have all my networks on a different subnet, so i can connect to more of them at the same time while going on the internet locally (ie 10.0.0.0/24, 10.10.0.0/24, 10.20.0.0/24, etc).

the video is a great help for making a tun server.. after you get it working, you can tweak things and see how it is behaving. it takes some time, but you can learn a lot.

i dont use unbound dns resolver for resolving hostnames on my LAN, mostly because all devices have a static IP and i know all there IP addresses (i use an reversible mind logic to assign IP addresses).
and most of the things that need to connect locally or remotely, have in there configurations the static IP addresses of other devices. i like to make a local network working even if the router is it connected to is down (ie no dhcp, dns, etc).

[firbc]

I have kind of same problem with OpenVPN TAP connection. When connected I can access all the local network but not pfsense machine. Which is very wierd. I am on last version of pfSense (2.4.2-RELEASE-p1) and last version of OpenVPN on client side (v2.4.4-I601).

Did you find solution?

[seewolf]

@firbc:

I have kind of same problem with OpenVPN TAP connection. When connected I can access all the local network but not pfsense machine. Which is very wierd. I am on last version of pfSense (2.4.2-RELEASE-p1) and last version of OpenVPN on client side (v2.4.4-I601).

Did you find solution?

hi firbc.

are you running pfblockerng on your sistem? in my case there was some weird conflict with pfblockerng and openvpn. and after an update of the pfblockerng and pfsense everything started working as expected, and i never found the cause of the problem, and with that the real solution.

[firbc]

@seewolf:

@firbc:

I have kind of same problem with OpenVPN TAP connection. When connected I can access all the local network but not pfsense machine. Which is very wierd. I am on last version of pfSense (2.4.2-RELEASE-p1) and last version of OpenVPN on client side (v2.4.4-I601).

Did you find solution?

hi firbc.

are you running pfblockerng on your sistem? in my case there was some weird conflict with pfblockerng and openvpn. and after an update of the pfblockerng and pfsense everything started working as expected, and i never found the cause of the problem, and with that the real solution.

no, the only package I have installed is OpenVPN client export tool…

[firbc]

seewolf, does pinging pfsense machine in your case work? I cannot even ping it. On tun it is working normally.

[seewolf]

@firbc:

seewolf, does pinging pfsense machine in your case work? I cannot even ping it. On tun it is working normally.

at the moment everything works nominally.
when i had the problem, as i said in the first post, i could ping the router and that was the only thing i could do towards the router.

did you try to reinstall the machine?
unfortunately that is the only thing i can suggest at this moment.

and are you sure that the openvpn server configuration is correct?