Remote access Layer 2 works, Layer 3 no
-
Hi,
I've setup a PFSENSE OPENVPN SERVER but if I use Layer 3 only some service works:
- HTTP request works only if no path is specified i.e. http://SITEA/ WORKS but http://SITEA/login NO.
- SITEA file share samba access would no t work
SITEA is pfsense LAN
SITEB is another client remote vpn network and the client connected via layer 3 reach SITEB (both http://SITEB/login and file share works)These are the route I can see on client:
- connected to internet:
IPv4 Tabella route =========================================================================== Route attive: Indirizzo rete Mask Gateway Interfaccia Metrica 0.0.0.0 0.0.0.0 192.168.220.247 192.168.220.187 55 127.0.0.0 255.0.0.0 On-link 127.0.0.1 331 127.0.0.1 255.255.255.255 On-link 127.0.0.1 331 127.255.255.255 255.255.255.255 On-link 127.0.0.1 331 192.168.56.0 255.255.255.0 On-link 192.168.56.1 281 192.168.56.1 255.255.255.255 On-link 192.168.56.1 281 192.168.56.255 255.255.255.255 On-link 192.168.56.1 281 192.168.220.0 255.255.255.0 On-link 192.168.220.187 311 192.168.220.187 255.255.255.255 On-link 192.168.220.187 311 192.168.220.255 255.255.255.255 On-link 192.168.220.187 311 224.0.0.0 240.0.0.0 On-link 127.0.0.1 331 224.0.0.0 240.0.0.0 On-link 192.168.56.1 281 224.0.0.0 240.0.0.0 On-link 192.168.220.187 311 255.255.255.255 255.255.255.255 On-link 127.0.0.1 331 255.255.255.255 255.255.255.255 On-link 192.168.56.1 281 255.255.255.255 255.255.255.255 On-link 192.168.220.187 311-connected with layer 3:
IPv4 Tabella route =========================================================================== Route attive: Indirizzo rete Mask Gateway Interfaccia Metrica 0.0.0.0 0.0.0.0 192.168.220.247 192.168.220.187 55 TUNNEL.1 255.255.255.255 TUNNEL.5 TUNNEL.6 259 TUNNEL.4 255.255.255.252 On-link TUNNEL.6 259 TUNNEL.6 255.255.255.255 On-link TUNNEL.6 259 TUNNEL.7 255.255.255.255 On-link TUNNEL.6 259 SITEA.0 255.255.255.0 TUNNEL.5 TUNNEL.6 259 SITEB.0 255.255.255.0 TUNNEL.5 TUNNEL.6 259 127.0.0.0 255.0.0.0 On-link 127.0.0.1 331 127.0.0.1 255.255.255.255 On-link 127.0.0.1 331 127.255.255.255 255.255.255.255 On-link 127.0.0.1 331 192.168.56.0 255.255.255.0 On-link 192.168.56.1 281 192.168.56.1 255.255.255.255 On-link 192.168.56.1 281 192.168.56.255 255.255.255.255 On-link 192.168.56.1 281 192.168.220.0 255.255.255.0 On-link 192.168.220.187 311 192.168.220.187 255.255.255.255 On-link 192.168.220.187 311 192.168.220.255 255.255.255.255 On-link 192.168.220.187 311 224.0.0.0 240.0.0.0 On-link 127.0.0.1 331 224.0.0.0 240.0.0.0 On-link 192.168.56.1 281 224.0.0.0 240.0.0.0 On-link TUNNEL.6 259 224.0.0.0 240.0.0.0 On-link 192.168.220.187 311 255.255.255.255 255.255.255.255 On-link 127.0.0.1 331 255.255.255.255 255.255.255.255 On-link 192.168.56.1 281 255.255.255.255 255.255.255.255 On-link TUNNEL.6 259 255.255.255.255 255.255.255.255 On-link 192.168.220.187 311 =========================================================================== >tracert TUNNEL.5 Traccia instradamento verso TUNNEL.5 su un massimo di 30 punti di passaggio 1 47 ms 40 ms 25 ms TUNNEL.1 2 * * * Richiesta scaduta. 3 * * * Richiesta scaduta.- connected with layer 2
IPv4 Tabella route =========================================================================== Route attive: Indirizzo rete Mask Gateway Interfaccia Metrica 0.0.0.0 0.0.0.0 192.168.220.247 192.168.220.187 55 TUNNEL.0 255.255.255.0 On-link TUNNEL.2 259 TUNNEL.2 255.255.255.255 On-link TUNNEL.2 259 TUNNEL.255 255.255.255.255 On-link TUNNEL.2 259 SITEA.0 255.255.255.0 TUNNEL.1 TUNNEL.2 259 SITEB.0 255.255.255.0 TUNNEL.1 TUNNEL.2 259 127.0.0.0 255.0.0.0 On-link 127.0.0.1 331 127.0.0.1 255.255.255.255 On-link 127.0.0.1 331 127.255.255.255 255.255.255.255 On-link 127.0.0.1 331 192.168.56.0 255.255.255.0 On-link 192.168.56.1 281 192.168.56.1 255.255.255.255 On-link 192.168.56.1 281 192.168.56.255 255.255.255.255 On-link 192.168.56.1 281 192.168.220.0 255.255.255.0 On-link 192.168.220.187 311 192.168.220.187 255.255.255.255 On-link 192.168.220.187 311 192.168.220.255 255.255.255.255 On-link 192.168.220.187 311 224.0.0.0 240.0.0.0 On-link 127.0.0.1 331 224.0.0.0 240.0.0.0 On-link 192.168.56.1 281 224.0.0.0 240.0.0.0 On-link TUNNEL.2 259 224.0.0.0 240.0.0.0 On-link 192.168.220.187 311 255.255.255.255 255.255.255.255 On-link 127.0.0.1 331 255.255.255.255 255.255.255.255 On-link 192.168.56.1 281 255.255.255.255 255.255.255.255 On-link TUNNEL.2 259 255.255.255.255 255.255.255.255 On-link 192.168.220.187 311 =========================================================================== Route permanenti: NessunaI've switched to LAYER 2 the openvpn server config and now all works fine, but I'd like to understand why (also because with previous pfsense 22.01 version layer 3 was working).
-
@Summer this is on Windowss 10 openvpn client.
If I try to import profile on Android it fail with error:
and it's fine: [https://openvpn.net/faq/why-does-the-app-not-support-tap-style-tunnels/]( no layer 2 on android)Instead old layer 3 show:
tun_prop_error: ifconfig addresses are not in the same /30 subnet topology net30 -
Now I've enabled level 5 debug with LAYER 3:
ovpn_client/WANPFSENSE:13211 MULTI: bad source address from client [10.12.225.115], packet dropped message repeated 2 times: [ ovpn_client/WANPFSENSE:13211 MULTI: bad source address from client [10.12.225.115], packet dropped] ovpn_client/WANPFSENSE:13211 MULTI: bad source address from client [10.12.225.115], packet droppedI don't understand why the 10.12.225.115 client public ipv4 is showed here, it's not the OVPN TUNNEL.
-
@Summer said in Remote access Layer 2 works, Layer 3 no:
the 10.12.225.115 client public ipv4
10.x.x.x is not a public IP, that is rfc1918
-
Thanks @johnpoz, my tunnel is using the 10.xxxx too...
This is what see the app Network Analyzer Lite v.3.12:
I can see only pfsense DNS is used not the gateway (even if all flag are setted on the server config) could it be that packet goes randomly out from cell gateway instead vpn?
-
@Summer how would the vpn connection work without a gateway?
-
@johnpoz I'd like to understand it too, you see the upper icon it's linphone a voip client that is connected trought the tunnel using the right gateway.
Also the browser issue that http://SITEA works and http://SITEA/login fail got no sense for me.
And how could it be that http://SITEB/login works while the app show these informations.
This is happening on Android, linux and windows clients -.- But on LINUX and Windows I've solved using Deprecated Layer 2.
-
@Summer said in Remote access Layer 2 works, Layer 3 no:
http://SITEA works and http://SITEA/login
Those to things are exactly the same to browser - but you could have the browser showing you cache with one, and not able to access the site on the other.
Well with layer 2 you don't need a gateway to talk to Ips in the same network.. fix your no gateway issue.
-
@johnpoz thanks for the infos, I'm reading :
https://docs.netgate.com/pfsense/en/latest/vpn/openvpn/assign.html#figure-assign-openvpn-interface
Actually I've no Interface associated to ovpn.
Even the wizard hadn't created this association.Are those step optional or must be done to get it work?
-
@Summer you don't need to assign an interface to openvpn when its a server. You only need an interface when its a client..
On your tool you showed, do a trace to an IP on your internal network that you setup a route too..
There is a pretty good write up - in what you linked too..
Did you actually setup a s2s connection, where you setup routes on both ends of the s2s to point to the different networks on each site?
-
@johnpoz thanks, client log are same as yours in Traceroute I can see:
- ovpn.server
- SITEA.1
So the gateway in use should be the correct one even if in main VPN connection info it show N/A.
Now if I open the SITEA/login in the browser I can see: HTTP SERVER LOG:
ovpn_client2 - - [28/Aug/2023:16:48:41 +0200] "GET /login HTTP/1.1" 404 45099 "-" "Mozilla/5.0 (Linux; Android 13; Nokia X10 Build/TKQ1.220807.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/116.0.0.0 Mobile Safari/537.36" ovpn_client2 - - [28/Aug/2023:16:48:41 +0200] "GET / HTTP/1.1" 403 94 "-" "Mozilla/5.0 (Linux; Android 13; Nokia X10 Build/TKQ1.220807.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/116.0.0.0 Mobile Safari/537.36" ovpn_client2 - - [28/Aug/2023:16:49:41 +0200] "GET /favicon.ico HTTP/1.1" 404 46502 "http://SITEA.1/" "Mozilla/5.0 (Linux; Android 13; Nokia X10 Build/TKQ1.220807.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/116.0.0.0 Mobile Safari/537.36"So routing incoming traffic to LAN should be good.
But on Pfsense LOG:
ovpn.client/PFSENSEWAN:40039 MULTI: bad source address from client [10.19.51.88], packet droppedNow I'm reading but not understanding:
https://serverfault.com/questions/646130/multi-bad-source-address-from-client-any-one-off-solutions
-
@Summer said in Remote access Layer 2 works, Layer 3 no:
bad source address from client [10.19.51.88],
What is this 10.19.51.88 address? If your remote site routes traffic down to the vpn server, how does the openvpn server know about this 10.19.51 network? If its not a known IP, and it sees traffic from it - then yeah it should drop it.
There is pretty good write up in what you linked too, if you setup a s2s - did you create the appropriate routes on each site for what is available on each site?
-
@johnpoz said in Remote access Layer 2 works, Layer 3 no:
What is this 10.19.51.88 address?
This is mobile client CELL CONNECTION (has changed from Information screenshot was 10.12.225.115 ) ip is not External ipv4.
@johnpoz said in Remote access Layer 2 works, Layer 3 no:
f your remote site routes traffic down to the vpn server, how does the openvpn server know about this 10.19.51 network?
If there's a way to look to iroutes of pfsense maybe we can discover it.
@johnpoz said in Remote access Layer 2 works, Layer 3 no:
did you create the appropriate routes on each site for what is available on each site
I've setup Remote Access is not a site to site, just a road warrior that should be able to reach LAN.
-
@Summer said in Remote access Layer 2 works, Layer 3 no:
This is mobile client CELL CONNECTION
if you have sitea and siteb - then they should a s2s connection. If you are going to have road warriors, ie cell out and about - then those should be a remote access setup..
You can setup both, you can have more than 1 openvpn server running on each pfsense.
If your going to have say phones using the wifi at say siteA, then you need to setup routes for this network the the phones will be on at either site a or site b, c etc..
-
@johnpoz said in Remote access Layer 2 works, Layer 3 no:
if you have sitea and siteb - then they should a s2s connection. If you are going to have road warriors, ie cell out and about - then those should be a remote access setup..
That's exactly how it is, now the issue is road warrior can access siteb but not sitea that is pfsense LAN.
@johnpoz said in Remote access Layer 2 works, Layer 3 no:
If your going to have say phones using the wifi at say siteA, then you need to setup routes for this network the the phones will be on at either site a or site b, c etc..
No wifi, only traffic from outside but I've only made a client specific override and with this I can reach SITEB but not SITEA
-
@Summer said in Remote access Layer 2 works, Layer 3 no:
now the issue is road warrior can access siteb but not sitea that is pfsense LAN.
If you want your road warrior connection to get to some site other than the one they are connecting to for vpn. Then you have to create the routes for whatever tunnel network the roadwarrior is going to use.
If some rw IP comes in say 10.10.10.x and wants to go to site A from site B, even if site B has route to go to site A. how does site A know to get back?
-
@johnpoz said in Remote access Layer 2 works, Layer 3 no:
Then you have to create the routes for whatever tunnel network the roadwarrior is going to use.
I don't undertstand route should be created by client specific overrides, however these are current routes:
based on this:
- if roadwarrior.2 (10.7.31.2) open http://SITEA (http://10.7.208.1) page is loaded.
- if roadwarrior.2 (10.7.31.2) open http://SITEA/login (http://10.7.208.1/login) error:
ovpn.client/192.168.100.100:40039 MULTI: bad source address from client [10.19.51.88], packet droppedinstead if roadwarrior.2 (10.7.31.2) open http://SITEB/login (http://10.7.218.1/login) correctly load the page.
Could it be that default Gateway 192.168.100.100 that is managed here interfere with SITEA routing?
-
@Summer said in Remote access Layer 2 works, Layer 3 no:
if roadwarrior.2 (10.7.31.2) open http://SITEA/login (http://10.7.208.1/login) error:
Again not sure why you think that has anything to do with some pfsense problem.. to this road warrior there is no difference between http://something and http://something/otherthing/etc
There is no difference in those when it comes to networking that something will resolve to an IP.. the IP is the same in both.. what the full url is has nothing to do with anything unless your running through a proxy.. IP is IP the only thing that is in play here.. The syn will be sent to whatever that something IP is.. anything that happens after that has zero to do with routing..
Be it you do a get to just something or something/otherthing has nothing to do with routing or firewall rules since they are both to the same IP, which is whatever that something resolves too.
Now it could be a permissions thing on whatever is serving up something httpd.. But from a networking perspective those 2 things are exactly the same.. the something is an IP, which would be the same in either.
-
@johnpoz said in Remote access Layer 2 works, Layer 3 no:
Now it could be a permissions thing on whatever is serving up something httpd.. But from a networking perspective those 2 things are exactly the same.. the something is an IP, which would be the same in either.
I'm just observing this behavior with layer 3 vpn, thanks for confirming this should not happen.
So routes are all here and they seems fine.
So next step for debug is turn off layer3 and androids clients and wait for next pfsense upgrade.
-
@Summer said in Remote access Layer 2 works, Layer 3 no:
So next step for debug is turn off layer3 and androids clients and wait for next pfsense upgrade.
Sure you do you - but if you can get to http://something and not http://something/else has zero to do with being layer2 or layer3 or anything at all to do with openvpn.. So you could wait for the next 20 upgrades, not going to fix whatever problem is your seeing.
That is not a networking/openvpn issue -- to networking, firewalling and openvpn those 2 things are exactly the same. Something resolves to an IP or it doesn't - talking to that IP is no different if its just something or something/otherthing
Now you could have a permissions thing on your httpd server, where client from IP has permission to the root folder that something is suppose to serve up, but something/other that IP does not. This could explain why both work with layer2 because your going to be on the same IP the httpd is on most likely. But when your routed your going to be from different IP than the httpd server. And maybe this other IP on the httpd doesn't have permission to look at something/other, but it does have permission to look at just something.
what version your running of pfsense would have zero to do with that..
