Remote access Layer 2 works, Layer 3 no

johnpoz

@Summer said in Remote access Layer 2 works, Layer 3 no:

the 10.12.225.115 client public ipv4

10.x.x.x is not a public IP, that is rfc1918

Summer

Thanks @johnpoz, my tunnel is using the 10.xxxx too...

This is what see the app Network Analyzer Lite v.3.12:

I can see only pfsense DNS is used not the gateway (even if all flag are setted on the server config) could it be that packet goes randomly out from cell gateway instead vpn?

johnpoz

@Summer how would the vpn connection work without a gateway?

Summer

@johnpoz I'd like to understand it too, you see the upper icon it's linphone a voip client that is connected trought the tunnel using the right gateway.

Also the browser issue that http://SITEA works and http://SITEA/login fail got no sense for me.

And how could it be that http://SITEB/login works while the app show these informations.

This is happening on Android, linux and windows clients -.- But on LINUX and Windows I've solved using Deprecated Layer 2.

johnpoz

@Summer said in Remote access Layer 2 works, Layer 3 no:

http://SITEA works and http://SITEA/login

Those to things are exactly the same to browser - but you could have the browser showing you cache with one, and not able to access the site on the other.

Well with layer 2 you don't need a gateway to talk to Ips in the same network.. fix your no gateway issue.

Summer

@johnpoz thanks for the infos, I'm reading :

https://docs.netgate.com/pfsense/en/latest/vpn/openvpn/assign.html#figure-assign-openvpn-interface

Actually I've no Interface associated to ovpn.
Even the wizard hadn't created this association.

Are those step optional or must be done to get it work?

johnpoz

@Summer you don't need to assign an interface to openvpn when its a server. You only need an interface when its a client..

On your tool you showed, do a trace to an IP on your internal network that you setup a route too..

There is a pretty good write up - in what you linked too..

Did you actually setup a s2s connection, where you setup routes on both ends of the s2s to point to the different networks on each site?

Summer

@johnpoz thanks, client log are same as yours in Traceroute I can see:

ovpn.server
SITEA.1

So the gateway in use should be the correct one even if in main VPN connection info it show N/A.

Now if I open the SITEA/login in the browser I can see: HTTP SERVER LOG:

ovpn_client2 - - [28/Aug/2023:16:48:41 +0200] "GET /login HTTP/1.1" 404 45099 "-" "Mozilla/5.0 (Linux; Android 13; Nokia X10 Build/TKQ1.220807.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/116.0.0.0 Mobile Safari/537.36"
ovpn_client2 - - [28/Aug/2023:16:48:41 +0200] "GET / HTTP/1.1" 403 94 "-" "Mozilla/5.0 (Linux; Android 13; Nokia X10 Build/TKQ1.220807.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/116.0.0.0 Mobile Safari/537.36"
ovpn_client2 - - [28/Aug/2023:16:49:41 +0200] "GET /favicon.ico HTTP/1.1" 404 46502 "http://SITEA.1/" "Mozilla/5.0 (Linux; Android 13; Nokia X10 Build/TKQ1.220807.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/116.0.0.0 Mobile Safari/537.36"

So routing incoming traffic to LAN should be good.

But on Pfsense LOG:

ovpn.client/PFSENSEWAN:40039 MULTI: bad source address from client [10.19.51.88], packet dropped

Now I'm reading but not understanding:

https://serverfault.com/questions/646130/multi-bad-source-address-from-client-any-one-off-solutions

johnpoz

@Summer said in Remote access Layer 2 works, Layer 3 no:

bad source address from client [10.19.51.88],

What is this 10.19.51.88 address? If your remote site routes traffic down to the vpn server, how does the openvpn server know about this 10.19.51 network? If its not a known IP, and it sees traffic from it - then yeah it should drop it.

There is pretty good write up in what you linked too, if you setup a s2s - did you create the appropriate routes on each site for what is available on each site?

Summer

@johnpoz said in Remote access Layer 2 works, Layer 3 no:

What is this 10.19.51.88 address?

This is mobile client CELL CONNECTION (has changed from Information screenshot was 10.12.225.115 ) ip is not External ipv4.

@johnpoz said in Remote access Layer 2 works, Layer 3 no:

f your remote site routes traffic down to the vpn server, how does the openvpn server know about this 10.19.51 network?

If there's a way to look to iroutes of pfsense maybe we can discover it.

@johnpoz said in Remote access Layer 2 works, Layer 3 no:

did you create the appropriate routes on each site for what is available on each site

I've setup Remote Access is not a site to site, just a road warrior that should be able to reach LAN.

johnpoz

@Summer said in Remote access Layer 2 works, Layer 3 no:

This is mobile client CELL CONNECTION

if you have sitea and siteb - then they should a s2s connection. If you are going to have road warriors, ie cell out and about - then those should be a remote access setup..

You can setup both, you can have more than 1 openvpn server running on each pfsense.

If your going to have say phones using the wifi at say siteA, then you need to setup routes for this network the the phones will be on at either site a or site b, c etc..

Summer

@johnpoz said in Remote access Layer 2 works, Layer 3 no:

if you have sitea and siteb - then they should a s2s connection. If you are going to have road warriors, ie cell out and about - then those should be a remote access setup..

That's exactly how it is, now the issue is road warrior can access siteb but not sitea that is pfsense LAN.

@johnpoz said in Remote access Layer 2 works, Layer 3 no:

If your going to have say phones using the wifi at say siteA, then you need to setup routes for this network the the phones will be on at either site a or site b, c etc..

No wifi, only traffic from outside but I've only made a client specific override and with this I can reach SITEB but not SITEA

johnpoz

@Summer said in Remote access Layer 2 works, Layer 3 no:

now the issue is road warrior can access siteb but not sitea that is pfsense LAN.

If you want your road warrior connection to get to some site other than the one they are connecting to for vpn. Then you have to create the routes for whatever tunnel network the roadwarrior is going to use.

If some rw IP comes in say 10.10.10.x and wants to go to site A from site B, even if site B has route to go to site A. how does site A know to get back?

Summer

@johnpoz said in Remote access Layer 2 works, Layer 3 no:

Then you have to create the routes for whatever tunnel network the roadwarrior is going to use.

I don't undertstand route should be created by client specific overrides, however these are current routes:

based on this:

if roadwarrior.2 (10.7.31.2) open http://SITEA (http://10.7.208.1) page is loaded.
if roadwarrior.2 (10.7.31.2) open http://SITEA/login (http://10.7.208.1/login) error:

ovpn.client/192.168.100.100:40039 MULTI: bad source address from client [10.19.51.88], packet dropped

instead if roadwarrior.2 (10.7.31.2) open http://SITEB/login (http://10.7.218.1/login) correctly load the page.

Could it be that default Gateway 192.168.100.100 that is managed here interfere with SITEA routing?

johnpoz

@Summer said in Remote access Layer 2 works, Layer 3 no:

if roadwarrior.2 (10.7.31.2) open http://SITEA/login (http://10.7.208.1/login) error:

Again not sure why you think that has anything to do with some pfsense problem.. to this road warrior there is no difference between http://something and http://something/otherthing/etc

There is no difference in those when it comes to networking that something will resolve to an IP.. the IP is the same in both.. what the full url is has nothing to do with anything unless your running through a proxy.. IP is IP the only thing that is in play here.. The syn will be sent to whatever that something IP is.. anything that happens after that has zero to do with routing..

Be it you do a get to just something or something/otherthing has nothing to do with routing or firewall rules since they are both to the same IP, which is whatever that something resolves too.

Now it could be a permissions thing on whatever is serving up something httpd.. But from a networking perspective those 2 things are exactly the same.. the something is an IP, which would be the same in either.

Summer

@johnpoz said in Remote access Layer 2 works, Layer 3 no:

Now it could be a permissions thing on whatever is serving up something httpd.. But from a networking perspective those 2 things are exactly the same.. the something is an IP, which would be the same in either.

I'm just observing this behavior with layer 3 vpn, thanks for confirming this should not happen.

So routes are all here and they seems fine.

So next step for debug is turn off layer3 and androids clients and wait for next pfsense upgrade.

johnpoz

@Summer said in Remote access Layer 2 works, Layer 3 no:

So next step for debug is turn off layer3 and androids clients and wait for next pfsense upgrade.

Sure you do you - but if you can get to http://something and not http://something/else has zero to do with being layer2 or layer3 or anything at all to do with openvpn.. So you could wait for the next 20 upgrades, not going to fix whatever problem is your seeing.

That is not a networking/openvpn issue -- to networking, firewalling and openvpn those 2 things are exactly the same. Something resolves to an IP or it doesn't - talking to that IP is no different if its just something or something/otherthing

Now you could have a permissions thing on your httpd server, where client from IP has permission to the root folder that something is suppose to serve up, but something/other that IP does not. This could explain why both work with layer2 because your going to be on the same IP the httpd is on most likely. But when your routed your going to be from different IP than the httpd server. And maybe this other IP on the httpd doesn't have permission to look at something/other, but it does have permission to look at just something.

what version your running of pfsense would have zero to do with that..

Summer

@johnpoz thank you for the advices

@johnpoz said in Remote access Layer 2 works, Layer 3 no:

But when your routed your going to be from different IP than the httpd server. And maybe this other IP on the httpd does have permission to look at something/other, but it does have permission to look at just something.

I will inspect but based on pfsense log MULTI : bad source the request got to http server host and it register to access log not error log then I suppose packet is dropped by pfsense but there's a way to understand if this is coming from Firewall rules or elsewhere?

@johnpoz said in Remote access Layer 2 works, Layer 3 no:

what version your running of pfsense would have zero to do with that..

I Hope it, but layer3 from pfsense 22.01 version with same config showed on GUI and road warriors where working.

johnpoz

@Summer Again from a networking perspective, routing/firewalling even openvpn there is NO difference between http://something and http://something/otherthing

The only thing that comes into play with both is the IP that something resolves too.. pfsense, the firewall nor openvpn have any care to what the full url is, only the IP the "something" resolves too.. It is not possible for something to resolve to something different if you go to something/other

Now if you were running traffic through proxy - the proxy would or could care and could do things differently depending on the full url. Your httpd could care what IP your coming from when talking to something/other vs just something.

maybe when you go to something/other you getting redirected to some other IP?? on the httpd server? But from a networking perspective that something is going to resolve to an IP.. What the full url is has nothing to do with if you can talk to that IP or not through your routing / firewalling

Summer

@johnpoz said in Remote access Layer 2 works, Layer 3 no:

Now if you were running traffic through proxy - the proxy would or could care and could do things differently depending on the full url

Is there any Cache / Proxy enabled by default, how can I look from command line if there's anything running this kind of controls?

I've found docs about Reverse Proxy this could be the case but I don't have any package installed.