VMs traffic doesn't go through the right gateway
-
Hi all,
I have set up my pfSense on Esxi as a VM, it also has freenas and ubuntu set up as VMs.
I have OpenVPN (PIA) setup on pfsense which works fine. I have 2 physical ports on my machine setup as:WAN > physical LAN port 1> vSwitch0 > port group WAN connected to pfSenese only as WAN
LAN devices through an ethernet switch > physical LAN port 2> vSwitch1 > port group LAN connected to pfSenese as LAN + other VMs
Now, I am having a really interesting issue. I decided to exempt some devices' traffic from going through the VPN.
So I created a rule on LAN interface that has a source as the devices I want to exempt and set the gateway to the WAN-DHCP.The funny thing is the rule works for physical (real) devices such as phones and tablets. but the VMs traffics (ubuntu and freenas) still goes through the VPN!
I have logged the traffic and I can see that pfsense does pick up the traffic from this devices by the exemption rule but somehow the gateway still is the openVPN.
so it perfectly works for real devices but VMs traffic still goes through the vpn even though it has been associated to the rule!I have no idea what is going on here!?
I would appreciate any help on this as it is bugging me so much since I cant figure it out!
Thanks
-
Just something I forgot to mention if I set the action for the rule to block or reject it does block and reject the traffic from the VMs! but if set it to pass doesn't apply the gateway correctly for VMs! so weird!
-
When you change rules you have to make sure you clear any states that might exist for the previous rules.. This is any firewall, an existing state would supersede any rules.
-
If you mean resetting states, I have tried that, it doesn't work.
-
Well your going to have to post your rules so we can see what else might be wrong. I have run pfsense for years and years on esxi.. Never any sort of issues.
-
So these are pictures of my LAN, WAN and the Alias I set up. There is nothing in floating and openvpn.
currently, the alias only has plex in it.https://ibb.co/mVbaFJ
https://ibb.co/f319aJ
https://ibb.co/bTWmNyEdit:
This is system log for firewall. You see the rule is applied to the traffic but somehow plex still connected through VPN!
https://ibb.co/kPbvhy -
That is NOT the way I would do it.
When you setup your vpn client on pfsense, do not grab routes.. Block it from adding routes and then just use policy routing to send traffic where you want.
How are you thinking your plex is using your vpn?
-
I used the instructions from the PIA website, sorry im new to all of this. I will try to google what you said to see if I can figure setting up that way.
I know plex still using VPN because it says in plex client and also when check my public IP through SSH in freenas ( second pic)
https://ibb.co/iqcXvJ
https://ibb.co/mczCvJBut it is not just plex, Freenas itself also shows the same behaviour and also another ubuntu VM on esxi
Thanks for your help so far!
-
yeah pia instructions are for EVERYTHING to use them.. Don't grab default route, and just policy route what you want to go through vpn and what you want to not use vpn.
Not sure where you get that idea that plex is going your vpn... Thats your browser getting that IP.. So whatever machine your running your web interface to plex is where that public IP is coming from.
Your running that web browser on your plex box? hitting the local.
-
That web browser is not on the plex box. but the IP is the public IP address of the plex box as it can also be seen from the SSH image. So even though I set the gateway to my actual public IP through the rule but whatsmyip.com still sees plex's IP as my VPN IP.
The weird thing is the fact that real devices like my phone and tablet are absolutely fine so if I add them to the noVPN alias when I check my IP it is actually my ISP IP. but the VMs behave differently this is soo odd!
-
So from a cmd line of your plex VM... Lets see it traceroute to some public IP..
