Pfsense no internet access on Esxi server

KOM

https://doc.pfsense.org/index.php/Connectivity_Troubleshooting

Start at the beginning. What are your pfSense LAN config details re address, mask, DNS etc. What are your DHCP server config details?

craig121

Thanks Kom this should help me identify.

I have just notice that it states

Check that the WAN IP address has the correct subnet mask (Interfaces > WAN)

An improper subnet mask such as /1 could cause connectivity issues to large portions of the Internet, using /32 for a mask can prevent the gateway from being found/used

I am using a 32 mask

route add -net 51.x.x.x/32 -iface em0

Should i use another Mask i am unsure why the tutorial i followed specified a 32 mask?

Or how do i calculate the Mask, i really appreciate your help Kom

Thanks

craig121

I have check the logs and i have this in the routing log:

Sep 19 17:35:24 radvd 37048 invalid all-zeros prefix in /var/etc/radvd.conf, line 9

Line 9 - prefix ::/64 {

KOM

The mask you should use are details from your ISP or colo. Nobody but them can tell you what it is since it's their network you're on.

That radvd error is to do with IPv6.

craig121

Ok i finally got it working, the tutuorial that i originally followed actually caused me all the headache.

I remove the

route add -net 51.x.x.x/32 -iface em0
route add default 51.x.x.x

On startup

Went into the graphically interface and added a gateway to the following ip address. - Hey presto it all worked

Thanks Kom for pointing me to the https://docs.netgate.com/pfsense/en/latest/routing/connectivity-troubleshooting.html

Cheers

KOM

Yeah, I should have mentioned that you don't need to manually fiddle with the routing table or anything, and that those entries you added should be removed. Glad its working for you now.

craig121

Thanks again Kom, I have noticed one that if i dont continually ping the router stops i cant ping or do anything afterwards.

Strange i guess it could be to do with VMware or something. I have checked the logs and nothing in there.

Is there any kind of hibernation or anything that could cause this?

KOM

No, and it shouldn't be doing that. Is it hung or can you access the console?

craig121

Yeah can still access the console, i just cant ping any longer.

KOM

If you can still get to WebGUI, check the System log for anything unusual.

awebster

@craig121 So the netmask on the WAN interface is it /32 now? The default gateway is what?
If /32, it would mean they are not on the same subnet and Layer 2 is going to behave strangely.

I have noticed one that if i dont continually ping the router stops i cant ping or do anything afterwards.

How did you get it to started working again?

craig121

Hi Awebster, i removed the

route add -net 51.x.x.x/32 -iface em0
route add default 51.x.x.x

from the start up scripts.

For me to get it working - i logged in via the GUI (web interface) and changed the WAN interface.

So my current setup is that i have a static ip on the IPv4 address 51.89.243.x my original mask was set to 32. This is ip is a virtual ip from the hosting provider OVH which is has a virtual MAC address, this is set on the ESXI virtual machine NIC adapter (virtual).

I added the IPv4 Upstream gateway - which is set to the ESXi host ip address. In my case 51.89.227.x. So when i added the gateway i had to change the mask to 16 on the IPv4 address as it was out of the subnet.

I may have set this wrong as i believe this give full scope of the 3rd mask as well.

Once i did that hey presto everything works but only for a period of time (around 10 -15mins).

My new problem is that after that period of time it stop being able to access in the WAN and thats even within the Shell of pfSense. I cant even ping google.com.

If i navigate into the Web UI and re save the WAN interface it comes back to life for 10 -15mins.

I am going to have a play around with the ESXi settings in case its anything in there. Overall very impressed with pfSense and will be using it going forward.

craig121

Ok i have check the system logs (General) and found an entry that seems to be causing the issue:

Sep 20 07:27:03 kernel arp.51.89.227.x moved from 00:00:0c:9c:f0:04 to d0:40:d3:01:23

It looks like the MAC is being updated by the host for some strange reason, i will post an update when i have fixed and finalised.

craig121

I have no idea why this address is changing and i am not sure how to resolve or fix. Is there anything in pfSense that can detect and update accordingly.

craig121

I have looked at the Diagnostics / ARP table

Interface IP address MAC address . Hostname
WAN 51.89.227.x 00:00:0c:9f:f3:04 ns34322.ip-xx-xx-x . Expires in 645 seconds

Once this has run out this is when the WAN connection stops working. How do i stop the expiry

awebster

This is an OVH thing. Basically they give you a single /32 and the gateway is on a different subnet.

You might try changing the WAN interface mask back to /32, then going into System > Routing > Gateways, add your default gateway and click on the Advanced settings button and scroll down to the bottom.
Check on Use non-local gateway.
This will allow use of a gateway outside of this interface's subnet.

craig121

I have set as you have suggested and it worked after a reboot.

I suppose now its a question of time to see if it works :)

craig121

Unfortunately the same error - the gateway that i set is to the IP of ESxi server - which is what a tutorial specified.

awebster

@craig121 said in Pfsense no internet access on Esxi server:

Unfortunately the same error - the gateway that i set is to the IP of ESxi server - which is what a tutorial specified.

I'm not sure that setting the gateway to the IP of the ESXi server is correct. Which tutorial did you follow?

craig121

The tutorial that i followed which is very long winded and you have skip a lot is the playlist:

https://www.youtube.com/watch?v=iiQNQNdAAdA&list=PLf0swTFhTI8qOAbrJhihZZTWfZA_arTYZ&index=13