New Config Loses Connectivity in Seconds

[The_Boss]

I have configured a VM in VirtualBox for proof of concept. After setting the LAN IP to 192.168.0.1 for real-world use, I have connectivity only for a few minutes or even seconds - then no internet, and no pfSense GUI via the LAN IP, only the VM console. I am sure I have made a silly mistake somewhere.

Here is my basic setup and status:

When it fails, the interface status shows that LAN/WAN are up:

Here is a System log with a couple fails and reboots:
system log.txt

Thank you for your support with my first-time effort.

[stephenw10]

Ok, you need to check exactly what's failing and what isn't when this happens.

It seems like you still have WAN connectivity as you can ping out, by IP and URL?

Can you ping LAN side resources from the pfSense console?

Check the ARP table in pfSense and on a LAN side client, are they correct?

You might have some asymmetry here, that would fit the times you're seeing.
How is the network connected here?

Steve

[The_Boss]

@stephenw10
After it goes down, there is no ping response on LAN or WAN interfaces. At the same time, my desktop can still ping LAN devices fine of course.

The basic config is: Fiber ONT->pfSense VM->LAN (Switches/APs, etc.)

A route test in failing mode:

Ping:

Addresses:

At least to me, the ARP table looks OK:
Failing mode:

From a basic config VM on the same LAN, working:

I am reading about asymmetric routing, but the issue is net yet clear to me.

Thanks for your help with this.

[stephenw10]

Hmm, OK. So you can't ping anything from either WAN or LAN from the console after the failure happens?

Do the pings just show a timeout?

That starts to feel like a hypervisor problem.

[NollipfSense]

Please mask your gateway. How do you configured VirtualBox, is it like this? How do you access LAN, is it on a network you have? Did you assigned it a static address/24?
Both Wan and LAN should be on the same network but LAN set to static with no DHCP. You should never see your real gateway in VirtualBox...just the host IP or if bridge adapter mode, it will get an IP from your real LAN via DHCP.

Now that I am home, I can show what I said above, below...here is VirtualBox with WAN and LAN on same network with LAN on static IP/24 and access to webGUI.

You should be able to configure your firewall to access the other two networks when you access LAN.

[stephenw10]

If the WAN bridged adapter is connected to a modem you will see a real IP on there.
You would need to be sure the host OS is not using that NIC.

Since it works at all and it's set as DHCP it looks like it's correctly pulling an IP from the ISP.

I now note that the gateway shown in the route is different to that in the ARP table but that could simply have changed across a reboot? If not that's a problem.

Steve

[NollipfSense]

@stephenw10 said in New Config Loses Connectivity in Seconds:

If the WAN bridged adapter is connected to a modem you will see a real IP on there.
You would need to be sure the host OS is not using that NIC.

I see, he's directly connected {Fiber ONT->pfSense VM->LAN (Switches/APs, etc.)}

[The_Boss]

@stephenw10 @NollipfSense

After it fails, via the console, there is no ping response from WAN or LAN IP addresses. WAN shows 100% packet loss. LAN shows:

My VirtualBox network config works reliably in other scenarios, and is follows. I will try switching to VirtIO later.

I did my homework, and the Host OS has no access to the WAN NIC:

I had enough functionality to save a backup from the GUI, When I ran SpeedTest.net, the interfaces would stop talking as soon as the bandwidth topped out at 250M.

I imported the fully-configured settings into my super-basic VM that was working, and shockingly networking WORKS! I guess it was a hypervisor issue, sorry for that. I believe settings are the same so the root cause is not clear. Anyway, at least I have basic function. Thanks for your support with this!

[stephenw10]

Mmm, probably.
'Host is down' implies the local device is not responding to ARP. So I would have expected to see it missing from or incomplete in the ARP table.

Steve

[The_Boss]

@stephenw10
The ping test could have been after ARP expiration.

Once the network failures were history, I tried switching to VirtIO, but speed was 35M even with offloading disabled. I set it back to Bridged, and got 250M with 4 CPUs. 2 CPUs in Virtualbox gave me over 500M. Not bad, but still ~50%.

Each reboot I would lose connectivity on the i211 LAN and the only way I could get it to work was to switch it promiscuous mode on/off while the VM was running. Crap...

I gave up on Virtualbox and moved it over to Hyper-V with 8 CPUs set. I got 30M, researched, and disabled RSC (even though it was already reported as disabled) via PowerShell with these commands:
netsh int tcp set global rsc=disabled
Get-NetAdapterRsc | Disable-NetAdapterRsc

Then I could get a solid 940M in Hyper-V, AND have the luxury of auto-start after host reboot. (Lessons for anyone reading this)