Serious Network Issues, Slow Speeds, Cannot Ping, Cannot Figure Out Issue.

[george___42]

Hello, I have a PFSense CE installed on my ESXi Server, running as my home server, and recentlyy I've dealt into a meriad of issues that I cannot figure out.

This post will be rather long, as it seems to be a rather complex issue, but please scroll to the bottom for a TLDR.

I've had this install running for well over 2 years without any issues whatsoever, until recently where I've dealt with numerous issues that I cannot explain/might be out of my depth. Hence, my post here in hopes that you guys might be able to help me.

I noticed these issues when files transferred from my NAS (OMV) also inside my ESXi Host would not go past 40MBps, where they usually will saturate the line to my PC @ 125MBps. I did a few experiments to try and see what was going on:

• Transfering Files from the NAS to a Windows VM in the same Host, Reached 1Gbps as it should

• IPerf3 Between Desktop and Windows VM/NAS: 300-400Mbps

• IPerf3 Between Windows VM and NAS: 1Gbps

• Looked at Network Graphs on ESXi and noticed traffic was flowing through Router when desktop and NAS were both put on the same Vlan

• Unable to reach ESXi Management WebUI from desktop without PfSense running (Was never like this before, they are both on Vlan 0)

• Tried Disabling LAG (IP Hashing) to no avail/difference.

And in the midst of trying to fix this, today my internet seems to be rather slow, despite speedtests showing the full 75mbps service I get.

I'm not sure whats going on, but in the midst of trying to fix this, I have resoted to replacing both the managed switch and 4 port NIC on the host, both of which were configured the way the old hardware was, as I thought it might have been a hardware issue.

Please, take a look at my crudly made diagrams and attachments below, which hopefully should paint a better picture. I feel like this issue is a bit out of my depth and any help would be greatly appreciated.

TLDR: I'm having numerous issues of slow speeds from ESXi Host-Out of Host Devices, while devices within the host seems to be communicating fine...ish. This was never a issue before, and has only come up in the past week. New Switch and NIC have been used to no avail, and I'm no longer able to directly access ESXI without the PFsense VM Running, which to my memory was never the case before.

Please Let me know If I can attach more pics to help in finding the issue. I wasn't to sure which ones would be most useful, hence the one below.

Thank you,
Sincerely

[stephenw10]

Do you mean VLAN0 there or untagged? Things actually tagged as VLAN0 can cause interesting issues.

Which hosts were you testing between when you saw traffic through pfSense unexpectedly?

[george___42]

@stephenw10

I made sure everything Vlan0 (LAN) was untagged. Only Vlan 2 and 3 facing the router, switch and ESXi Host were tagged. All Vms and physical devices only say untagged ports to their respective Vms. Let me send a screenshot just to be sure I'm not messing something up.

I used multiple hosts to test my issue.

• Windows VM <-> NAS (All Contained within ESXi, worked fine)

These Had Issues:

• Windows VM <-> Desktop

• Windows VM <-> Laptop

• NAS <-> Desktop

• NAS <-> Laptop

Here are my Switch Config. I very well could have messed something up. Swithces 13-16 are blanked out as they are now part of TRK 1 (LAG using IP Hashing)

Vlan 0 (Switch Config)

Vlan 2 (Switch Config)

Vlan 3 (Switch Config)

[stephenw10]

If you test beween OMV in ESXi and the Media Machine in the same VLAN but external is it still slow? And it that traffic you still see going through pfSense when it shouldn't?

[george___42]

@stephenw10 I tired this out right now. I had unplugged the media machine in diagnosing it, but I plugged my desktop into port 24, which in the switch config images above is part of VLAN 2 but the external, same as the media machine. I tried transferring files from the desktop and it pretty much flatlined.

Doing Iperf3 from the NAS and my desktop did show activity on the router despite being on the same vlan. Not sure why...

File Transfer (Usually will saturate gigabit)

IPerf Test

PfSense Network Graph

[stephenw10]

Hmm, thats shown as mbps but can I assume it's actually Mbps?

Does the traffic graph in pfSense itself also show traffic during that iperf test?

If so It sounds like one of those devices on VLAN2 has the wrong subnet mask set and is sending traffic to it's gateway rather than directly.