VMware Workstation VMs Web Traffic Being Blocked

dfinjr

@stephenw10
Ok got it all done, noticed something interesting this morning that was a little different. Still sites failing but I tried bestbuy.com and amazon.com this morning and amazon super struggled to load but I think eventually completed and bestbuy also navigated. However, gmail, gmail webstore and speedtest.net (and the sites I use for work) all failed to load/render.

Attached is from the client as well as the capture from the "labsystems" vlan where the client resides. packetcapture-3.cap clientcaptureshort.pcapng

dfinjr

@stephenw10
Just realized I didn't have that "IP Do-Not-Fragment compatibility" turned on so I turned it on and reran the test. Same results. packetcapture-4.cap clientcapture443-3.pcapng.gz Both perspectives are again attached.

johnpoz

@dfinjr Yeah your frame size is all messed up!

Client capture

Here is from a normal client with your typical mtu of 1500. Talking to a local server

dfinjr

@johnpoz
Hello, thanks for jumping in!

Is the fix for this then to bump up the MTU on the VLAN Interface? If yes, what value do you think I should use to best test?

johnpoz

@dfinjr why would your client have such an odd mtu set?

Your clients mtu should be the standard 1500...

dfinjr

@johnpoz
Honestly no idea at all. It is a stock VM build. The VM works on that Cisco appliance so never really considered it was anything to do with the client. Happy to try any and all suggestions for testing!

johnpoz

@dfinjr well you got something messed up that is for sure.. Either on the vm itself on on the VM hosts interface.

That 172.16.0.202 shouldn't be putting that sized frame on the wire!

Here is a vm of mine... See the mtu of its interface is 1500

if I look at every possible interface on my vm host, they are all 1500, which is the standard ethernet frame size

dfinjr

@johnpoz
Trying to understand a little more so please don't take this as anything but naivety on my part. Why would it work on a Cisco appliance and not the pfsense appliance? That same VM has been running to expectation for the last 2 years or so with 0 modification. If I were to fail back to the Cisco appliance it would work without me modifying any of the VM settings and then if I put it back on the pfsense this issue comes back.

Not saying that there isn't something wrong on the stock load for the VM (which I am currently researching how to adjust based on what you said) but also my brain isn't letting go of that difference and is trying to understand it.

DaddyGo

@dfinjr said in VMware Workstation VMs Web Traffic Being Blocked:

Honestly no idea at all.

for a test try going with vmxnet3 instead of E1000e, edit the vmx config file so:

ethernet0.virtualDev = "e1000" change to ethernet0.virtualDev = "vmxnet3"

johnpoz

@dfinjr Well that would only work on a network where that frame size was normal.. Maybe VM to VM traffic across only a VM switch, or on a network set for that.. But its not even a typical jumbo size frame.

I have no idea where that came from, but it sure and the hell isn't going to work on a normal ethernet network.. That uses standard frame sizes.

dfinjr

@johnpoz
Checked the virtual settings and it looks like the ethernet 0 adapter itself is set to 1500 MTU as well:

dfinjr

@daddygo
Hi there @DaddyGo

I'd be happy to try that but can't find the config file you're speaking about. I went to the virtual machine directory and don't see a vmx config file.

Can you help me understand where that might be if it isn't under the directory holding the VM?

Thanks!

DaddyGo

@dfinjr said in VMware Workstation VMs Web Traffic Being Blocked:

Can you help me understand where that might be if it isn't under the directory holding the VM?

Sure,

as others have said too, something is very wrong with this VM, hihihihi

as I said, I prefer ESXi, but this should work (on WS16):

https://kb.vmware.com/s/article/1003880

DaddyGo

@dfinjr said in VMware Workstation VMs Web Traffic Being Blocked:

I'd be happy

comes to my mind....

if you have a chance throw up a fresh windows install already to see if it behaves similarly, I wonder?

(let's rule out a possible problem with the "squeaky" windows)

dfinjr

@daddygo
Thank you that got me to the config file.

Powered off the host and modified the file:

Made the modifications and started the machine again.

Unsurprisingly it needed me to set the info again for the static IP (Not a DHCP subnet) and is now showing up like this:

However I am sad to say it is the same result.

Same sites don't load same sites do. Captured a fresh packet capture with the new vmxnet3 device just in case it would be handy.clientcapture443-4.pcapng.gz

dfinjr

@daddygo
I'll do you one better. Spun up a Ubuntu instance.

Same results:

Netgate.com loads but none of the other sites in question.

Open to any and all thoughts :)

johnpoz

@dfinjr and you still have same issue in that pcap

That is going to have nothing but problems on a normal 1500 mtu ethernet network..

Forget about what loads or doesn't load - putting that sized frame on a etherenet network that is not designed for that size is going to have nothing but problems.

Even if you have something that fragments it back down to the standard, that is not a fix.. The fix is to use standard frame size.

Where exactly are you sniffing that - inside the virtual machine, on the vm host to the wire? That is going to be nothing but problematic no matter if sniffing on the vm, or the physical nic..

DaddyGo

@dfinjr said in VMware Workstation VMs Web Traffic Being Blocked:

However I am sad to say it is the same result.

OMG.... this is get more interesting
but I still say it's not pfSense that's doing this

totally agree with John, but what makes this MTU value?

stephenw10

Mmm, isn't that internal traffic to VMWare though? Is that actually failing?

It looked like it was using standard sized frames for external connections previously. Though this absolutely does feel like an MTU issue.

Steve

johnpoz

@daddygo said in VMware Workstation VMs Web Traffic Being Blocked:

but I still say it's not pfSense that's doing this

pfsense has ZERO to do with this that is for sure!! Pfsense has nothing to do with some client on the network putting odd ball sized frames on the wire.