VMware Workstation VMs Web Traffic Being Blocked
-
Try to get a pcap of the failing registration session from 10.10.0.10. See what's actually happening there. Seems highly likely it will be something similar. Might be easier to see with all local traffic though.
-
@dfinjr said in VMware Workstation VMs Web Traffic Being Blocked:
Your virtual box build idea works. All traffic all sites work. Problem is localized to the vm workstation area on that host machine.
So then you have to say that it is possible that the rumor is true and there are problems between the i219-v and VMware stuff (hypervisors)
+++edit:
Yup and you need to open a ticket for this at VMware
-
Son of a gun! Based on what you were saying I dropped the hardwire lan and bumped it over to wifi (just to switch the network interface) and it worked. VM is able to navigate regularly. Color me shocked!
Ok, trying to figure out now where I go from here.
I still gotta make sure that I can VPN to the lab systems but clearly there is an issue with the hardline network. May be taking a short retail trip to get a usb network adapter just like you mentioned.
-
@dfinjr said in VMware Workstation VMs Web Traffic Being Blocked:
Ok, trying to figure out now where I go from here.
It will be difficult because it's a laptop (vm host), hihihi
A really good quality USB adapter will help.
You rarely hear that from my mouth here on this forum++++edit:
Beware of the Realtek chip, please,....you can save a lot of crying -
Yeah nothing wants to connect to the Bigfix resources well being on Google Wifi so I think your hardline USB adapter is the place to start. I'll get my hands on that first, test, and let everyone know.
-
And just to clarify on the Cisco vs pfsense deal. The reason that the Cisco was able to cope is because of the rfc1191 that Cisco has correct? Is that something that PFSense doesn't support?
BTW thank you all for the great help, beginning to see the lights at the end of the tunnel.
-
@dfinjr said in VMware Workstation VMs Web Traffic Being Blocked:
And just to clarify on the Cisco vs pfsense deal.
Yes it looks very much like that MTU RFC1191
Cisco is my other big favorite, I've been working with them for 18 years, has a lot of small, sophisticated services for which you pay a lot afterwards
but this does nothing to dampen the greatness of pfSense,
unfortunately pfSense can't do these MTU tricks, but it is better at other things -
Hmm, it's hard to imagine pfSense/FreeBSD would break MTU path discovery. Many, many things would be broken by that. Though clearly the path MTU is set far lower with the Cisco in place.
Also that would apply equally to any NIC on the VM host. Unless maybe the wifi NIC itself has a reduced MTU.
-
@DaddyGo
Thank you for that clarification! I am CCNA security certified (education and not a ton of real world) but it is all very interesting for me. It is tough to have a small capable/affordable personal Cisco firewall deployment with VPN capabilities. That and they're somewhat difficult to buy and license on an individual scale. Bet you've run into that too.@stephenw10
I'm very happy to shed any further desired light on the issue. Anything else that I can see/learn in the process is welcomed for sure.Anything else that I can diagnose/packet capture/etc that would do well for anyone trying to nail this down in the future?
-
@stephenw10 said in VMware Workstation VMs Web Traffic Being Blocked:
Also that would apply equally to any NIC on the VM host.
certainly not break it, just...
Yeah, the trick must be in Cisco ASA FW and its MTU handling, probably more sophisticated
this will soon become clear after a good USB NIC test
BTW:
the vExpert forum is full of Intel i219-v problems -
@stephenw10 said in VMware Workstation VMs Web Traffic Being Blocked:
Unless maybe the wifi NIC itself has a reduced MTU.
I am not a fan of wifi, , but I remember something like this... :)
with (MTU+MAC+Encryp.):
default 802.11 MTU = 2304
- wep = 2346
- wpa2 = 2354
-
@dfinjr said in VMware Workstation VMs Web Traffic Being Blocked:
Bet you've run into that too.
you wondon't tell me, I'm here from Fortinet and ASA firelines
-
@daddygo said in VMware Workstation VMs Web Traffic Being Blocked:
vExpert forum is full of Intel i219-v problem
Good to hear I'm not the only one.
I think you're right, we'll know the truth as soon as I have that USB Nic. I'll have that within the next 1-2 hours (have to wait for the store to open) and then I'll resume testing and share results.
-
Alright, new network adapter, hardlined to the netgate, unfortunately same results. Doesn't appear network adapter for the VMware side is part to the occasion anymore:
Taken from .202
I have processed every conceivable update everywhere.
This is the adapter in use now:
I'm running out of ideas :)...
Anyone know how to turn on RC1191 on a netgate appliance or is that even an option?
Thanks!
-
@dfinjr again going to ask - this happens on every single one of your vmware workstations? Or all these vm workstations you showing just VMs on a single laptop?
The host OS you are running vm on has this issue? Or not?
So this host .204, vms on it have no problems? Your hosts don't have the problem? Just vms running on this .201 host?
-
@johnpoz
The only VMs in play outside of testing are the ones that exist off that hosting laptop, no others outside of that. The 172.16.0.201 is the hosting laptop which browses fine but anything it is hosting isn't getting anywhere on the netgate.No issue at the hosting laptop.
.204 is a vm and does have the problem.
I have a total of 6 VMs running off that laptop normally some times up to 10. So far all of them have this issue.
I am starting to hang on the idea of what @DaddyGo brought up earlier about RFC1191. Cisco's can do that amongst a good amount of other network hardware (based on some shooting from the hip research) and it appears that netgate appliance does not. I am hoping to find a away to enable something for it if it is there. I am just running out of ideas.
@johnpoz you seem to be very comfortable with VMware Workstation. Any recommended changes there above what I've already done?
-
@dfinjr I don't normally use workstation - I used esxi for many years, and have supported it at the enterprise level, etc. Its not much different to be honest.. esxi just more bells and whistles, etc.
If the host can talk fine.. and has no issues... Then its something with workstation or workstation on that hardware, or some setting, etc.
If there is talk about some problem with specific nic, that seems to have gone out the window when you are using the usb nic and still having a problem.
You shouldn't have to do anything odd to handle weird mtu - because there shouldn't be any odd mtus on your network. Everything should be set and be using 1500 mtu..
Do you not have another laptop or pc you could fire up workstation on?
I have never seen such an issue in all my years working with workstation, player or esxi or even back in the day when they called it server version 1.. I have been using vmware products since they came out really - and have never run into such an issue... And have never seen sniffs with odd ball mtu sizes like that. I work with sniffs all the time from DCs and all kinds of customers network - Yes I have seen mtu issues, but they are standard jumbo size, where someone thought it would be good idea to turn on jumbo - but the network doesn't support it, or they think they can enable it on 1 host on the network, etc.. A 1767 frame with a 1753 mtu?? Just really nothing comes to mind that could cause that - other then setting it to that. And your other sniffs are non standard as well.. The one that said cisco in the capture had a 1434 frame?? Makes no sense!
-
Another tack here might be to examine VMware's network logs, those on the virtual hosts, and any applicable firewall rules.
-
@johnpoz
Sounds like you have an awesome level of experience to say the least!I did a test just like you described a little earlier today. I spun up a VM (ubuntu) off of a linux mint laptop I had laying around on virtual box. I was able to browse just fine. Because of that I feel that the issue seems to be weirdly localized between how the packets are being generated from VMware Workstation and how pfsense is having to deal with it. Literally my only guess at this point. Keyword being guess :)
Tell me if this is an accurate thought process here. I see one of two things happening.
1 - The VMWare Workstation is always generating crazy packets like this but the Cisco ASA is pulling it down and putting it in check (RFC1191) as the packets traverse making the vms under VMware Workstation work correctly.
2 - The VMware Workstation vms are only generating these crazy packets when the PFSense is in the mix (somehow).
All I know is when the Cisco ASA is in the mix the packets don't go crazy and with the PFSense in the mix they seem to go crazy. But that sentence is specifically tied to VMware Workstation since another system hosting a vm on virtual box seems to be fine.
I feel like my gut idea is that it feels less likely that the packets are changing on the presence of a network appliance, that seems far fetched.
Thinking to an earlier test I did as well I put the VMhosting laptop on google wifi for a bit and that also worked fine. So... that makes me think that it must be something about how google wifi is handling those packets which happens to be the same way that the Cisco ASA is handling those packets.
In summary to not miss anything.
VMware Workstation - works when connected to Cisco ASA - packets normalize
VMware Workstation - works when connected to Google Wifi - assuming packets normalized due to the passing of web traffic
VMware Workstation - does not pass all traffic and client creates odd sized MTUs. Assuming this is the same problem from my test with the TPLink VPN router I tested with earlier. I think perhaps the google wifi and the Cisco ASA are just able to tolerate/fix that traffic on the fly.Call me crazy please...
-
2 makes no sense, the router or the hardware used for your router at the edge of the network would have zero to do with what mtu the host or vm uses..
@dfinjr said in VMware Workstation VMs Web Traffic Being Blocked:
Cisco ASA is in the mix the packets don't go crazy
What I saw in the sniff where you said ciscoasa was still ODD ball frame size... These are ODD.. I sorted the sniff by the largest frame - and there are loads and loads of this max frame of
In a normal ethernet network you should see 1514 with a 1500 mtu, wireshark would prob hide the other 8 bits where the frame could be 1522 if you were doing vlan tags..
If you sniff at pfsense for some normal say your host of your vm that is working correctly - do you see normal sized frames?
