Bandwidth test= fine, browsing unusable (HYPER-V)

stlnstln

Hello! (I posted this somewhere incorrect)

I have pfSense 2.1 in both physical and virtualized forms. Physical has worked flawlessly for years but I want more time on our UPS so moving it into the virtualized domain would be a good idea.

I have hyper-v 2012 and it has been nightmares all around (understandably so) until I received a pre-built VHD(X) for it. Boots up and everything is perfect, the interface is full and snappy. I add the proper ISP and it automatically picks up the DHCP (simple cable modem, no PPPoE or MAC spoofing needed). I try to browse a web page and it is incredibly slow and only loads a portion before timing out. I run a bandwidth test and after it loads, I get the full 10mbps down and 0.5mbps up.

So it seems like the bandwidth is not an issue, the LAN has no issues…..it seems like it is just the routing. I have defaulted everything with no firewall rules apart from allowing everything on the outbound interface.

I have assigned it 2 cores of the Xeon L5520 and they aren't spiking. I have assigned 512mb RAM but will see about expanding it a bit later today even though an insignificant portion has been used. The VM does have the integration services added as well as it is using the 10gb adapters.

I am also using a separate internal DHCP (not pfSense's) as well as a separate internal DNS (not pfSense's) and this works when I'm using the physical pfSense.

Any help would be appreciated!
Thanks

kejianshi

Can you post a snapshot of your rules page here?
Also you interface assignments, DHCP server settings.
And what IP are you using on the LAN? How about the "Modem".
Is that a modem modem or a modem+router?
Is it routing?

stlnstln

@kejianshi:

Can you post a snapshot of your rules page here?
Also you interface assignments, DHCP server settings.
And what IP are you using on the LAN? How about the "Modem".
Is that a modem modem or a modem+router?
Is it routing?

yep, give me a few minutes, I'm rebuilding from scratch. LAN rules are literally default anti-lockout and allow any from LAN to WAN
LAN pfSense interface IP is 192.168.10.1, WAN is DHCP (valid). DHCP server is OFF because I have my own separate one
192.168.10.1,
it is a modem modem. I specifically asked for it. I don't want my ISP to NAT for me at all. I want my 2 public IPs per WAN connection for DMZ and Lync
it is routing. It does work, and total throughput on a single download works. But when I go browse, pages only load half way and artifact everywhere (tested on multiple machines). Most times they just time out and fail to load, though.

stlnstln

@stlnstln:

@kejianshi:

Can you post a snapshot of your rules page here?
Also you interface assignments, DHCP server settings.
And what IP are you using on the LAN? How about the "Modem".
Is that a modem modem or a modem+router?
Is it routing?

yep, give me a few minutes, I'm rebuilding from scratch. LAN rules are literally default anti-lockout and allow any from LAN to WAN

LAN pfSense interface IP is 192.168.10.1, WAN is DHCP (valid). DHCP server is OFF because I have my own separate one

192.168.10.1,

it is a modem modem. I specifically asked for it. I don't want my ISP to NAT for me at all. I want my 2 public IPs per WAN connection for DMZ and Lync

it is routing. It does work, and total throughput on a single download works. But when I go browse, pages only load half way and artifact everywhere (tested on multiple machines). Most times they just time out and fail to load, though.

http://imgur.com/wUha5bC is the rules page

stlnstln

I've moved the virtualized pfsense to 192.168.10.254 so that I can keep the physical pfsense active for the users.

http://imgur.com/mDkJnrF is a screenshot of me trying to refresh this page when using the virtualized pfsense as my gateway

kejianshi

Just the one pic? I see nothing broken on that.
Describe your DHCP mechanism? What is pfsense connecting to on the LAN side? What is that connecting to? what connects to that?

stlnstln

@kejianshi:

Just the one pic? I see nothing broken on that.
Describe your DHCP mechanism? What is pfsense connecting to on the LAN side? What is that connecting to? what connects to that?

My LAN DHCP is a Microsoft DHCP. My WAN DHCP is provided by my ISP. pfSense internal LAN IP is static. pfSense connects to the virtual switch on the LAN side (same VLAN as the other devices on my internal network. This works as evidenced by me being able to log onto the pfSense configuration page with absolutely no issues at all.

My network topology consists of 2 zones (WAN & LAN) linked by pfSense.

LAN side has (all on same VLAN and subnet): Domain controller with DNS, Application server with AD CS, DHCP and a few other roles, Second Application Server with WDS, WSUS and PRTG, KMS server, SQL 2012 server and 6 laptops, 2 desktops, 3 access points and a few other phones.

The internal LAN side is all working correctly because when I use the physical pfSense, the internet is normal. When I use the virtual pfSense, then things become unstable.

I am able to get an IP from my ISP. I am able to ping google from internal machines. I am able to load some content but not everything loads before timeouts hit.

kejianshi

OK - Sounds like its going to get complicated now.
So, you have a physical pfsense (with its own separate public IP and modem?) and switch and LAN and LAN clients and that is somehow connected to your virtual pfsense which has a /22 (not a /30?) and its on LAN (connecting to I'm not sure what or how) getting DHCP from the first pfsense?

I think we are going to need a network diagram. Even just a snapshot of something scribbled on paper is ok with me.

kejianshi

haha - Seems you keep posting answers to questions I'm about to ask. Lets see if you beat me to the punch this time.

OK. So, I know there are at least 2 separate WAN IPs now.

How is outbound routing of packets from the LAN being handled with 2 (or more) WAN IPs?

Also, who is your ISP?

I'm not saying definitively that no one hands out DHCPed multi-ip connections that are static I presume?
Usually, they assign you a few IPs and you have to bridge the WAN interface to them like this:
http://www.youtube.com/watch?v=zrBr0N0WrTY (sorry if you cant get that) Basically, its bridged and then Virtual IP assignment usually for me here)
So, are you 1:1 NAT from virtual pfsense > physical pfsense?

stlnstln

http://imgur.com/KXeyuvc,xBi1s24

Here are the 2 scenarios.

Basically I can swap between the virtualized pfsense and the physical using VLANs and such to test. I can even keep them on the same network (with different LAN IPs.1 and .254) for testing purposes since each have their own separate IP from the ISP and they are also on separate WAN subnets, too.

So basically changing my default gateway on my laptop to the physical pfSense leaves me with a solid internet connection and everything is great. Changing my gateway to the virtualized pfSense, things fall apart.

I don't see anything out of the ordinary on the logs, either :(

stlnstln

@kejianshi:

haha - Seems you keep posting answers to questions I'm about to ask. Lets see if you beat me to the punch this time.

OK. So, I know there are at least 2 separate WAN IPs now.

How is outbound routing of packets from the LAN being handled with 2 (or more) WAN IPs?

Also, who is your ISP?

I'm not saying definitively that no one hands out DHCPed multi-ip connections that are static I presume?
Usually, they assign you a few IPs and you have to bridge the WAN interface to them like this:
http://www.youtube.com/watch?v=zrBr0N0WrTY (sorry if you cant get that) Basically, its bridged and then Virtual IP assignment usually for me here)
So, are you 1:1 NAT from virtual pfsense > physical pfsense?

There are actually 4 public WAN IPs(to make your head hurt more, I actually have 2 WANs but we won't touch the second one until this works).
The outbound routing is being handled ONLY by pfSense computers. One is a virtualized one residing in a Hyper-V host with LAN IP of 192.168.10.254. The other is a physical one residing in a shitty computer with 192.168.10.1.

I can change in my TCP/IP settings and alter my default gateway between the two to test one or the other.

The ISP is Shaw in Vancouver, Canada.

There is ZERO packet flow between the virtual and the physical pfsense. Each are entirely separate gateways on the network. To make things easier, let's imagine I don't have 2 pfsense boxes.
Lets say I am a user who has had a physical pfsense forever. I want to remove it and install a virtual pfsense appliance. The new virtual pfsense appliance fails to perform as well as the physical. What should I check?

kejianshi

Yeah - But I don't see where you have set up a virtual IP to use one of your 2 public IPs?

How are the VIPs (or the VIP) being mapped out and assigned?

stlnstln

Also, my public IPs are DHCP'd dynamics. I do not need statics for what I do with them.

PS: thanks for your help!

stlnstln

@kejianshi:

Yeah - But I don't see where you have set up a virtual IP to use one of your 2 public IPs?

How are the VIPs (or the VIP) being mapped out and assigned?

I am not using any virtual IPs at all

kejianshi

"Lets say I am a user who has had a physical pfsense forever. I want to remove it and install a virtual pfsense appliance. The new virtual pfsense appliance fails to perform as well as the physical. What should I check?"

OK - Easier.

Your WAN IP assignments seem weird to me. You can't call up your company and tell them "Give me two IPs. Give me your gateway address".
Connect to that by bridging to their network (not DHCP) and then hand out the public IPs to Virtual IP?

This setup you have now with multi-IP dhcp at the WAN…. Did that work on a physical box there ever?

kejianshi

OK - Welllll…. If this setup worked for you on a physical box before but not on the VM, I'm stumped.
It does seem complex bordering on unnecessarily so. That must be one sweet Microsoft DHCP machine to keep it around with these kinds of headaches (-;

stlnstln

@kejianshi:

OK - Welllll…. If this setup worked for you on a physical box before but not on the VM, I'm stumped.
It does seem complex bordering on unnecessarily so. That must be one sweet Microsoft DHCP machine to keep it around with these kinds of headaches (-;

Hahah it works surprisingly well. The only non-Dell/Cisco/Microsoft item is the pfSense. I'm waiting for the ASA 1000v to hit hyper-V :-)

For dualWAN I pick 2 IPs and use them as multiple gateways load balanced in the routing section on the far left tab.

For the DMZ IPs I put them right on the edge.I'll move them in later when I have fully set up Lync.

I basically use pfSense as a multiwan capable DD-WRT

stlnstln

@kejianshi:

"Lets say I am a user who has had a physical pfsense forever. I want to remove it and install a virtual pfsense appliance. The new virtual pfsense appliance fails to perform as well as the physical. What should I check?"

OK - Easier.

Your WAN IP assignments seem weird to me. You can't call up your company and tell them "Give me two IPs. Give me your gateway address".
Connect to that by bridging to their network (not DHCP) and then hand out the public IPs to Virtual IP?

This setup you have now with multi-IP dhcp at the WAN…. Did that work on a physical box there ever?

I can't get statics on a non-Business line. I have considered it but with the fantastic Dynamic DNS, I haven't needed to yet. The multidhcp WAN IPs currently are working. :-)

kejianshi

For my own education, can you post a snapshot of your WAN interface assignment?
I'd actually like to see how you are doing that, the VIP assignment, the VLAN and your multi-wan handling also incase I ever need to work with something like yours. There are lots of how-to pages for multiwan/load balance/fail-over etc. I'd love to compare your settings to those for educational purposes.

stlnstln

It just baffles me. I give it a pair of Xeon cores, 1GB RAM, Dual 10GbE NICs, a decently fast RAID storage upgrade and it just says "LOL NOPE" :-(

This works brilliantly in VMWare ESXi but I no longer have a RAID card :-(