I'm not using LDAP and I'm not on the LAN, this router is in another city…

Latency is primarily caused by one of two things, bufferbloat or distance. One way to fight the only one you have control over is to rate limit your connection. You can find better info about this in the Traffic Shaping forum.

@cmb: That's a Supermicro MAC address. Have an IPMI with a shared port? Maybe it's grabbing your IP, which creates the conflict. Something with a Supermicro NIC in it is creating the issue. The issue is introduced at your ISP's next hop router, rebooting just sends a gratuitous ARP which lets your WAN NIC take back the IP for a period of time until the other device takes it again. Super, that appears to be it! I am running a supermicro A1SRi board with a physical management port, but it was not connected to anything. I hooked it up to the switch and identified the MAC address. Appears to be a setting in the web interface with for the network port with options such as "failover", "dedicated" and "shared". It was currently configured in "failover" mode. I suppose the issue might have been fixed now that it receives a proper IP by DHCP, but I'll switch it to "dedicated" anyway. Thanks.

Like, verify the checksums and redownload it?

Bear in mind that some packages seem to make the transition better than others.

already tried vmtools via "nox" and via ESXI install CD and the results are always the same. The firewall is working ok minus this disk problem…. I think I will install an 2.2.0 version and see what values I will get on disk access. If its ok I no longer will upgrade pfsense until something or someone finds this and fixes... :( But first will try a clean 2.2.2 install, maibe I am luck with that Thank you everybody

I can watch 4K videos from YouTube over my 100Mb Internet connection, no buffering. The initial start of  the video has a hair bit of hesitation, like 1-3 seconds, but once the video is playing, I can jump to non-buffered parts of the timeline and it starts playing in less than 1 second. 4K UHD Bluray is 82Mb/s-128Mb/s. Jumbo-frames is not going to fix your 1,000Mb/s network not being able to handle 128Mb/s. Find the real bottleneck. It's probably the protocol being used to remotely stream the file. If  you're using a web client, maybe your web service needs to have its IO buffers, network buffers, or caches tweaked.

For the scenario you presented, you do not need a layer 3 switch. In your original post, your heavy duty data was on VLAN 99. If you just buy a Cisco 2960 (or other switch that supports LAG and VLANs) and use LAGs to the ESX servers, you'll be OK. Do keep in mind that LAGs don't magically balance traffic across the links. You have to configure them to use bits in the source or destination addresses to determine which physical port in the LAG gets used.

It's wooooorking! For those that come after me with the same issue: the solution was to go into the WAN interface configuration, click the link for "Advanced and MLPPP" and in the "Link Interfaces" box select the right interface (there weren't any selected - that's never going to work!), then save. The problem was possibly caused by my having some settings carrying over from the old ISP, not sure if completely new pfsense-ers will have to do this.

Thanks a lot for help. So can I install phantomJs on  pfSense itself?

OpenVPN may do it. Discaimer:  I've NEVER architected a network with a need for a site-to-site VPN bridge.  Site-to-site bridge, sure, using L2 switches.  VPN, absolutely, but they have always been routed. I had initially started to write an "It can't be done" reply, but started reading openVPN's doc's and howtos.  OpenVPN supports a bridged VPN config, but I'm not clear if it works with site-to-site, or only in a road warrior scenario.. Regardless, a site-to-site bridged VPN would be a huge waste of network and processor bandwidth.

The process is similar though not quite exactly the same in 2.2x's services.inc. Two lines to change there for IPv4: $lighty_config .= "server.bind  = \"0.0.0.0\"\n"; $lighty_config .= "\$SERVER[\"socket\"]  == \"0.0.0.0:{$lighty_port}\" { }\n"; To change 0.0.0.0 to 127.0.0.1. Then after making those changes in services.inc, run /etc/rc.restart_webgui to reload. Check the output of "sockstat -4" and "sockstat -6" to check its IPv4 and IPv6 bindings afterwards.

making an overly complex network with extra overhead in performance for no reason is not fun ;) Why would you be using powerline adapters from different makers?  That they work at all is amazing actually.  Get powerline from the same maker if you want to encrypt their traffic. you could look to something like tcpcrypt or ipsec

@peterk: But in this case we have also installed packages and if im correct those wont be backup in the config.xml? Right ? Wrong.

you want to bound 2 different wifi connections with different ssids?  If you want to see if effects you speed, why don't you connect a device to it and download something large and then connect to your normal wifi and download, do you see any difference if the other wifi is being used?

I found the issue with Pings reading 500 using a speed test and Webpages loading slow prb was IP Do-Not-Fragment compatibility in Firewall advanced section checked Clear invalid DF bits and resolved the issue thanks for your help.

There is no such thing as gig without autonegotiation.  gig is always auto.  I don't know why pfSense shows 1000baseT and 1000baseT full-duplex.  There is no such thing as 1000baseT half-duplex.  My guess is both those selections do the same thing. The problem lies when the other end is hard-set at 100-full or 100-half.  Both sides have to be either hard-set to the same settings or both set to auto.