VMWare Pentest lab: Extremely high CPU on host
-
@Veni: Sorry I was away for a bit - to clarify this, are you using direct access for the USB- device or are you routing it through the host- OS?
No problem :). Always nice to keep the thread going so that we can come to an end. At this point this is a home machine so basically nobody is loosing anything on this problem.
-
I'm running the USB phone routed through the ESXi to the guest.
-
I've tried(see my last post) several guest installations without the USB phone, just basic pfSense installations from scratch.
-
I've checked with vmstat -i that I'm not using any shared IRQ addresses in the guests OS's mentioned in my last post.
-
My real pfSense 2.0.1 installation(the one with the USB phone) is using shared IRQ's(em3 shared with uhci0+ @ rate 6 and em0 shared with ehci0 @ rate 1186).
Update:
Forgot that i run one more FreeBSD based guest, running FreeNAS(FreeBSD 7.3-RELEASE-p7) and it shows up @ 26 MHz according to vSphere Client while top shows 100% idle.
Sounds that something @ FreeBSD 8+ and some hardware in our boxes are not working as they should.We would need a FreeBSD guru that knows what differs between FreeBSD 7 and 8,
or somebody that can find the common denominator within our boxes. -
-
I'm wondering if what you guys are seeing is the load generated from the vSwitches.
These are certainly not CPU cost free, and also are not known by the guest OS, and as such would not register as a load in the guest.
To someone's point above, has anyone experiencing this issue tested using DirectPath I/O to forward the NIC's directly to the pfSense guest and see if the extra load goes away or not?
-
New update with the following:
FreeBSD 8.1 i386: 26 MHz.
FreeBSD 8.2 i386: 26 MHz.
FreeBSD 9.0 i386: 0 MHz.
OpenBSD 5.1 i386: 26 MHz.Values given by vSphere Client when clients are idle.
So at a first glance it does not sound like it's the BSD part that is the problem, but I'm not familiar
with how to export the NAT and rules from pfSense to packet filter on these platforms, because I would
like to try to put some real load on them, but for that I need inbound NAT rules for it to be effective.I'm wondering if what you guys are seeing is the load generated from the vSwitches.
Hi mattlach. That one slipped by me. Have not been been taking into account that the vSwitches do take up some cpu cycles, but why would there be such a big difference between pfSense 1.2.3 and 2.0.x?
-
Can you see realtime performance on the ESXi host?? That is where the high CPU use is and not in the client….
-
Can you see realtime performance on the ESXi host?? That is where the high CPU use is and not in the client….
I'm seeing high CPU usage with the help of vSphere client when running pfSense 2.0.x guest. Feel that that is enough to show the high CPU usage by the guest.
-
Hello,
i'm having similar problems with pfsense (2.0 and 2.1) and vmware. I'm not sure witch vmware version i'm using since it's in the "cloud", but cloud management concole says "powered by vmware". We used to run pfsense on very old PC and it worked like a charm, but now when we moved to "cloud" and got more CPU and RAM recources for pfSense it's suddenly not enought for pfSense. pfSense mamagement console says it's only using a few percents of CPU, but cloud administrator tells me we're using 100% and that's the reason our internet connection is so slow. I tried reinstalling and even installed 2.1 RC1, but it didn't help much. Does anyone have any idea what can be done or should i look for another firewall? -
Not good :(
-
Hello,
i'm having similar problems with pfsense (2.0 and 2.1) and vmware. I'm not sure witch vmware version i'm using since it's in the "cloud", but cloud management concole says "powered by vmware". We used to run pfsense on very old PC and it worked like a charm, but now when we moved to "cloud" and got more CPU and RAM recources for pfSense it's suddenly not enought for pfSense. pfSense mamagement console says it's only using a few percents of CPU, but cloud administrator tells me we're using 100% and that's the reason our internet connection is so slow. I tried reinstalling and even installed 2.1 RC1, but it didn't help much. Does anyone have any idea what can be done or should i look for another firewall?Have you tried pfSense 1.2.3? I'm about to revert from 2.0.1 to 1.2.3 this month due to high CPU usage on 2.0.x. I will be canceling the 3G backup route so 1.2.3 will be more than enough.
-
Thats one of the reasons I havent upgraded yet.
It showed the same on both of my physical server setup running in a VM.
1.2.3 works great!
-
Hello,
i'm having similar problems with pfsense (2.0 and 2.1) and vmware. I'm not sure witch vmware version i'm using since it's in the "cloud", but cloud management concole says "powered by vmware". We used to run pfsense on very old PC and it worked like a charm, but now when we moved to "cloud" and got more CPU and RAM recources for pfSense it's suddenly not enought for pfSense. pfSense mamagement console says it's only using a few percents of CPU, but cloud administrator tells me we're using 100% and that's the reason our internet connection is so slow. I tried reinstalling and even installed 2.1 RC1, but it didn't help much. Does anyone have any idea what can be done or should i look for another firewall?Have you tried pfSense 1.2.3? I'm about to revert from 2.0.1 to 1.2.3 this month due to high CPU usage on 2.0.x. I will be canceling the 3G backup route so 1.2.3 will be more than enough.
Thank you for a quick response. I haven't tried 1.2.3 yet. Is it secure? I mean it's an older release so there might be know security bugs, or isn't there?
-
Not as far as I am aware.
-
I cant say I have had this issue yet, but then again I cannot get it configured where i can reach the lan from wan side. I am running it in virtual box, not sure if that would make a complete difference. although if you may have a guide on setup I could use that. I will try to redo and test out vmware this coming weekend.
-
There are XSS and CSRF vulnerabilities in 1.x's web interface. Though if you follow general best practices for managing any web-administered device (use a diff browser than you ever use for Internet), that's a non-issue. Every web-managed device has had some XSS and CSRF issues, and many commercial security-related products have a number of known unpatched XSS and CSRF. Some have released updates fixing them. There isn't anything imminently exploitable in any pfSense version, but I wouldn't recommend running anything prior to the latest stable release.
We and many, many others run most or all our production firewalls on ESX. This very site runs behind firewalls in ESX, and can route gigabit wire speed between internal VLANs, without any excessive CPU usage on the host. All of our production colos run their firewalls in ESX without any issues at all, and they're pushing significant loads. Why a minority of people see this, I don't know, but it's something we plan to investigate post-2.1 when time permits. It may be something that just goes away when we get to a newer FreeBSD base.
Note you do need to make sure you're on the latest ESX (5.0U1 or 5.1 should be fine), while I'm not aware of any ESX issues exactly along these lines, they have patched several bugs related to FreeBSD guests over the years, and there is at least one ugly one in 5.0 pre-update 1.
-
[…]I'm about to revert from 2.0.1 to 1.2.3 this month due to high CPU usage on 2.0.x. I will be canceling the 3G backup route so 1.2.3 will be more than enough.
Reverted last month. Runs perfectly smooth. Right now 42 Mbps makes the guest drive up the clock to 506 MHz(out of 2,66 GHz) on the host. Perfect!
You guys that run your large setups on ESXi without any CPU utilization issues, what type of motherboard, pCPU and pNIC are you using?
-
Use IBM X3550M4 with Intel 10GbE cars X520-T2.
-
Use IBM X3550M4 with Intel 10GbE cars X520-T2.
If you are running at 10 Gbit/s uplink, do you use DirectPath I/O with the pNIC's to pfSense or do you virtualize them to pfSense?
Otherwise the platform is a more current generation than mine. -
I dont see any issues at all related to high CPU on 2.0.2 release.
Running 3 seperate FW's on 4.1 U3.
Packages:
File Manager
Open VM-Tools 8.8.1
PFBlocker
Snort
-
I am not. I only use 10Gbit internally and use the build-in 1Gbit for PF WAN.
No directpath, but virtualized through VmWare.
:)
I am running the 32bit version of Pfsense and VM version 7.
2vCPU and 1GB of memory. 11% memory used at the moment and 37% disk.
Use IBM X3550M4 with Intel 10GbE cars X520-T2.
If you are running at 10 Gbit/s uplink, do you use DirectPath I/O with the pNIC's to pfSense or do you virtualize them to pfSense?
Otherwise the platform is a more current generation than mine. -
Are people still having this issue with the latest esxi 5.1?
I am running 4.1 at the moment and basically any sort of large download kills access to every VM running on the host until it is complete! :(
I have tried everything I can think of and is listed on here, I am getting it on all versions of pfsense from 1.2.3 up to the latest.
It has got to the point now, I am either going to have to setup Pfsense on some dedicated hardware or switch to monowall but I really need openvpn :(
So is upgrading to esxi 5.1 a fix for this?
-
Try to limit the download bandwidth. ;)
Currently seeing ~2% CPU on the ESXi host on 2.0.3 REL.