Heavy CPU load?

Burken

Hello,

The "device polling" option did not help me.
I get 80Mbit/s MAX…

80Mbit/s = 100% CPU-load...

What NICs works best with pfsense then?

tell me and i will throw those shitty 3x "Intel PRO/1000 MT Dual Port Server Adapter" out the window!

My old D-link router for €20 gave me more throughput...

Perry

I was thinking. Your running on a pci bus not a pci-x bus correct?
1. I don't seem to remember seeing any post with 5 LB and p2p being used before. So before you throw them out it would be interesting to see the load when downloading a dvd distro with a download manager like getright.
2. Under System -> Advanced
Change Firewall Optimization Options to aggressive and set Firewall Maximum States to 500000
3. Will the load change if it was 5 different clients using 1 wan each.

Basically I tent to believe the bottleneck could be related to the pci bus, pps or slbd.

Eugene

According to graphs lion's share of load comes from 'system', why don't you give us

top
```that you get during the test.
slbd is known for its bad manners to create cpu-load.
try```
killall-9 slbd
```  and repeat (or during) the test. I do not believe that it's Intel's driver problem.

Burken

Hi guys!
Thx for trying to help me out..

Yes i only have PCI… but why all the CPU usage then?
I now have aggressive mode + 500000 states.
no change there..

then i did Eugenes tip.

No change…. :(

When the top were taken the total speed was ~88Mbit/s (torrents)

I need to find a fast server to do the getright/FTP test..

Perry

You could try with systat -vm 1 and others http://www.acmesecurity.org/~thiago/public/freebsd/FreeBSD_Bottleneck_Detection.pdf

I need to find a fast server to do the getright/FTP test..

For Ubuntu there lot's of location. AFAIR In Getright split the file into 5 segment and use file mirror to search for different locations.

Cry Havok

Torrents are generally a good way of doing bandwidth testing. Grab yourself the DVD of your chosen Linux distro (or a randomly selected one).

wallabybob

Your top output shows an unexpected (by me) large number of dhclient processes using an unexpectedly large amount of CPU time.

On my system:

uptime

7:22AM up 43 days, 10:39, 2 users, load averages: 0.31, 0.31, 0.26

ps ax | grep dhclient

335 ?? Is 0:00.51 dhclient: rl0 (dhclient)
284 con- I 0:00.10 dhclient: rl0 [priv] (dhclient)

On my system there are two dhclient processes which in 43 days haven't even used a second of CPU time between them while in yours (uptime of over 55 days) you have at least 7 dhclient processes which have each used at least 130 MINUTES of cpu time and all 7 are in the RUN state. On my system only the WAN interface (rl0) acquires an IP address by DHCP.

How many interfaces should be trying to acquire an IP address by DHCP?

Why are so many dhcp clients all in the run state? (Are your leases expiring every milli-second? :) )

Are there any log files which would give a hint as to why the DHCP clients are so busy?

Burken

its true it seems like _dhcp (dhclient) that makes the CPU load?

I have 5 NICS on DHCP… and DHCP-Server on the 6 interface...

i get alots entrys like this:
May 28 19:51:02 kernel: arp: 85.226.120.1 is on em1 but got reply from 00:03:a0:3b:80:00 on em4
May 28 19:51:02 kernel: arp: 85.226.120.1 is on em1 but got reply from 00:03:a0:3b:80:00 on em3
May 28 19:51:02 kernel: arp: 85.226.120.1 is on em1 but got reply from 00:03:a0:3b:80:00 on em5
May 28 19:51:02 kernel: arp: 85.226.120.1 is on em1 but got reply from 00:03:a0:3b:80:00 on em2

but i know why...:

They are all on the same VLAN…
:/

can this be the problem?

Eugene

May I ask you about the reason you have five WAN interfaces? with one ISP… :-\

wallabybob

@Burken:

I have 5 NICS on DHCP… and DHCP-Server on the 6 interface...

Its not clear to me what this means. I guess you are saying you have most (or all) of your interfaces serving DHCP addresses AND requesting DHCP addresses from another DHCP server. This is not a good idea. Your DHCP server interfaces should have static (fixed) IP addresses.

@Burken:

i get alots entrys like this:
May 28 19:51:02 kernel: arp: 85.226.120.1 is on em1 but got reply from 00:03:a0:3b:80:00 on em4
May 28 19:51:02 kernel: arp: 85.226.120.1 is on em1 but got reply from 00:03:a0:3b:80:00 on em3
May 28 19:51:02 kernel: arp: 85.226.120.1 is on em1 but got reply from 00:03:a0:3b:80:00 on em5
May 28 19:51:02 kernel: arp: 85.226.120.1 is on em1 but got reply from 00:03:a0:3b:80:00 on em2

Your network topology and/or address assignments are messed up. 85.226.120.1 is accessible on multiple interfaces, it should be accessible over only one interface (unless you have bridged interfaces, but then why would you have a switch?) And printing these messages repeatedly will be another consumer of CPU time.

What are you trying to accomplish with this configuration? At first sight it appears overly complex.

marsboer

@Burken:

can this be the problem?

Yep. This is most likely your problem as the DHCP processes shouldn't be using any CPU at all.
It still doesn't solve the em-problems, but that's probably not what's limiting you with that massive CPU-usage from DHCP.
Probably your problem is solved by ensuring that the DHCP-server is not running on the WAN-interfaces as it seems that you are actually running DHCP-server on those in addition to the LAN-interface.
This should be a configurable setting.

Burken

My ISP gives me 10Mbit/s for every IP we use.
Max 5 IP-addresses

Thats why i use five NIC's to get my five IPs.

So with one IP 10Mbit/s with two 20Mbit/s.. and five 50Mbit/s

Okay?

My ISP will never give me static IPs
Always DHCP…

here is how it works:

em0/LAN Static 192.168.1.111. And runs DHCP Server for LAN clients.
em1/WAN Dynamic DHCP Client
em2/WAN1 Dynamic DHCP Client
em3/WAN2 Dynamic DHCP Client
em4/WAN3 Dynamic DHCP Client
em5/WAN4 Dynamic DHCP Client

If i do killall dhclient

My CPU usage get low. But pfsens stop working after a while.....

So what is wrong ?:(

Okay... kill dhclinet works... but the firewall dies so i have to restart it after a while....

chpalmer

All your ports share the same switch?

Ive never had good luck when I had two dhcp servers (your pfsense lan and your isp's modem) on the same switch…

Can you move your lan to another switch to rule that possible issue out?

Burken

I use VLANs.
So its physically one switch but inside they are different.

You can read about it here: http://en.wikipedia.org/wiki/VLAN

chpalmer

I know what you are trying to do so I guess Ill ask outright…

Have you ruled out a misconfiguration on your switch as the root cause of your problem?

What else have you tried in your troubleshooting process?

Start with the basics and add one element at a time until you can reproduce the result.

Your setup while innovative is not typical.

Good Luck!

Burken

There is nothing wrong with the Switch. As you can se here the Vlan settings is so simple.

You guys just helpt me to see that it is wrong with the dhclient.

what is wrong with my setup thats makes it non typical?

What else can i do to troubleshoot? i have killd dhclient and everyhing works fine..

Cry Havok

Most people either have:

Multiple interfaces, connecting to different ISPs
Multiple static IPs (possibly with one dynamic), on a single interface

It's very uncommon to have a single ISP, with multiple dynamic IPs across the same subnet on multiple interfaces, particularly using a single VLAN capable switch to separate WAN(s) and LAN.

Burken

I have tested with a second GS724T so VLAN works. Thats not the problem..

No1 else have problem with dhclient CPU usage?

Eugene

I am afraid your problem is network design. Everything esle is the result of this problem.

wallabybob

I'm trying to understand your configuration rather better because I also think its unusual. I take it you have 5 "WAN" interfaces from your ISP purely to get additional bandwidth.

From what you have displayed about your switch it looks as if you MIGHT be purely using its "VLAN" capability to segment the ports so as to isolate one group of ports ("LAN") from another group of ports ("WAN"). Correct?

Apparently 6 of the 24 switch ports are in the "WAN" LAN. Of those 6 ports 5 go to pfSense interfaces em1 through em5. From your network diagram your sixth port goes to your ISP but what does it actually connect to? Is there is a web page (in English) describing it or holding a pointer to a downloadable manual or datasheet? I'm guessing that its something that will allow up to 5 systems to connect to it, each able DHCP request an address and that these addresses are all on the same IP subnet. I've not come across anything like this that would assign additional bandwidth on the WAN (Internet) side with each additional IP address assigned. If we can find out a bit more about the equipment that connects you to the ISP we may be able to help solve your configuration problem.

The fact that you have 5 pfSense interfaces on the same LAN is a configuration error unless they are bridged. (Each interface should be on its own distinct IP subnet.) And why would you bridge them in pfSense when they are connected to a switch?