Ethernet keeps dropping to 10Mbps

jonnytabpni

Hi Everyone,

I have a Dell R210 server which has 2 Broadcom NetXtreme II BCM5716 Gigabit network cards in it. I have pfsense running in Xen HVM mode with the NICs passed through directly to the guest.

Initially, the connection speed it detected as being only 10Mbps. To bring it back up to 1000Mbps, I have to remove the cable, and put it back in (sometimes a few times!).

Any ideas on how I can force BSD to keeep at 1000Mbps? Maybe this is a driver issue?

Thanks

Cry Havok

Sounds like a Xen problem - what version of pfSense did you install?

jonnytabpni

I installed 1.2.3-RELEASE

I've tried this on Xen 3.4.3 and Xen 4.0 and the same thing happens.

I also tried to install the latest Beta of pfsense however it coudn't find a lot of php file??

overand

I'd suggest you specify an "e1000" adapter, if you haven't already, per http://forum.pfsense.org/index.php/topic,25592.0.html

cougarmaster

Hi jonnytabpni,
I think it is best to stick with the 3.3 versions of xen for the moment. I have tried 3.4 and 4.0 but there seems to be some sort of quirkiness in it. I have good stable experience with 3.2 the one that is original with Debian Lenny and 3.3.2 which also uses the kernel from 3.2 from Lenny. I did a lot of studying and planning before I took the direction of XEN which was (for me) the right decision. Though it may be a pain the ass to setup initially I think I got it mostly under control. Maybe the following tips might help you stabilize your system.

1] Make sure you have the minimal installation of Debian for Dom0. The only other application I added to this was SSH
2] Make sure you set the the dom0 according to this webpage http://wiki.xensource.com/xenwiki/XenBestPractices
3] Since 3.2 and 3.3 have a bug that you must assign all cpu core to dom0 otherwise it may hang. But pin all dom0 cpu to 1 core.
e.g : Name ID VCPU CPU State Time(s) CPU Affinity
Domain-0 0 0 0 -b- 45860.5 0
Domain-0 0 1 0 -b- 272.4 0
Domain-0 0 2 0 -b- 493.8 0
Domain-0 0 3 0 r– 296.3 0
pfsense 8 0 0 -b- 73304.0 0
pfsense 8 1 1 -b- 42091.2 1
trixbox 9 0 2 -b- 195545.2 2
trixbox 9 1 3 -b- 3132.8 3
ubuntumul1 2 0 1 -b- 242572.6 1
ubuntusigcl 15 0 3 -b- 52.1 3

Look at the values under VCPU, CPU & CPU Affinity. All four cores in my system for Domain-0 are set to CPU core 0. The others you can set according to your need or requirements. Another feature is to set weight on domains as you may have read in the webpage lined above. The higher the value more priority is give to that domain which should be given to dom0 as it does a lot of I/O work.

I used the model=e1000 only in XEN 3.3.2 as it is not available in 3.2. It does take a bit of time to tune it but at least the past year the XEN 3.2 never did give me any trouble and it never once hanged or crashed from the day it was installed. Though it took about 10 reinstallation till I got the hang of it. :D The above vcpu-list is that system and its serving trixbox with a 4 port analog telephony pci card using pci-passthrough serving 20 people in different parts of Asia and two X2go ubuntu desktop serving 10 people for internal and external access. The internet access is a DSL 4/4Mbps. Well I hope this will help you or others in working with XEN. I am no expert but I find XEN very easy to use and understand (also fun) if you put time and effort into it.

All the best
Eric

p.s. :- Try to make sure you nics are good, cables and especially the switch. Nothing is what they seem on the surface. OMG this is the longest post for anything I have done, I can't believe it I typed so much since I am those people that are very lazy hehe.

cougarmaster

Hi jonnytabpni ,
Another thing you could try is to run this following command on Dom0 :-

ethtool -K eth0 tx off

Test this if it shows any sign of improvement. If it does then add it to your network scripts e.g. :-

Debian or Ubuntu /etc/network/interfaces :-

iface eth0 inet static
address 206.124.146.177
netmask 255.255.255.0
post-up ethtool -K eth0 tx off