Dell Powerconnect switch autonegotiation with pfsense RC1.0

pmk · Jun 27, 2006, 12:22 AM

To kick off, my hat goes off to the developers and contributors of pfsense, i've only just began to work with it, but i boy am i impressed! Keep up the good work!

I've been testing a config for a production environment, and have seen what appears at first glance to be a "funny" between pfSense a Dell PowerConnect Switch.

Here's the config:

pfSense on a Dell OPtiplex GX150 (PIII/1GHz/256RAM/8gig hdd), dual pci 10/100 nics and one onboard 10/100.
The config is LAN/WAN/DMZ.

A single pci nic'd GX150 running Debian (Sarge) and proftpd lives in the DMZ.

Port 20 & 21 are permitted incoming into the DMZ, and everthing is allowed outbound (currently) with a source address matching that of the proftpd box.

The LANside of the pfSense test environment is gigabit (Dell Powerconnect gigabit switch) and gigabit nic in the test client pc (LAN side).
On the pfSense WAN i've a xircom 10/100 nic in a laptop (yeah you guessed it, another Dell ;-) )

Anyway, the weirdest thing is that uploads from the gigabit LAN client to proftpd in the 10/100 DMZ are always real slow, sub 500kbps approx.
In every other direction/interface combination speeds are consistantly in the 40-60mbps range.

Before pfSense was introduced, ie when proftpd was directly on the gigabit lan, speeds in all directions were 40-60mbps.

I've tried re-assigning the pfsense interfaces (rebooted it, cleared all the client arp caches) but the slow speeds in the up direction allways follow the pfSense LAN interface no matter which hardware it's on as long as it's connected to the Dell PowerConnect switch.

There are no oddities in this network cfg, no vlans, no qos, no jumbo's.

When i put the pfSense LAN interface into a 10/100 Cisco switch, speeds are full rate in all directions, so it's something to do with the PowerConnect and pfSense.

If anyone can offer methods to pinpoint the source of the problem then i'd be more than happy to give them a go, but i'll retest using gigabit switch from another vendor and see how it goes.

hoba · Jun 27, 2006, 12:41 AM

What does status>interfaces tell for the speed and duplex of the interface that is connected to the dell switch? What does the dell switch report? Also, do you see some kind of in/out errors and/or collisions? In case you have someduplex missmatches take a look at the hidden config.xml option ( http://faq.pfsense.com/index.php?action=artikel&cat=10&id=38&artlang=en&highlight=hidden ) though I don't recommend messing with this. Another thing to try is a longer cable between pfSense and the switch (or maybe just another cable but I guess you tried that already). Some old sis nics had problems when too short cables were used. Maybe your switch has some issues with short cables too. If this doesn't help, try another nic at the pfSense end for this connection to see if that helps.

pmk · Jun 27, 2006, 2:42 PM

Embedding responses for the sake of clarity
What does status>interfaces tell for the speed and duplex of the interface that is connected to the dell switch?

[pmk] 100baseTX <full-duplex>for both the LAN and DMZ, WAN is unconnected currently.

What does the dell switch report?
[pmk] On the incoming interface from the ftp client. it's clean and green
Drop Events 0, CRC& Align Errors 0, Undersize Packets 0, Oversize Packets 0, Fragments 0, Jabbers 0, Collisions 0

Even interestingly enough the transmission is made of a large proportion of packets greater than 64bytes.
Frames of 64 Bytes 5342
Frames of 65 to 127 Bytes 1248
Frames of 128 to 255 Bytes 169044
Frames of 256 to 511 Bytes 1236
Frames of 512 to 1023 Bytes 610
Frames of 1024 to 1518 Bytes 52601

I'm assuming these stats are a summation of in & out.

On the PowerConnect interface connected to pfSense:
CRC& Align Errors 0, Undersize Packets 0, Oversize Packets 0, Fragments 0, Jabbers 0, Collisions 0
Frames of 64 Bytes 31879
Frames of 65 to 127 Bytes 12762
Frames of 128 to 255 Bytes 201
Frames of 256 to 511 Bytes 264
Frames of 512 to 1023 Bytes 56
Frames of 1024 to 1518 Bytes 1448

Not sure why i'm seeing fewer large frames here…because ethereal doesn't show the layer 4 activity to be too bad, some duplicate ACKs, but no Congestion Window Reduction...though it could have been set to a small value at the outset of the comms. Points for further investigation.

Also, do you see some kind of in/out errors and/or collisions?
[pmk] No errors, and given the fact that it's switched we'd expect no collisions, which is as reported by pfSense and the PowerConnect.

[pmk] The autonegotiation process occurs at the time the interfaces first sync ie connect, and i'm not sure if an interface can apply one set of capabilities in the inbound direction and a different set outbound with the a single link-partner. That'd be a buffering nightmare.

In case you have someduplex missmatches take a look at the hidden config.xml option ( http://faq.pfsense.com/index.php?action=artikel&cat=10&id=38&artlang=en&highlight=hidden ) though I don't recommend messing with this.
[pmk] I'll try this in read-only mode ;-)

Another thing to try is a longer cable between pfSense and the switch (or maybe just another cable but I guess you tried that already).
[pmk] I have tried different cables, a different gigabit switch, DOS and wsftp as ftp clients. Same result, slow in the up.

Some old sis nics had problems when too short cables were used. Maybe your switch has some issues with short cables too.
[pmk] The cables are CAT5e, 10m, manufactured ones. Though i've tried 3m ones too.

If this doesn't help, try another nic at the pfSense end for this connection to see if that helps.
[pmk] The issue always follows the LAN side of pfSense, but maybe that's because it's the only gigabit segment…i can put the test environment's lan (gigabit) onto pfSense's WAN, change the ip's around, and run the gigabit there to see what happens.

I wonder is anyone else using pfSense in a gigabit environment? [answers on a postcard please]

BTW top on both pfSense and proftpd boxes is showing:
CPU Free RAM Free/Total RAM
pfSense: 99% 208/512
pfoftpd 99% 20/128

I'll pop more ram into proftpd, just in case.

Thanks for your help.</full-duplex>

lsf · Jul 2, 2006, 3:02 AM

Any chance you have any network loops ? or possibly Traffick shaper issues ?

I have a Xeon 2.4 with 1 gig ram and 7* intel pro 1000 server cards (2 onboard and 5 pci cards). most of them are connected to some simple 8 port SMC gbit switches. and they all play nicely.

Not sure what you are seeing in your end. I never encountered something like that.

Maybe you FTP is BW trottled ? or the ftp has interface/duplex/cable issues ?

pmk · Jul 11, 2006, 9:53 PM

I really appreciate your guidance, and it's great to know it's just me ;-) because pfsense is really sharp!

To respond to your probing questions:

Nope, no loops, 1 switch in the test config, other than pfsense all test machines have single nics.

Traffic shaper is off.

If i re-cable everything with no config changes onto my Cisco 2900 10/100 switches…the issue goes away and the pfsense interfaces still negotiate to 100, full duplex. Changing gigabit switches from Dell to SMC made no difference.

The cables are the same, for all testing. So yes, it's possible that there's something in there to review.

The proftpd config has no throttling at all, it's a minmal proftpd.conf.

Our configs appear to be slightly different though: you've gigabit on the pfsense platform and (maybe 10/100??? elsewhere...the simple switches???) whereas i have gigabit switches and 10/100 on the pfsense platform.

My hardware platform is Dell GX150's with an onboard 3COM nic (i think), and 2x pci nics.

It is possible that i have been unlucky and hit two nic vendors with issues, though i think it's a 3COM onboard the Dell GX150, and the two pci nics are ASX 88814 (or something along those lines).

Though I've GX240s to try next with intel pci 10/100 nics, i'll let you know how it goes.

I appreciate the help, thanks.

pmk.