Need help with VLANs and weird problem

[jhabers]

I have a very weird problem that I will try to explain to you:

I have 2 Dell Powerconnect 2824 Switched in Managed Mode
I have 2 PFSense firewalls with 4 NICs (2 WAN, 1 LAN, 1 pfsync)
I have 1 server with 2 NICs
I have 2 network ports for internet provided by the data center

The powerconnect switches are default except for me creating a VLAN2 on the first switch and a VLAN3 on the 2nd switch. I have the VLANs set to Untagged. I am using ports 17,18,19,20 on each switch for these VLANs

Internet port1 from the data center goes into SW1/port 17
Internet port2 from the data center goes into SW2/port 17

SW1/port18 goes into PF1/WAN1
SW2/port18 goes into PF1/WAN2
SW1/port19 goes into PF2/WAN1
SW2/port19 goes into PF2/WAN2

I have failover configured on each PFSense for the WAN's

I have a cable going from SW1/port24 to SW2/port24 to link the two default VLAN 1's

I have Server NIC 1 plugged into  SW1/port1
I have Server NIC 1 plugged into  SW2/port1

PF1 LAN port is plugged into SW1/port3
PF2 LAN port is plugged into SW2/port3

With this configuration if I unplug Server NIC 2 I cannot get to PF2 from the server. Its like the switches are not connected to each other.

Now here is the kicker, if I unplug Internet port1 from the data center that goes into SW1/port 17 I can now get to PF2. Its like that link is causing a loop or something, or I have my VLANs set up wrong.

Any help on this one?

[jhabers]

OK I did some more research and think I found out what is going on…correct me if I am wrong...

I found this statement
"PowerConnect switches are not PerVlan Spanning Tree aware like Cisco devices.  If you connect the two switches with both cables you will create a physical loop and one of the ports will go to blocking, thus blocking an entire vlan."

I believe what is happening is that the 2 internet connections are tied back together on the datacenters equipment and along with my crossover going from sw1 to sw2 its causing a loop since I am assuming that my switches dont support PVST.

So if this is the case is my only option to get 2 more physical switches and use them for each internet port and totally get rid of trying to do VLAN's?

Thanks
Jon

reference: http://en.community.dell.com/support-forums/network-switches/f/866/p/17002352/17133453.aspx

[Guest]

Or you could get switches that handle VLANs and Spanning Tree properly.

[clarknova]

I tried diagramming what you have described and it's pretty confusing. Mind you, I'm pretty novice.

One things stands out to me though, you have 2 switches, and yet you have opted to run LAN and WAN on both switches. If you want to review a thorough discussion on why this is bad, have a look at this recent thread on the mailing list:

http://marc.info/?l=pfsense-support&m=128098748819739&w=2

I've never used pfsync, so I won't comment on that, but I think the first thing to do would be to plug your internet connections and pfsense WAN ports into SW1. Plug your server NICs and pfsense LANs into SW2. Arrange your vlans the way you like and put your loop woes behind you.

[jhabers]

@clarknova:

I tried diagramming what you have described and it's pretty confusing. Mind you, I'm pretty novice.

One things stands out to me though, you have 2 switches, and yet you have opted to run LAN and WAN on both switches. If you want to review a thorough discussion on why this is bad, have a look at this recent thread on the mailing list:

http://marc.info/?l=pfsense-support&m=128098748819739&w=2

I've never used pfsync, so I won't comment on that, but I think the first thing to do would be to plug your internet connections and pfsense WAN ports into SW1. Plug your server NICs and pfsense LANs into SW2. Arrange your vlans the way you like and put your loop woes behind you.

For the LAN WAN issue on both switches I saw that issue too. I had bought 2 addition switches to handle the WAN connections and got rid of the VLAN problem. I ran into another problem though with WAN Failover not working right because the WANs were on the same subnet. So I ended up witha whole new solution:

I Got rid of the Dual WANs on each PF and just went with single WAN connctions on each. I plugged the independant WAN cables from the datacenter directly into each PF WAN port. I am just using the 2 Dell switches now and have the LANs of each PF going to different switches. All servers have 4 NICs that are loadbalanced, having 2 cables going to each switch for redundancy. This way I can loose a switch and everything will still work. I also set up carp between the PF's So I could loose a WAN and still get out and same with loosing a LAN. It seems this setup is alot cleaner and is working great. The 2 WAN lines coming from the datacenter are already running HSRP on their end.

Jon