VLANs and PFsense
-
Almost 3 years ago I created this thread https://forum.pfsense.org/index.php?topic=51807.msg277105#msg277105
And also this ticket https://redmine.pfsense.org/issues/2577I'm running this router on an Atom system with 2 Intel NICs
To cater the 85 LANs on my network I need to make use of VLANsAll this was working until my system broke down 3 years ago and I replaced the motherboard with that Atom system.
I couldn't get LAN working.
I tried a lot until I got it working by turning off VLAN hardware acceleration.I created a ticket but that one got rejected without really giving more comments than "it works here"
I solved it myself by creating a cronjob turning the hardware acceleration off.In the meanwhile I replaced the motherboard with a full-size PC and bought more expensive Intel NICs in the hope to get it working without that trick.
Alas, it didn't
I resorted using that same trick again. This cronjob:*/4 * * * * root ifconfig igb0 | grep -q VLAN_HWTAG && ifconfig igb0 -vlanhwtag && logger "I had to remove hardware tagging on igb0"Recently we received 50 Mbit fibre.
I've been investigating performance issues for more than 3 weeks when I finally discovered it was again this vlan hardware acceleration.
It turned out it needed this on the NIC where I have my WAN-interfaces.Although I like many aspects of Pfsense I was very disappointed by the handling of this issue.
Especially because I think I thoroughly investigated it. -
No reply..
Not 3 years ago.
Not now….Still the same problem....
-
Is your hardware in the Compatibility List?
This is for FreeBSD10 now since pfSense 2.2 uses this as base system. -
Yes, the Intel Ethernet drivers (igb) are on that list.
But I've tried many NICs.
The consensus here is that I can't go wrong with Intel.Has anyone tested it with 85 VLANS?
I had 2 identical Atom-based Pfsense systems and I sold them.
They are now, already for a while, running without any problems a configuration with only a few VLANS.
They run fine.I'm waiting for that Netgate motherboard to get released in Februari.
It has 4 NICs and I want to give that fibre-connection its own NIC. -
What kind of switches are you using? Creating Vlans is pretty straight forward. If you are using older Cisco Gear make sure that you are using 802.1Q and not ISL. I've used VLANs on some really cheap NIC $5.00 not 85 but close to 20.
-
Yes, the Intel Ethernet drivers (igb) are on that list.
Has anyone tested it with 85 VLANS?Have you tested it with -say- 5 VLANs?
We can rule out general errors if that works right. -
As I wrote already it works with only a "few" vlans
I have 2 NICs
1 NIC I'm using for the WAN-connections and the other NIC is for all the LAN-connections.Many LAN-connections don't work until I turn off vlan hwtagging with the command "ifconfig igb0 -vlanhwtag"
It's this way for 3 years on different hardware.
Each time I change hardware I'm secretly hoping it is working without turning that off.On the WAN side I have about 5 VLANs.
The latest one I added was the 50 Mbit fibre connection.
I was having performance problems on that which went away if I directed the traffic to a 6 Mbit ADSL-line.
Very strange…
I had 6 Mbit throughput on that ADSL-line, but only 0.5 Mbit on the 50 Mbit line.
I attached a laptop to the core switch where I had that VLAN untagged and got my 50 Mbit without any issues.Then I used that same trick on the WAN-NIC and all of a sudden I have 50 Mbit throughput...
It's as if that vlan is performing less because it is later created....
Again... no problems if vlanhwtagging is offI fail to understand why this could have anything to do with my switches.
As I said it's working as I want it as long as I have hardware vlan tagging turned off.I'm using a Netgear GS724T as my core-router and many GS108T's for distributing the VLANs in a campus-like situation.
Most GS108T's are configured like this
On port 1 they receive all the VLANs tagged and on port 2 they give all the VLANs to the next GS108T minus the ones that are meant for that appartment-blockI have little room for testing as many people depend on this router.
I'm now again hoping it will work on the Netgate motherboard with 4 NICs that's getting released on february.Today I tried an update to 2.2 but then it has the same behaviour as with vlanhwtagging turned on.
So my trick doesn't work on 2.2 anymore.
If this is a FreeBSD issue I don't know. -
Sorry, I'm out.
I'll never touch Netgear Prosafe switches again, as I've seen too much odd behavior up to complete fails with those devices in the past.I'm using a Netgear GS724T as my core-router …
Those switches are L2 only, hence it can only be your core-switch. pfSense does the routing for you, right?
Most GS108T's are configured like this
On port 1 they receive all the VLANs tagged and on port 2 they give all the VLANs to the next GS108T…Are those ports 1 & 2 configured as Trunk-ports?
Do you use LACP and Jumbo-frames in your setup somewhere?
How many GS108Ts are in your setup?
-
That switch can only have up to 64 VLANs according to https://forum.pfsense.org/index.php?action=post;topic=87222.0;last_msg=480875 This may be the cause of some of your issues.
edit: forgot to add 24 port based Vlans
-
That switch can only have up to 64 VLANs according to https://forum.pfsense.org/index.php?action=post;topic=87222.0;last_msg=480875 This may be the cause of some of your issues.
edit: forgot to add 24 port based Vlans
Nice catch.
OP - have you tried another switch?
I know it seems like it HAS to be the vlanhwtag on the interface, and you've tried multiple hardware on pfSense. That might point you in a direction other than the hardware you keep changing out to something that you haven't.
There was just a bug fixed in captive portal for 2.2.1 that didn't raise its ugly head until 117 VLANs were attached to the same CP. And that was because the rule was getting too long.
Do you have any of those old systems that also exhibited this behavior? Maybe you can get something going on the bench to test it. I know. It's a lot to build. I'm trying to think of a good way to do it. Probably some perl scripting to generate the configs, two pfSenses and a Cisco 3550.
If I do get a wild hair and decide to lab this up, what is the exact behavior you're seeing? Is it something straightforward like not being able to ping the pfSense VLAN interface IP address at all or is it more nuanced like slow performance intermittently? Are you running any limiters, shapers, captive portals or anything else it might also be? Any virtualized pfSenses? I don't think I have any igb(4) nics. It'll be em(4).
-
Sorry, I'm out.
I'll never touch Netgear Prosafe switches again, as I've seen too much odd behavior up to complete fails with those devices in the past.How could these Netgear switches be involved with this issue?
You seem to be missing that it's working without a problem as long as I disable "hardware vlan tagging" on the NIC.
This is about the NIC interfacing with FreeBSD.Furthermore it works fine and full throttle if I untag that fibre vlan to a specific port and have a laptop connected to it.
I'm not denying any issues around Netgear, but I think I've already identified the culprit.
According to specs I have 128 static VLANs
I have no reason to believe it's less. Especially because I'm able to define these.I just went through the trouble of adding another 36 bogus vlans on my switch.
I got this message when trying to define the 129th vlan.
-
Are you going to answer my questions or not?
You're the one who came here for help. I'm labbing this thing up as we speak.
Here's the deal. You have a layer 2 problem. You are using shit switches, apparently daisy-chained. Doesn't take a genius to start going in the right direction.
What, exactly, are you seeing?
If I do get a wild hair and decide to lab this up, what is the exact behavior you're seeing? Is it something straightforward like not being able to ping the pfSense VLAN interface IP address at all or is it more nuanced like slow performance intermittently? Are you running any limiters, shapers, captive portals or anything else it might also be? Any virtualized pfSenses?
-
-
I see you tried updating to 2.2 now. Sorry. Have you tried fresh install?
I'm not aware of pfsense having vlan issues. Mine never has.
-
If I don't enable that cronjob that turns off vlan hardware tagging I'm getting this behaviour.
My office is in vlan100 and I have assigned 10.0.0.138 to pfsense.
I start pinging the router.
I do a reboot of the pfsense system. It comes up and for a short while I'm able to ping the router as it's creating the vlans.
Then this stops.
If I walk to the console of the pfsense I'm having an Internet connection and am able to ping addresses on the Internet.
I can't ping anything in the vlan100 office LAN.If I invoke ifconfig igb0 -vlanhwtag it starts working.
And the whole network is then working.
I don't have full performance on my fibre network though. I need to invoke ifconfig igb1 -vlanhwtag for that…But why being this defensive?
It seems it's all focused on getting any other culprit than the pfsense system. -
WHAT VERSION OF PFSENSE?
-
That's because if there was a VLAN HW TAGGING problem in FreeBSD everyone would already know about it, bro. Google it. It doesn't exist.
WE have to help YOU figure out what's wrong in YOUR network so we can help YOU unwrong it.
-
That's because if there was a VLAN HW TAGGING problem in FreeBSD everyone would already know about it, bro. Google it. It doesn't exist.
WE have to help YOU figure out what's fucked in YOUR network so we can help YOU unfuck it.
As I said…
You've already found the culprit....
This is exactly what made the challenger explode....And you probably didn't notice I helped myself 3 years ago by finding that solution.
If this is the attitude with which you are offering help I like to pass on that...
I'm offering you feedback, I guess you're blind for that. -
Complaints I've seen recently that I know are not true.
Pfsense can't VLAN
Pfsense can't NAT
Pfsense can't resolve
Pfsense can't routeLook deep enough, long enough and you will always find OP is making a simple mistake.
So, just need to go ahead and drop the idea that pfsense can't or won't vlan and find where the user error is. -
Complaints I've seen recently that I know are not true.
Pfsense can't VLAN
Pfsense can't NAT
Pfsense can't resolve
Pfsense can't routeLook deep enough, long enough and you will always find OP is making a simple mistake.
So, just need to go ahead and drop the idea that pfsense can't or won't vlan and find where the user error is.You are paraphrasing me.
I have a network here that's working in full as long as I turn off vlan hardware tagging.3 years ago I had a working setup with hardware I can't remember exactly what it was.
That hardware broke down and I replaced it with a dual NIC atom board that was introduced at the time. Something with DCC2500…
I took the config.xml of the previous router and was unable to get it working again.
... until I turned off vlan hardware tagging...
