VLANs and PFsense

frater

There was no refusal to give specific info.
While I was answering one question 3 others came in and told

Why oh why are you all so busy putting the blame on my Netgear 724Tv3.
First one says it only supports 64 vlans, then the next tells him how good he so quickly proved I was in error.
I checked the specs and according to specs it does 128 vlans and to be more thorough I added so many vlans that I hit that ceiling of 128 vlans…
While I'm testing this another 3 messages come in telling me what I'm also probably doing wrong...

I am not in an ideal situation here. I'm on a small peninsula with almost 100 places I have to provide with Internet.
Before I came the cables were already in place and there were some soho routers at certain places.
I've used that same infrastructure using daisy-chained GS108T netgear switches.
1 building has several 108T switches directly connected to the GS724T and the 2nd building has a GS108T as well with many GS108T's behind it.
This network could have been much cleaner, but given the real world I live in, the amount of money I can spend it's a good network.
Furthermore it's a network that's not built from scratch.

I have no reason at all to think there's something wrong with my layer-2 network besides you saying so.
I had a moment with 2 pfsenses. 1 working and 1 not working. The config.xml on the running pfsense was the same as the config.xml of the other. I only did a search/replace to change the realtek card into an Intel card (em0 and em1). I couldn't get the new pfsense system to work until I turned off "hardware vlan tagging". The one with the realtek cards and scrap hardware was working without any problem.

If my layer-2 network is so wrong how in earth could turning off vlan hardware tagging fix that. Why don't you give me an answer to that question instead of attacking me like wolves only because I'm asking if you also know of some problem with the Intel NICs I'm using?

jahonix

@frater:

If my layer-2 network is so wrong how in earth could turning off vlan hardware tagging fix that.

Because it cures an upstream problem of incorrect VLAN tag handling in the switch?

Don't worry, not only Netgear has its problems.
HP switches don't handle IGMPv2 properly in bigger IP-TV systems (approx. 1000 IP HD Cams). In a casino install there were HP switches for about $1M replaced by Cisco even after HP engineers had been on site and couldn't solve it. The system worked immediately with the Ciscos in place.

Which version of GS108 switches do you use - there are quite some with very different capabilities!

frater

I have GS108T switches after the GS724T switches…
I had only GS108E and don't use that anymore.
The GS108T have an IP and can be managed without a utility...

I have both NICs connected to the Netgear GS724Tv3 switch.
I have created 6 VLANs on NIC igb1 (10~15) and about 80 on NIC igb0.

On igb1 I have a 10.250.250.1/25 and on igb0 it's 10.250.250.129/25
I don't think it's the best way to do it, but I'm using the 10.250.250.0/25 network to reach all the GS108T's through pfsense.

On 10.250.250.10 I have the GS724T configured.
I send VLAN1 to the next building with about 50 VLANs (mostly the LAN interfaces, but also vlan14 which is a NAT adsl router/modem with VoIP server)
That first GS108T will have all these VLANs configured and sends VLAN1 to 2 GS108T's. 1 GS108T has a few vlans, the 2nd GS108T has many vlans and sends these to the next GS108T

From that GS108T there are several GS108T's in series until it hits 1 GS108T which splits it into 2direction. From this moment on I've already reached a new building...
I have no drawing of this network and I just described only a part of it.

Each vlan has a 10.0.x.0/24 interface to it.
If the vlan is 150 then it will get a 10.0.150.0/24 network.

I haven't done this now, but 3 years ago when I was faced with this problem for the first time I only used only 1 GS108T on which I only extracted vlan100
In that test situation I was unable to ping my pfsense if I turned off vlan hardware tagging...

...
It's too complex to describe really,,,
I think it's best to have another go with new hardware.
This new hardware (a Netgate) is something I need anyway. Not to solve this problem (as I already have a solution for it).

jahonix

@frater:

I have GS108T switches…

There are at least GS108Tv1 and GS108Tv2 versions available.
Do you run the latest firmware on all GS108T switches?

~~The interconnect ports are configured as Trunk or as General Port~~
After reading the manual it seems as if those switches can't be configured with Trunk ports at all. You can only stack tagged PVIDs on a port.
Why would anyone want to massively trunk and daisy-chain them then? :-\

Those switches do have VLAN issues which makes it suspicious for others:```
Known issues: Port PVID (Switching > VLAN > Advanced > Port PVID Configuration) does not automatically changed back to 1 after its associated VLAN is deleted.
Workaround: Manually change the PVID back to 1.
Limitations: Combined MAC and IP ACL do not work with double VLAN tagged traffic.

mikeisfly

Let's see a diagram. http://www.gliffy.com// as much detail as possible.

frater

I have these GS108T's
Several are attached to the GS724T and only 1 has to handle more than 40 VLANs
I don't use trunks.
I don't use ACL's
Only port-based vlans

model: GS108Tv2
boot: B5.1.0.2
revision: 5.0.5.10

The one involved in handling the fibre connection only has to deal with 7 VLANs and is an endpoint (2 WAN connections and 5 LAN-connections).

For the problem of having no IP-traffic going from the office network (vlan100) to the Pfsense there is no GS108T involved.
The GS724T switch and Pfsense are in the same office.
I have a normal switch attached to a port on the GS724T which has vlan100 untagged on it.

The issue of having slow performing traffic does have a GS108T attached to the GS724T.
The funny thing there is that it also carries the vlan for a 6 Mbit ADSL-modem.
The GS108T sits in a house where both the 50 Mbit fibre switch is and that 6 Mbit ADSL-modem.
If I don't remove vlan hardware tagging I will have slow performance on the 50 Mbit connection (2~3 Mbit). That 6 Mbit connection is full speed all the time.

When I was troubleshooting this performance problem I wasn't suspecting the Pfsense at all. My prime suspect was the ISP giving us the fibre connection.
I made an alias for all the WeTransfer IP networks (Amazon's).
I found out that if I direct the traffic to the 6 Mbit ADSL modem's gateway I would get full speed.
The default route to fibre gave me only 2 Mbit.
I then needed to test directly on the fibre switch. Because of bad weather I decided to check the speed with a laptop connected to the GS724T switch.
I was amazed to get that full speed.
Only as a long shot I tried to turn off that hardware tagging for the NIC that holds the WAN-interfaces.
I was again amazed to find out I can now have full speed on the fibre connection through Pfsense.

I'm not saying it's definitely NOT the Netgear, but my prime suspect is still the NICs or the way they are interfacing with FreeBSD.

AFAIK I'm running Intel server NICs.

As a side note I once installed Windows SBS2011 on an Intel Desktop board.
I was only able to find drivers for desktop Windows operating systems.
I did some searching and it had to do with Intel having NICs with vlan issues that were used on Desktop boards.
For this reason there were no drivers for the server OS's
Patching the inf file of the server drivers to be able to use the desktop NICs was a solution (not an elegant one).
It wasn't my idea to buy a desktop board in the first place.

I'm not using expensive Intel NICs, but they aren't cheap either.
It does give me an uneasy feeling about these Intel NICs.

I just googled again and found this:
http://www.ivobeerens.nl/2012/08/08/enable-the-intel-82579v-nic-in-windows-server-2012/

I haven't read this (just stumbled on it) and it mentions hardware vlan tagging:
http://wiki.wireshark.org/CaptureSetup/VLAN

I would really like to put it to rest until my Netgate motherboard is ordered and arrived.

I always thought my NICs just didn't properly support hardware assisted vlan tagging and the FreeBSD drivers were not capable of making a difference between those Intel NICs that do and those that don't.
Again… not substantiated...

It's not that strange for hardware vendors to sell faulty chips and merely disable certain features.

kejianshi

I hope this gets resolved with the new board.

frater

@Derelict:

That's because if there was a VLAN HW TAGGING problem in FreeBSD everyone would already know about it, bro. Google it. It doesn't exist.

WE have to help YOU figure out what's wrong in YOUR network so we can help YOU unwrong it.

Googled:
https://www.freebsd.org/cgi/man.cgi?query=em%284%29&sektion=#end

kejianshi

BUGS
Hardware-assisted VLAN processing is disabled by default. You can enable
it on an em interface using ifconfig(8).

I'm not reading where that says its broken - just that the bug is that is comes disabled by default.

Of course, I've never gone in turning it on and off either… If the only bug you have is that it only works when Hardware-assist is on, then leave it off.
Does leaving it off give you a noticed performance hit?

frater

http://blog.gmane.org/gmane.comp.security.firewalls.m0n0wall.devel/month=20091201

doktornotor

Why the heck are you referring to some 2009 thread regarding m0n0wall/FreeBSD 6.4? Additionally involving Realtek NICs and Xen? ::) :o

Still could not be bothered with replacing the Netgears with another switch brand to do some basic debugging?

frater

@kejianshi:

BUGS
Hardware-assisted VLAN processing is disabled by default. You can enable
it on an em interface using ifconfig(8).

I'm not reading where that says its broken - just that the bug is that is comes disabled by default.

Of course, I've never gone in turning it on and off either… If the only bug you have is that it only works when Hardware-assist is on, then leave it off.
Does leaving it off give you a noticed performance hit?

No, my system only works when vlan hardware tagging is turned OFF
It's on by default…

I think they didn't trust hardware vlan tagging then and decided to turn it off by default.
Nowadays it's turned on by default.

But now I would like to start drinking beers...

frater

@doktornotor:

Why the heck are you referring to some 2009 thread regarding m0n0wall/FreeBSD 6.4? Additionally involving Xen? ::) :o

Still could not be bothered with replacing the Netgears with another switch brand to do some basic debugging?

I've done more debugging than I'm capable of writing here.
I have at least found a solution to my problem and I wasn't coming for a solution here…
You are all presuming too much.

I'm not even sure I had these Netgear switches then (3 years ago). I still had a HP Procurve.
But let me guess... these are shite too?

I have no problem using other brands of hardware like VoIP phones to walk through my network and go where I want them to go.
Only pfsense has a problem and that's solved by turning off vlan hardware tagging.
Again... maybe pfsense does things the proper way and all the other hardware the wrong way.
Maybe there is some user error when we finally find out.
But all of you already ruled out the option that it's indeed PFsense (FreeBSD) having issues with (certain) Intel cards.

I'm off for my beers and try to resist coming back to this forum the next month.
I think I was able to stay away for more than 2 years the last time.

doktornotor

I mean, instead of wasting days googling for completely irrelevant reasons why FreeBSD sucks…

Try different NICs
Try different switch

Instead of this very basic troubleshooting/diagnostics we repeatedly get tirades about DSL modem and laptops?!

frater

We have several managed switches. All of them Netgear.
I have tried several NICs. They all didn't work and paid too much for it.
I am waiting now for that Netgate motherboard.
It has 4 NICs and 1 NIC I plan to dedicate to the 50 Mbit fibre.
I will of course test it first with my current config.

And now I'm really gone…

doktornotor

OK, none of NICs worked. All switches Netgear… Either VLAN's HW tagging is completely broken for any NIC on FreeBSD, or Netgear is POS. Guess which is more likely.

hda

@frater:

model: GS108Tv2
boot: B5.1.0.2
revision: 5.0.5.10

Case, if, but, and, there is a firmware revision 5.4.2.13 8)

jahonix

@doktornotor:

Either VLAN's HW tagging is completely broken for any NIC on FreeBSD, or Netgear is POS. Guess which is more likely.

He won't hear you. Even if you'd tattoo it on his forehead he wouldn't.
Unfortunately, the Netgate won't solve his problem either for obvious reason. Go figure what he's doing then…

Nature of a forum is that we're all talking at once.
Every time questions come too close to the Netgear switches he prefers to answer other posts instead.
We cannot "win" this thread unless we get someone on-site, swap-in another brand switch and show it is working (or solve other unrecognized problems).

One of the other problems might be ground loops! He's connecting different buildings and only mentioned two fiber connections. Without proper isolation and unless this is a three building only campus it screams for ground loop problems. The L2/L3 problems might only be mediocre compared to L1...

jahonix

@hda:

…there is a firmware revision 5.4.2.13

As I just wrote: every time we come close to his Netgear Babies he jumps on another post.

Derelict

@frater:

I haven't read this (just stumbled on it) and it mentions hardware vlan tagging:
http://wiki.wireshark.org/CaptureSetup/VLAN

Which is why, for anything but the simplest of captures, I like to use a switch mirror port.

If you want to have a prayer at solving this problem, you need to consider the possibility that you are seeing a problem with the combination of your NIC and your switching gear. This is a combination apparently nobody else on the planet is experiencing, meaning it's unique to something about the way your network is rigged. Pretty much nobody else ever feels the need to touch vlanhwtag on em(4) NICs.

Considering you've tried multiple NICs and they all do the same thing…..

It might be as simple as another switch that accepts all VLANs tagged on two ports between the NIC and the netgear switch.

I presume all these switch ports also pass VLAN1 untagged to manage all these netgear switches. Or do they support a management VLAN? I know my GS108PE doesn't.