Trying to get pfSense QinQ to cooperate with extreme networks vman which is really just QinQ.
I gather from http://svn.freebsd.org/base/projects/amd64_xen_pv/contrib/tcpdump/ethertype.h that vman ethertype should be 0x8a88 and configured the extreme networks switch accordingly (default).
I also tried vman ethertypes 0x8100 and 0x9100.
Didn't get it to work, so I tried something else. Read below please.
Hmm.. after having no success with pfSense<–>Extreme Networks Switch, I tried a setup involving two pfSense boxes (on ALIX hardware).
First, I created a QinQ interface on each pfSense with parent Tag 10. I added one member 100. Those QinQ interfaces have the WAN physical interface assigned as their parent.
I then created a bridge with members OPT (physical interface) and vr_10_100 (QinQ interface).
I connected the two pfSenses together on their physical WAN interfaces and disabled all firewalling on both boxes.
So the setup looks like this:
For diagnostic purposes, I gave every interface an IP (Subnet 10.70.10.0/24):
10.70.10.10: OPT interface on pfSense1
10.70.10.20: vr_10_100 interface on pfSense1
10.70.10.30: OPT interface on pfSense2
10.70.10.40: vr_10_100 interface on pfSense2
Then I did some ICMP Ping testing. The results:
PC1 can ping 10.70.10.1 (itself), 10.70.10.10 (the OPT interface on pfSense1) and 10.70.10.20 (the vr_10_100 interface on pfSense1).
PC2 can ping 10.70.10.2 (itself), 10.70.10.30 (the OPT interface on pfSense2) and 10.70.10.40 (the vr_10_100 interface on pfSense2).
PC1 can NOT ping 10.70.10.2 (PC2), 10.70.10.30 (the OPT interface on pfSense2) or 10.70.10.40 (the vr_10_100 interface on pfSense2).
PC2 can NOT ping 10.70.10.1 (PC1), 10.70.10.10 (the OPT interface on pfSense1) or 10.70.10.20 (the vr_10_100 interface on pfSense1).
It appears as if the QinQ part is not working between the two pfSense boxes. Cabling checked ok.
Looks to me like you have a routing issue here.
What subnets are you using for the various interfaces?
When you ping, say, PC2 from PC1 how do you expect that packet to reach PC2?
It must go to the first pfSense box, which is configured as the default gateway for PC1, and then be routed to the second pfSense box which in turn routed it on to PC2. However since they are both in the same subnet PC1 won't send the packet to it's gateway but will look for PC2 as a directly connected local box.
Perhaps I have completely misunderstood this but I can't see how this could work. Very small subnet masks perhaps?
Edit: Ah, the OPT interfaces in each case are those that the client connects to? Thus bridging to the QinQ interface should pass packets. Your LAN interfaces are not shown then?
Yup, the OPT is the physical interface where the PCs connect (PC1 on pfSense1 OPT, PC2 on pfSense2 OPT).
All the IPs are in one and the same subnet 10.70.10.0/24. All Layer 2, no routing whatsoever involved.
If I connect PC1 and PC2 directly by crossover or switch, they can ping each other as expected.
If connected through the pfSense boxes, PC1 can see as far as the QinQ on pfSense1 and PC2 as far as the QinQ on pfSense2. But it doesnt work across the QinQ.
I didn't use the LAN interfaces in my setup, except for accessing the pfSense WebGUIs.
Ok. Have you tried this using just a VLAN connection or just a straight connection without any tagging?
Can you ping the other end from the pfSense CLI?
Since you are attempting to setup QinQ I assume you have some experience in these things but it's easy to completely misjudge this and offer either completely incomprehensible or very patronising advice.
Yep, VLANs work as expected.
PC1 can ping PC2 and vice versa.
Pinging anything across the WAN didn't work either from a shell on the pfSenses, with the QinQ setup I tried above.
Looks like packets pass across the bridge from the Physical OPT at least to the QinQ member vlan.
I have to wireshark on the WAN (QinQ) link to see if packets actually make it onto the wire.
Maybe it's some weird ARP issue?
I will have to recheck that the alix boxes I used had the bridging fix applied. I'm actually not sure cuz they were used in a previous lab setup!