802.1p/q pfsense setup
-
The netgear GS108Tv2 switch came today. Holy number of settings, Batman. I can't really be sure if I got them right or not, I was a little bit guessing having never really dug into Layer 2 like this. I ended up pulling the switch entirely out of the picture, but the pfSense box still wasn't able to pick up an IP address via DHCP when plugged into the OTN. I might be completely wrong, but I thought that was supposed to work - albeit with severely degraded bandwidth.
Prior to that, at one point I had things all messed up, and the pfSense WAN picked up an IP address from its LAN - I think that was because I had the VLAN mappings in the switch goofed. At least it tells me the WAN interface is capable of accepting and processing DHCP traffic, acting as a dhcp client.
The only things I changed on the pfsense were the settings in the doc, mostly the stuff around the IGMP traffic. The GFNB was able to talk to the OTN immediately through the same port on the patch panel. I might try a simpler off-the-shelf netgear router tomorrow, just to see what happens.
I posted a few screenshots of what the switch configuration looks like based on Atlantisman's document: https://www.dropbox.com/sh/ug31k8t6n9618ni/ligIuMmIiQ/gs108t_screenshots?lst. There is no way to delete or rename the first three VLANs. I really don't know what impact VLAN 2 being "Voice VLAN" has. I can disable "Voice VLAN" in another screen, or try to move it to VLAN 3, but it doesn't change anything as far as I can tell.
edit: corrected bad syntax. sorry, long day.
-
The netgear GS108Tv2 switch came today. Holy number of settings, Batman. I can't really be sure if I got them right or not, I was a little bit guessing having never really dug into Layer 2 like this. I ended up pulling the switch entirely out of the picture, but the pfSense box still wasn't able to pick up an IP address via DHCP when plugged into the OTN. I might be completely wrong, but I thought that was supposed to work - albeit with severely degraded bandwidth.
Prior to that, at one point I had things all messed up, and the pfSense WAN picked up an IP address from its LAN - I think that was because I had the VLAN mappings in the switch goofed. At least it tells me the WAN interface is capable of accepting and processing DHCP traffic, acting as a dhcp client.
The only things I changed on the pfsense were the settings in the doc, mostly the stuff around the IGMP traffic. The GFNB was able to talk to the OTN immediately through the same port on the patch panel. I might try a simpler off-the-shelf netgear router tomorrow, just to see what happens.
I posted a few screenshots of what the switch configuration looks like based on Atlantisman's document: https://www.dropbox.com/sh/ug31k8t6n9618ni/ligIuMmIiQ/gs108t_screenshots?lst. There is no way to delete or rename the first three VLANs. I really don't know what impact VLAN 2 being "Voice VLAN" has. I can disable "Voice VLAN" in another screen, or try to move it to VLAN 3, but it doesn't change anything as far as I can tell.
edit: corrected bad syntax. sorry, long day.
1. Yes it should work without the QoS settings, just highly reduced bandwidth (mine did, at about 930/10).
2. Are you plugging anything else into the other ports on the switch? Or just pfsense and the OTN?
3. Are you running pfsense in a VM or anything like that? If so you'd have to configure vlans on the virtual switch in esxi/hyper-v.
4. It shouldn't matter that it is labelled voice vlan. Port one and two do need to be on VLAN 2 no matter what though.
5. Some switches come with two different vlan options (private and normal), make sure you are not configuring a private vlan, otherwise the OTN won't send you packets.
Those screenshots look right to me, though i am not too familiar with that particular switch.
-
Thanks for the info.
1. Yes it should work without the QoS settings, just highly reduced bandwidth (mine did, at about 930/10).
2. Are you plugging anything else into the other ports on the switch? Or just pfsense and the OTN?
I tried setting up ports 1 (OTN), 2 (pfSense WAN) on VLAN 2 and the rest on VLAN 1 to isolate them. No luck. I also just plugged in the OTN and pfSense to the switch (everything else removed), and wiring my laptop into the pfSense LAN port to monitor the pfSense, no luck there either.
3. Are you running pfsense in a VM or anything like that? If so you'd have to configure vlans on the virtual switch in esxi/hyper-v.
No, it is running on metal. I'm starting to wonder if having switch port 2 tagged is causing an issue. I think the pfSense WAN interface MTU is 1492 but I'll have to check.
Would it be appropriate to set switch port 1 to tagged and port 2 to untagged? Both members of VLAN 2 as your point #4 states, yes.
5. Some switches come with two different vlan options (private and normal), make sure you are not configuring a private vlan, otherwise the OTN won't send you packets.
I don't see any options like those, but I'll keep looking.
-
Thanks for the info.
1. Yes it should work without the QoS settings, just highly reduced bandwidth (mine did, at about 930/10).
2. Are you plugging anything else into the other ports on the switch? Or just pfsense and the OTN?
I tried setting up ports 1 (OTN), 2 (pfSense WAN) on VLAN 2 and the rest on VLAN 1 to isolate them. No luck. I also just plugged in the OTN and pfSense to the switch (everything else removed), and wiring my laptop into the pfSense LAN port to monitor the pfSense, no luck there either.
3. Are you running pfsense in a VM or anything like that? If so you'd have to configure vlans on the virtual switch in esxi/hyper-v.
No, it is running on metal. I'm starting to wonder if having switch port 2 tagged is causing an issue. I think the pfSense WAN interface MTU is 1492 but I'll have to check.
Would it be appropriate to set switch port 1 to tagged and port 2 to untagged? Both members of VLAN 2 as your point #4 states, yes.
5. Some switches come with two different vlan options (private and normal), make sure you are not configuring a private vlan, otherwise the OTN won't send you packets.
I don't see any options like those, but I'll keep looking.
I would setup the switch with a different port (3-8). After the switch is setup unplug everything but the OTN and the pfsense box.
You may also need to setup vlans in pfsense, though i didn't have to. There is no reason why this wouldn't work.
-
I would setup the switch with a different port (3-8). After the switch is setup unplug everything but the OTN and the pfsense box.
You may also need to setup vlans in pfsense, though i didn't have to. There is no reason why this wouldn't work.
Switch or no switch, nothing except the GFNB so far seems to be able to be plugged into the OTN. Tried putting the TimeCapsule in DHCP+NAT mode (normally I just have it in bridge mode), WAN port plugged into the OTN and just like the pfsense box, it was unable to obtain a WAN DHCP address.
From everything I understand, this should be working but unfortunately I'm unable to make any progress until I can sort out why the traffic isn't making it past the OTN unless it sees a GFNB. (That's probably not the correct description of the relationship.)
Putting a laptop directly on the OTN and there was network traffic, but no response to DHCP client requests.
Edit: No luck spoofing the GFNB's MAC address on the laptop, and no luck manually configuring the IPv4 settings (with the spoofed MAC address).
 -
Score.
I don't have time tonight to mess with it anymore, but on a hunch, I figured out that I could create a VLAN virtual interface on my macbook. I gave it VLAN ID 2, plugged it into the OTN and immediately got a reply from the WAN DHCP server. So my problem is likely an issue where I'm going to have to either figure out what I'm doing wrong with the switch and/or get pfSense to use a virtual interface on VLAN 2 for the WAN side.
-
Score.
I don't have time tonight to mess with it anymore, but on a hunch, I figured out that I could create a VLAN virtual interface on my macbook. I gave it VLAN ID 2, plugged it into the OTN and immediately got a reply from the WAN DHCP server. So my problem is likely an issue where I'm going to have to either figure out what I'm doing wrong with the switch and/or get pfSense to use a virtual interface on VLAN 2 for the WAN side.
I had an issue where the DHCP on the WAN side would only assign a total of 2-3 Public IP addresses. So you could have the same issue with the DHCP servers holding your reservations, that's why it worked when you plugged the macbook in. So you might try spoofing the macbook's mac address to your pfsense machine and it might work.
EDIT: Also, i did some reading on that switch and it does have two different types of VLANs, port based (or private), and 802.1Q (the one you need). Be sure you're using the proper VLANs on the switch.
-
I had an issue where the DHCP on the WAN side would only assign a total of 2-3 Public IP addresses. So you could have the same issue with the DHCP servers holding your reservations, that's why it worked when you plugged the macbook in. So you might try spoofing the macbook's mac address to your pfsense machine and it might work.
EDIT: Also, i did some reading on that switch and it does have two different types of VLANs, port based (or private), and 802.1Q (the one you need). Be sure you're using the proper VLANs on the switch.
I finally did get things working partially. I could get the WAN interface up properly using DHCP as I said before. I was also able to get DNS queries to return correctly. However, I was not able to get any other traffic to the internet until I discovered that pfSense has a really strange way of coming up with the routing table:
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 192.168.2.1 UGS 0 8226 nfe1
8.8.8.8 00:04:4b:02:4c:92 UHS 0 124 nfe0
10.16.0.0/16 192.168.2.1 US 0 0 nfe1
10.30.0.0/16 192.168.2.1 US 0 0 nfe1
23.255.128.0/19 link#3 U 0 376 nfe0
23.255.146.22 link#3 UHS 0 0 lo0
127.0.0.1 link#7 UH 0 32 lo0
192.119.23.198 00:04:4b:02:4c:92 UHS 0 124 nfe0
192.168.2.0/24 link#4 U 0 630 nfe1
192.168.2.1 link#4 UHS 0 752 lo0The default route is completely wrong. I used a route command to fix that and set the ISP gateway properly. I also used a route command to delete the route to the 8.8.8.8 DNS server (again, no idea where these are coming from). I'm looking at the pfSense web ui and the Status > Gateways has the correct information (even before I manually fixed the routing table). Somehow that isn't translating into a correct routing table. Outside of the adjustments described here, the only configuration change I've made to the routing are the two static routes for the TV.
Fixing the gateway allowed traffic to the internet (ie I can telnet to an smtp server from the pfsense box). However, I'm now very suspicious of the rest of the routing table because I still can't get traffic from the LAN to the Internet. I'm able to write this post because I have an ssh tunnel to the pfSense box from my laptop. I left the NAT settings alone, but something could be wrong there?
I'm pretty confident the switch isn't the issue at this point. I'm accessing the pfSense box over a wireless AP that is plugged into the switch, on the same VLAN (1) as the pfSense LAN interface. The OTN and the pfSense WAN link are on the same VLAN 2. It seems like both VLANs are behaving properly. pfSense seems like the issue but I'm confused as to how it is coming up with some of its configuration so it is unclear where to look.
Edit: I should clarify - nfe0 is the WAN interface, and nfe1 is the LAN. The corrected routing table has this entry for the gateway:
default 23.255.128.1 UGS 0 463 nfe0
-
Just in case it matters and I've messed up the switch config to somehow cause the weird behavior - screenshots of the VLAN configuration in the switch. Port 1 is the OTN, port 2 is the pfSense WAN; port 7 is the wireless access point, port 8 is the pfSense LAN.
https://www.dropbox.com/sh/ug31k8t6n9618ni/ligIuMmIiQ/gs108t_screenshots
-
It looks like you may have fallen into the trap of adding a gateway to the LAN interface which, since it's done after WAN, then becomes the default. You shouldn't have a gateway on LAN at all. A lot of people seem to be doing that recently for some reason.
The correct place to set the default gateway (and remove any spurious ones) is System: Routing: Gateway:Steve
-
It looks like you may have fallen into the trap of adding a gateway to the LAN interface which, since it's done after WAN, then becomes the default. You shouldn't have a gateway on LAN at all. A lot of people seem to be doing that recently for some reason.
The correct place to set the default gateway (and remove any spurious ones) is System: Routing: Gateway:Steve
I didn't intentionally or explicitly add a gateway to the LAN interface that I can recall. You're right, it doesn't make sense for the LAN interface to have a gateway. I saw under System > Routing > Gateway that there is one for the LAN, and one for the WAN. I thought it was a little odd, but figured it must be the way pfSense is presenting the configuration in the UI.
The only possible time I can think when I might have done something to cause this LAN GW to end up in the routing table is setting up the LAN DHCP server. It is possible there was a question during that portion of the initial setup I should have left blank - probably thinking the question was asking what GW should the DHCP clients use.
-
Like you say it should be blank. If you change the LAN subnet at the initial console setup it asks you questions in order (IP address, subnet mask etc) and one of those is the gateway. It's hard to just return through it when it's explicitly asking you for the gateway.
The wording there especially could be changed to prevent this.https://forum.pfsense.org/index.php/topic,72694.0.html
If you have entered a gateway on LAN remove it from Interfaces: LAN: and then go to System: Routing: Gateways: and remove it there too making sure the WAN gateway is set as default.
Steve
-
You shouldn't have a gateway on LAN at all. A lot of people seem to be doing that recently for some reason
Btw. this happened to me when I set up pfSense from the serial console (on an ALIX board if that matters). I'm absolutely sure that I did not create a gateway, I logged every single step of my setup.
-flo-
-
I had an issue where the DHCP on the WAN side would only assign a total of 2-3 Public IP addresses. So you could have the same issue with the DHCP servers holding your reservations, that's why it worked when you plugged the macbook in. So you might try spoofing the macbook's mac address to your pfsense machine and it might work.
EDIT: Also, i did some reading on that switch and it does have two different types of VLANs, port based (or private), and 802.1Q (the one you need). Be sure you're using the proper VLANs on the switch.
I finally got it working to the point where I could get everything on the Internet. Thanks to Stephenw10 for the help on the routing table stuff.
However, the best speed I'm able to get is 30/10, which tells me I haven't figured out the QoS stuff yet. I apologize, I know the QoS stuff isn't strictly pfSense, but rather is configured in the switch. I'm banging my head trying to figure it out. The manual seems useless but maybe it will make sense to someone else?
http://www.downloads.netgear.com/files/GS108Tv2/gs108tv2_gs110TP_usermanual.pdf
There are two ways to configure QoS. CoS seems to mostly appear to be hardware based QoS internal to the switch. The DiffServ way seems to be what I need. I'm digging around in the DiffServ and nothing I try is making any difference. To make it simple, I'm trying to set everything to priority 3 and then once I figure that out try to handle DHCP, IGMP, and other separately.
There appear to be three levels of configuration: Class, Policy, and Service. The class looks like it is the filtering which matches the packet to be handled. The only setting I have there is VLAN 2. The service is where you map a policy to an interface.
https://dl.dropboxusercontent.com/u/36902/gs108t_screenshots/Screen%20Shot%202014-02-21%20at%2022.22.43.png
https://dl.dropboxusercontent.com/u/36902/gs108t_screenshots/Screen%20Shot%202014-02-21%20at%2022.23.04.png
https://dl.dropboxusercontent.com/u/36902/gs108t_screenshots/Screen%20Shot%202014-02-21%20at%2022.23.13.pngIt looks like the policy is where the real work happens. I've tried setting the policy COS to 3, the IP precedence to 3, and the IP DSCP to both cs3 and cs1, not really clear which one of these sets the correct bits. Nada - same speed test result. I'm running the test on ethernet through the tv box, but I fully expect from past tests to see something ~ 140/130.
Sorry if I'm missing something obvious here, but any ideas?
-
Finally got everything working. Part of the problem was the speed test was giving really bad results. I wrote up the instructions for configuring the Netgear GS108Tv2. Comments or other feedback is welcome. The QoS part especially was long enough that I broke VLAN and QoS into separate posts.
Part 1 - http://flyovercountry.org/2014/02/google-fiber-gigabit-speeds-your-router-part-1-vlans/
Part 2 - http://flyovercountry.org/2014/02/google-fiber-gigabit-speeds-your-router-part-2-qos/ -
I am using the netgear gs108t v2 switch and a pfsense box running the latest release. I have the switch set correctly as my internet connection is full speed both ways 984/978. The tv's guide comes up but no video is shown. I followed 1.2 version of the guide pdf starting from section 2 (setting up TV). Any ideas on something I could try to get my tv services back up and going?
-
I am using the netgear gs108t v2 switch and a pfsense box running the latest release. I have the switch set correctly as my internet connection is full speed both ways 984/978. The tv's guide comes up but no video is shown. I followed 1.2 version of the guide pdf starting from section 2 (setting up TV). Any ideas on something I could try to get my tv services back up and going?
One of the things that is easy to miss is setting the correct option on the 4 firewall rules:
Scroll down to Advanced Features -> Advanced Options and check the first box., It should read, “This allows packets with OP options to pass. Otherwise they are blocked by default. This is usually only seen with multicast traffic.”
At one point, I had the option set on only three of the rules and it caused weird issues.
-
You should only need the set opts box checked on the default ALLOW ALL rule in Firewall -> Rules -> LAN.
Also, it seems like pfsense doesn't handle the IGMP traffic (at least for me) 100% effectively, causing little hiccups in tv service where it stops working 10-15 seconds, i am still investigating this issue and will be doing more testing with pfsense 2.1.1
-
You should only need the set opts box checked on the default ALLOW ALL rule in Firewall -> Rules -> LAN.
D'oh! Maybe that's part of my problem. Completely misunderstood the doc on that. You did say default rule, not the IGMP FW rules. My fault.
-
Is IPv6 working for you all when you go test it? It doesn't seem to be working for me anymore.
