Single NIC pfsense box with Netgear GS108E - no WAN link.
-
Hi all - I tested all of this in a non-production setup and it all worked as expected. Moving the switch and pfSense PC upstream to the cable modem versus downstream of the wireless router / switch (ASUS RT-N53) was an epic fail.
I've seen other threads on this topic, mostly dealing with folks trying to figure out how to properly configure the VLANs…but I'm pretty sure my VLAN config is okay.
The hardware:
pfSense PC - Intel NUC D34010WYK (single NIC PC with Intel 1000 NIC). pfSense version 2.14:
Defined interfaces:
WAN em0_vlan10
LAN em0_vlan80
OPT1 ath0_wlan0 (present, down, not configured yet)Switch - Netgear GS108Ev2 firmware 1.12:
Defined VLANs: 1, 10, 80, 99
VLAN Port Membership:
VLAN 1 - port 7 (maintenance)
VLAN 10 - port 1 Untagged, port 8 tagged
VLAN 80 - port 2-6 Untagged, port 8 tagged
VLAN 99 - port 8 untaggedPort VID's:
Port 1 VID 10
Port 2-6 VID 80
Port 7 VID 1
Port 8 VID 99In non-production test, connections are as follows:
Cisco 3008 Cable Modem <=> ASUS RT-N53 Wireless Router WAN Port
ASUS RT-N53 LAN Port 1 <=> GS108E Port 1 (VLAN-10)
GS108E Port 8 (802.1q Trunk w/ VLANs 10, 80, 99) <=> pfSense (Intel NUC) NIC
GS108E Ports 2 through 6 (VLAN 80) - Test PCspfSense WAN (VLAN-10) pulled an IP address off of the ASUS router LAN side;
devices connected to any LAN (VLAN-80) port on the switch pulled IP address from pfSense DHCP.
All LAN test devices had access to Internet, pfSense pulled down packages, resolved DNS.After several hours of testing I was confident that moving the switch and pfSense "router on a stick" between the cable modem and the ASUS box would likewise be successful. No such luck.
Connecting Switch Port 1 (VLAN 10 WAN) directly to cable modem ethernet port, and NUC ethernet interface to Switch Port 8 (Trunk w/ VLAN 10, 80, 99) would indicate that a connection existed (layer 1? layer 2?)
In normal operation, Link LED (e.g. while connected to ASUS box) changes from green to bright yellow/amber to indicate that layer 3 connection is up. This will not happen with the pfSense NUC connected through the switch. :(
Obviously the pfSense box VLAN configuration precludes connecting it directly to the cable modem.
To troubleshoot, I tried the following:
- Power cycled cable modem - pulled power for 2 minutes. Disconnected cable to switch.
- Restored power, waited for all green lights (except for ethernet link) and reconnected switch. Link light goes green. Does not transition to yellow...pfSense NUC WAN interface does not get DHCP address from cable modem.
- Connected test PC to switch LAN ports (2 through 6) to verify pfSense was serving out DHCP LAN addresses on all of these ports.
- Modified switch vlan config to make ports 1 & 2 members of VLAN 1 with VID 1, both Untagged. Disconnected NUC from switch. Connected ASUS router to switch. Link established in seconds.
- Disconnected ASUS router from switch port 2. Connected laptop PC to switch port 2. PC negotiated DHCP address in seconds, and had unrestricted access to Internet. This step pretty much rules out any MAC address issues, but just to be certain, I tested with a new, unopened, default configured ASUS router (I bought 4 of them cheap-cheap about a year ago, and had only used 1) connected to port 2 - went through the config wizard and all went as expected - no connection issues.
- Applied MAC address of "old" ASUS router WAN to WAN VLAN interface of pfSense. Restored VLAN 10/80 configuration to switch. Connected NUC to switch port 8. Kept my fingers crossed, to no avail. Still no link.
I have (I think) ruled out MAC spoofing as a solution. 3008 modem spec states that the unit supports up to 64 customer-side devices...so I gather the MAC table on the modem LAN side has 64 possible entries...and I assume that these are stored in volatile memory and not persistent across reboots / power resets...safe to say the bridge table isn't full. Safer still to say that the modem has no trouble serving a MAC it hasn't seen before, so at a loss as to why this isn't working.
Any ideas? It's a head-scratcher to me.
Non-production tests downstream of the ASUS router validated pfSense and switch VLAN configuration; and live-tested the cables used to connect all devices.
One thought comes to mind - speed and duplex settings. I hard set everything but the switch ports (which are not that configurable) to 100BaseT Full Duplex. My service is 30/10 (theoretically) and we don't run bandwidth intensive apps on the home LAN yet...though this might change when I finish the FreeNAS and media server builds...
Would eliminating GigE negotiation require a cross-over cable between modem and switch port? I am of course willing to try that, but it seems to me everything build in the last several years is auto-sense capable.
Any pointers or suggestions greatly appreciated.
I have limited windows of opportunity to do live testing. My wife and I both work from our home. Loss of network services is a Bad Thing(tm) for both of us; and when the kids are home? Fuggedaboudit. Thirty minutes without youtube, VOD, or online gaming is, to them, on par with severe neglect or child abuse.
Time to go pull out my hair some more. Best of the evening to you all!
-drew-
-
Did you call your cable company explaining you were putting a new router in and the MAC address would be changing and get their advice?
Resetting the modem should do it but I've had to call them. They reset something and it starts working (Cox).
What is VLAN 99 and why is it untagged on your pfsense port?
-
TLDR!
Take it down to two VLAN's, ditch everything else, report back. This is unreadable. I have the Core i5 NUC with the same switch and it works fine. If you don't get DHCP then you may have configured the VLAN or pfSense incorrectly and your top priority should be to get a ping going between your trunk port and untagged port … test with any hardware you have on hand.
-
Pardon folks - should have TLDR'd most of that…and thanks for your responses!
@ Derelict: I don't think it's a MAC issue - connecting the new wireless router to the cable modem proved that.
@ rkuo - the config as-is worked when I had it connected this way:
cable modem--ASUS router-wan
ASUS router lan--switch port-1
switch port 8--pfsense NUC trunk
switch port 2--laptoppfsense WAN interface (VLAN 10) pulled IP address from ASUS DHCP Server
pfsense LAN interface (VLAN 80) served DHCP addresses to laptop on ports 2 thru 6
pings LAN to WAN worked - as did web browsing, mail, etc.
That said, per your suggestion, next time I get unimpeded access to the WAN connection, I'll try again - with as skinny a config as possible; and report back how it went and what I had mis-configured. It could be awhile.
Thanks for the support!
-drew-
-
@ Derelict: I don't think it's a MAC issue - connecting the new wireless router to the cable modem proved that.
Proved nothing. Do not think that cable modem service DHCP works like anything close to normal.
do this on the switch:
VLAN 10 - port 1 Untagged, port 8 tagged
VLAN 80 - port 2-6 Untagged, port 8 taggedPut the cable modem on port 1, factory config pfSense on port 8 (ONLY Exceptions to default: WAN em0_vlan10, LAN em0_vlan80)
Plug your LAN devices into ports 2-6.
If it doesn't work, it's not pfSense. Look elsewhere for your problem.
It really is as simple as that.