Using OPT1 as another switched LAN port with DHCP?

[iamtommythorn]

I have perused the documentation, but I admit I have a hard time grasping the bewildering array of options. I have a Netgate XG-7100 and I'd like to use OPT1 and OPT2 just like LAN, that is, part of the same switch and with DHCP support. (In fact, I want to hang another switch off OPT1 as well).

So far I've only manage to connect hosts with static IPs. DHCP service doesn't appear to extend to OPT1 & OPT2. I'm clearly missing something obvious as to me this seems like to most natural usage.

[stephenw10]

The ix0 and ix1 ports are separate interfaces on the XG-7100. They are not directly connected to the internal switch with the Eth1-8 ports.

If you wanted them to be in the same subnet you would need to bridge them with the LAN. That is generally considered a bad idea though unless you really need to do it.

Steve

[iamtommythorn]

Thanks. What I do need is for hosts on the ix ports to talk to hosts on the LAN ports. I don't care if they are on the same subnet. What is the best (simplest?) way to achieve this? (I'm a bit puzzled by how else people are using the ix ports). EDIT: why is it bad?

[stephenw10]

It's bad because if you want things to be on the same subnet it's almost always better to use a switch.

Bridging interfaces in pfSense makes it behave (mostly) like a switch but it still has to process all the traffic as though it's a router/firewall which requires a lot of CPU cycles.
Sometimes that's exactly what you want, devices on the same subnet but traffic between them filtered, but most of the time we see people doing this it's just to make it act like a switch.

If you don't need them on the same subnet then just connect you other devices to the ix ports in a different subnet and pfSense will route between them.

Steve

[iamtommythorn]

Thanks, I'm still bewildered as to how people use the OPT ports.
Is the standard approach really to have different subnets on each?
In that case, how do I get DHCP to serve those subnets?
Since on the XG-7100 the OPTs are [the only] 10 GbE ports, it's unsurprising that it's where my servers go.

Is there documentation somewhere that details the expected usage, including the OPT ports?

[akuma1x]

You use those ix opt ports on the 7100 to connect to other 10Gb SFP ports on other high speed switches or high speed hosts.

Passing traffic across subnets in pfsense is simply a matter of creating firewall rules on one or both interfaces to move the type(s) of traffic you’re using.

Jeff

[stephenw10]

You can enable the DHCP server on any interface that has a static IP. If you have enabled either of the OPT interfaces and given them a static IP and subnet you can enable dhcp on those as you would any other interface.

And, yes, those ports are usually used if you need 10G connectivity so usually to a 10G switch with other things connected to it.

Steve

[iamtommythorn]

Thanks. I haven't change my interface assignments so they remain

WAN VLAN 4090 on lagg0 (WAN)
LAN VLAN 4091 on lagg0 (LAN)
OPT1 ix0 (...)
OPT2 ix1 (...)
Available network ports: BRIDGE0 (My Lan switch)

Both of my OPT1 (ix0) and OPT2 (ix1) interfaces are enabled and IPv4 configuration type set to Static IPv4. They have address 192.168.11.1 and 192.168.12.1 respectively. Nothing else was touched.

However going to the Services/DHCP Server menu option I only see "LAN".

Clearly my mental model is failing me here, but I'd like to end up with these three subset, each with DHCP enabled, and routing between them.

Any pointers or help much appreciated.

(I feel my use case must be so standard and obvious that I'm disappointed I have so much difficulty getting it working.)

UPDATE: I just noticed that it had defaulted to a /32 subnet on those two interfaces and fixing that made the DHCP option show up, so assume I can figure out the firewall rules (TBD) I should be good.

[iamtommythorn]

Nope, could not get that working.

My Firewall / Rules / LAN already had two rules (IPv4 and IPv6 respectively) with source LAN (Why? all traffic flows on the LAN switch regardless - so confused) and I added another IPv4 rule except with the source being the OP1 net.

The Firewall / Rules / OPT1 already had two rules called "Default allow OPT1 to any rule) with source set to LAN net.

Oddly enough my servers on OPT1 can ping and access LAN hosts, but not the other way around.

[mogarchy]

My most common use case for OPT ports is secondary WAN, also comes in handy if a port fails -- you can just quickly reassign.

[jpvonhemel]

@mogarchy

How often do ethernet ports “fail?”. Are we talking realtek hardware failure or a failure that is software and resolved with a reboot? I have read that the realtek ports is not as high of quality as intel ones.

[iamtommythorn]

Progress: the settings are as below (and my "Why?" question remains).

I have a macOS, FreeBSD, and Ubuntu host on the 192.168.11.0/24 OPT1 network. They are all getting an address from DHCP, but frustratingly pings to 192.168.11.1 times out and indeed no traffic reaches beyond this subnet.

[mogarchy]

@jpvonhemel Maybe more often than you might expect. I have several hundred devices in the field and have probably had this come up 10-15 times. All netgate hardware.

[mogarchy]

@iamtommythorn Your rules on OPT1 NET need to have OPT1 net as source, not LAN net

[iamtommythorn]

@mogarchy THANK YOU! That was the trick. Clearly I don't understand how the rules is supposed to read.