WAN2 + LAN on same physical network not working

[murdof]

Hi guys,

I have the following setup:
PFSense box - 4 NICs
Cheap 1GBit switch: Dlink GO-SW-8G

LAN: 10.0.0.0/24 - connects to GO-SW-8G
WAN1: PPPoE - VDSL connects to VDSL modem directly with cable
WAN2: 10.0.1.2/24 - connects to 5G router (10.0.1.1/24) over its own cable

My setup works ok and all. Problem is that I have all my cables coming into the center of the room in a closet hence my 5G reception is not that great.
My target is to move my 5G to another room which has a window. I have moved the modem there as a standalone and I see that I'm getting 100Mbit more.

So I try the following:

LAN: 10.0.0.0/24 - connects to GO-SW-8G
WAN1: PPPoE - VDSL connects to VDSL modem directly with cable
WAN2: 10.0.1.2/24 - connects GO-SW-8G that links over gigabit cable to the other room. It connects to a WiFi Router that has 4 Gigabit ports. The 5G router (10.0.1.1/24) over is then connected to the WiFi Router.

This starts working for a while - I'm getting PFSense notification that the 5G link is up but I can not ping the router at all (10.0.0.1) and traffic comes and goes. If I unplug the 5G router cable then I can login to the router. When I plug it in I lose connection again.

Note that although I'm using the same physical LAN (over 2 switches) I have two separate subnets (10.0.0.0/24 for LAN and 10.0.1.0/24 for 5G router).

Do I need to go the VLAN route here to tag only the 5G modem ports on the two switches and create a trunk?
Why would I need VLANs though.
Isn't it normal to have two subnets on the same physical network?

Maybe the cable from LAN,WAN2 that end up to the same switch are causing the issue? I assume that pfsense should offer isolation for that.

Maybe there is another config that I can do with a VIP?

Any advise would be greatly appreciated

[viragomann]

@murdof said in WAN2 + LAN on same physical network not working:

LAN: 10.0.0.0/24 - connects to GO-SW-8G
WAN1: PPPoE - VDSL connects to VDSL modem directly with cable
WAN2: 10.0.1.2/24 - connects GO-SW-8G that links over gigabit cable to the other room.

This might end up in ARP errors.
The communication in an L2 is based on ARP. When you connect two NIC to the same switch without VLAN separation, pfSense doesn't know which interface to go out to access a certain MAC.

So get a VLAN switch to connect the AP and the 5G router to untagged ports and a trunk port to pfSense.

[murdof]

@viragomann Thanks for this.
I have already ordered the switches to set this up.

I made a bit more testing on this though and I have found the following:

If I have VDSL and 5G on different Tiers in the Gateway groups all works well.

If I switch them to the same tier then things start to collapse - I can't even ping other devices that are connected physically to the same switch.
Restarting, unplugging the 5G modem sometimes fixes it but I need to have them on separate tiers to get a stable connection.

Does the above behavior still point to the ARP issue?