LAGG and VPNs

stephenw10

How is the LAGG configured at each end? It sounds like the switch is probably disabling the ports due to a loop detection / STP because it's not expecting a LAGG. Or at least not of that type.

Steve

stevencavanagh

Hi,

Draytek switch is set up as:-

LACP, auto negotiation (speed), fast timeout, priority 1, load balance algorithm "IP/MAC address", Flow control enabled

Pfsense : Set as LACP, static. Basically standard set up as per the guides

However, not sure what IP address I should enter for the static one in "interfaces". When I try to add the ip address of the switch, which is a static one it says already used in the LAN interface as the IP is 192.168.0.XXX/24 and the switch ip is 192.168.0.YYY

Can't easily get to the pfsense screen at the moment as I've had to put it all back to using the draytek router as WFH today, so will be able to change it all back after 5pm

stephenw10

pfSense uses slow timeout by default for LACP so i might just be that. Check the Status > Interfaces page or run ifconfig and see what the status of the lagg links are.

For example:

[23.09-DEVELOPMENT][admin@3100.stevew.lan]/root: ifconfig -v lagg0
lagg0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	description: LAGG0
	options=800bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
	ether 00:08:a2:0c:0b:ba
	hwaddr 00:00:00:00:00:00
	inet 192.168.221.10 netmask 0xffffff00 broadcast 192.168.221.255
	inet6 fe80::208:a2ff:fe0c:bba%lagg0 prefixlen 64 scopeid 0xd
	laggproto lacp lagghash l2,l3,l4
	lagg options:
		flags=14<USE_NUMA,LACP_STRICT>
		flowid_shift: 16
	lagg statistics:
		active ports: 2
		flapping: 0
	lag id: [(8000,00-08-A2-0C-0B-BA,01AB,0000,0000),
		 (8000,00-90-0B-76-8E-55,018B,0000,0000)]
	laggport: mvneta0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
		[(8000,00-08-A2-0C-0B-BA,01AB,8000,0001),
		 (8000,00-90-0B-76-8E-55,018B,8000,0006)]
	laggport: mvneta2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
		[(8000,00-08-A2-0C-0B-BA,01AB,8000,0008),
		 (8000,00-90-0B-76-8E-55,018B,8000,0005)]
	groups: lagg
	media: Ethernet autoselect
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

stevencavanagh

Think I changed to slow at one point to try it but will try again.

I will also run the ifconfig and see what I get.

Cheers
Steve

stevencavanagh

Forgot to mention that the LAGG link show green on the status page until I plug the patch cables in at which point they remain green for a short whilst then fail but no traffic goes through.

stephenw10

The link LEDs on the ports remain lit though?

stevencavanagh

The LAGG port link lights are and have always been off

stevencavanagh

The switch also indicates the LAG is down

stephenw10

Ok, that implies the switch is shutting down the ports which in turn implies it's seeing a flood/loop to trigger that.

stevencavanagh

so, i currently have:
Screenshot 2023-08-24 190337.png

stevencavanagh

Not sure where the loop could be!

The switch has 4 connections associated with pfsense:-

The WAN link - igb0.101
The 2 LAG links - igb2 & igb3
Additional link for this laptop - igb1

stevencavanagh

log for LAGG interface:-

+ Screenshot 2023-08-24 192635.png

stephenw10

Ok, so I'd guess there's some misconfiguration on the switch for the VLANs that are separating those 4 links and it's creating a loop.

stevencavanagh

Quite possibly but I cannot find it.

It is now back working with the draytek router without issue and all i have done is change the short timeout back to long, switch back to static from LACP on the switch and swap the WAN cable from modem to router! Spoke with draytek this morning and they couldn’t see an issue with the setup either.

Unless the loop is being created via the connections to the pfsense box somehow?

stephenw10

Unlikely unless you have and bridged NICs in pfSense.

Does the Draytek router use the same 4 links to the switch?

Does the LAGG come up correctly if only those links are connected between pfSense and the switch?

stevencavanagh

The router has 2 links to the switch (LAG) and there are 4 links from this switch to another but that is all working. Yes, the draytek router is using the same links.

Never seen the LAGG come up yet to the pfsense either with just those links or others.

Is it possible to let me know what the pfsense settings should be for a LAGG with VPNs and I can try and make the switch match, although there aren't that many options really on the draytek switch to get wrong to be fair.

stephenw10

@stevencavanagh said in LAGG and VPNs:

The switch has 4 connections associated with pfsense:-

The WAN link - igb0.101
The 2 LAG links - igb2 & igb3
Additional link for this laptop - igb1

What are the 4 connections then? That looks like 4 NICs in pfSense that are connected to the switch no?

Any VPNs you might have setup wouldn't have any effect here this is a layer 2 or even 1 issue.

I assume you are able to see a link between pfSense and the switch if there isn't a lagg in play?

stevencavanagh

the first connection is from the pfsense to the modem (igb0.101)
the LAGG (2 connections igb2 & igb3) from pfsense to the switch
the last is igb1 which is a connection from the pfsense to the laptop

Yes, if the LAGG is removed and a single cable put direct to the switch from the pfsense (obviously different port from the LAGG) then I get a connection, although at 100MB not 1GB as it should! No idea why though.

For info - the pfsense box is a DELL PC (i5) with 2 twin port NICs, giving a total of 5 ports if you include the motherboard one.

stephenw10

Ah, Ok only two links to the switch. Hard to see how that could be a loop then. The fact it only links at 100M is not a great sign! Is it set to fixed speed in the switch maybe?

stevencavanagh

Nope, changed the patch cable, changed the port and forced it 1GB where it stopped working completely so put back to auto and it auto negotiates at 100M