LAGG and VPNs

stevencavanagh

It is only the link from router to switch 1 that is 100M. The link from switch 1 to switch 2 is 1G as it always has been, no config changes have been done in switch 2. The VLANs in pfsense are the same as the existing ones in switch 2 but not sure why the NAS can be accessed (LAG on switch 2 to NAS) but no other client as VLANSs in pf sense are set up the same way. Unless it is the fact that the NAS is on a trunk (ie the LAG)??

stephenw10

A trunk would usually imply a link carrying multiple VLANs. If the NAS is on one or more VLANs dircetly then that could be different to any other clients which likely are using access ports (untagged).

But I would be trying to solve the 100M link negotiation problem. A lagg of two 100M links is of questionable benefit IMO. The fact that is behaving oddly implies other unexpected behaviour could be related.

stevencavanagh

The NAS was intended to be accessible from at least a couple of VLANs so that may explain that. However, the remaining VLANs should be on their own, which is why they are down as 'access' rather than 'trunk' I assume. I would have thought pfsense would still handle that though.

stephenw10

Sure if all the vlans are configured correctly pfSense will route between them.
But if the NAS is multihomed on several VLANs directly and you are able to reach it but not untagged clients that implies a VLAN error somewhere.

stevencavanagh

AS nothing has changed on switch 2 where all these are connected and I can connect to them currently i.e through the Draytek router, then when I replace the Draytek with pfsense does that not imply the issue is somewhere in the pfsense vlan config rather than the switch?

stevencavanagh

Also, is there a way to get to the port details of the NICs that are used in the pfsense box as I think I've been through every conceivable config on the switch to solve the 100M LAG speed issue. Was going to try and force the speed of both the switch and pfsense NIC port to 1000M

stephenw10

If you run ifconfig -vvvma you will see the full status of all the NICs and interfaces.

I agree it seems unlikely that the switch config would be wrong if it worked previously. I would be running pcaps to see what's actually on the wire.

I would also try connecting to the switch without a lagg configured to rule that out.

stevencavanagh

cheers, I'll go and have another play with it!

stevencavanagh

I used ifconfig -vvma and got the following:-

igb2: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
	capabilities=4f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
	ether 98:b7:85:00:fd:44
	media: Ethernet autoselect
	status: no carrier
	supported media:
		media autoselect
		media 1000baseT
		media 1000baseT mediaopt full-duplex
		media 100baseTX mediaopt full-duplex
		media 100baseTX
		media 10baseT/UTP mediaopt full-duplex
		media 10baseT/UTP
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
igb3: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
	capabilities=4f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
	ether 98:b7:85:00:fd:44
	hwaddr 98:b7:85:00:fd:45
	media: Ethernet autoselect
	status: no carrier
	supported media:
		media autoselect
		media 1000baseT
		media 1000baseT mediaopt full-duplex
		media 100baseTX mediaopt full-duplex
		media 100baseTX
		media 10baseT/UTP mediaopt full-duplex
		media 10baseT/UTP
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Note : I have set the LAG at 1000M to try but it causes it to fail.

I notice that there is no option to set igb2 & igb3 speeds to 1000M, at least not whilst they are in the LAG or is there?

stephenw10

The interfaces in the LAGG should inherit settings if you set it. That output above show those links are not linked at all though, were they even connected?

stevencavanagh

so I will need to delete the LAGG and set both interfaces to 1000M manually and then re-create the LAGG.

The LAGG was working but it always syncs to 100M. It is currently failed because I have set the switch manually to 1000M for those ports - something Draytek tech support asked me to try

stephenw10

It should inherit that setting from lagg0 but it's set as autoselect there.

Unconfiguring the lagg and then configuring it again would be no change. It is possible to make changes to links in a lagg with a manual command that can be run at boot.

As a test just run:
ifconfig igb2 media 1000baseT

However I would get a single link working at 1G first and then add the lagg back.

stevencavanagh

trying to get a single link but unsuccessful so far!

stevencavanagh

I have now tried every possible combination available to force the port to 1G (no LAG) but unable get it. If I connect the same cable from laptop to pfsense NIC I get 1G so not card or cable. Looks like I will have to go back to draytek to sort it as I'm now out of ideas!

stephenw10

What about if you use the em0 NIC?

stevencavanagh

Not tried using that but I'll give it a shot and see what happens!

stevencavanagh

Ok, swapped over to em0 and it connected at 1G!

So, took the same lead and plugged into a number of ports on both switches and always connected at 1G.

Consequently, went around each of the 5 ports individually with the same lead and results below:-

em0 - 1G
igb0 - 1G
igb1 - 1G
igb2 - 100M
igb3 - 100M

Hence card 2 has the problem. Cards 1&2 are the same type and new.

stevencavanagh

swapped over the LAN cards out of desperation and now have a 1G connection (not LAG)...

However, ignoring LAGG for now, I have a physical connection between the 2 switches but unable to access anything on the other switch. Must be VLAN issue but not sure where to go from here

stevencavanagh

So, I now have everything up in terms of connections but running on a single cable, not LAGG on igb1.

Firewall is fully open which at some point I will need to lock down but LAG now the remaining issue.

NIC 2 seems to be the issue.

Initially I installed pfsense with only 1 NIC card in and then added the second later once I had bought it. Pfsense detected it so I assumed all was good. Could that be causing the issue?

stephenw10

No it shouldn't make any difference how the cards were added. I agree there's something different about that second card.

Check the revision is the same as shown by: pciconf -lv

It could be a firmware difference. Looks for the eeprom version in the boot log like:

igb0: EEPROM V3.11-0 eTrack 0x80000469