Problem with 2nd vlan on lagg dot1q vlan interface
-
This is a supposedly standard configuration. I am using a lagg etherchannel to trunk 2 vlans.
Been trying to track down a problem with one of the vlans.
I have spent many many hours on this and it won’t work.
I think I better ask for help!
pfsense ips:
vlan160 192.168.160.2/24
vlan120 172.16.120.2/29Cisco svi ips:
vlan160 192.168.160.1/24
vlan120 172.16.120.1/29pfsense end of lacp dot1q link:
arp table:
? (192.168.160.2) at 00:0d:60:f3:15:cc on lagg0_vlan160 permanent [vlan]
? (172.16.120.1) at 00:0d:65:c8:78:7f on lagg0_vlan120 expires in 227 seconds [vlan]
pfsense (172.16.120.2) at 00:0d:60:f3:15:cc on lagg0_vlan120 permanent [vlan]ifconfig:
bge0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=8009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate>ether 00:0d:60:f3:15:cc
inet6 fe80::20d:60ff:fef3:15cc%bge0 prefixlen 64 scopeid 0x5
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
bge1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=8009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate>ether 00:0d:60:f3:15:cc
inet6 fe80::20d:60ff:fef3:15cd%bge1 prefixlen 64 scopeid 0x6
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lagg0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=8009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate>ether 00:0d:60:f3:15:cc
inet6 fe80::20d:60ff:fef3:15cc%lagg0 prefixlen 64 scopeid 0xb
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect
status: active
laggproto lacp
laggport: bge1 flags=1c <active,collecting,distributing>laggport: bge0 flags=1c <active,collecting,distributing>lagg0_vlan120: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=3 <rxcsum,txcsum>ether 00:0d:60:f3:15:cc
inet6 fe80::230:6eff:fe46:a9dc%lagg0_vlan120 prefixlen 64 scopeid 0xc
inet 172.16.120.2 netmask 0xfffffff8 broadcast 172.16.120.7
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect
status: active
vlan: 120 parent interface: lagg0
lagg0_vlan160: flags=108843 <up,broadcast,running,simplex,multicast,ipfw_filter>metric 0 mtu 1500
options=3 <rxcsum,txcsum>ether 00:0d:60:f3:15:cc
inet6 fe80::230:6eff:fe46:a9dc%lagg0_vlan160 prefixlen 64 scopeid 0xd
inet 192.168.160.2 netmask 0xffffff00 broadcast 192.168.160.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect
status: active
vlan: 160 parent interface: lagg0Switch end of dot1q lacp:
sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 172.16.120.1 - 000d.65c8.787f ARPA Vlan120
Internet 172.16.120.2 5 000d.60f3.15cc ARPA Vlan120
Internet 192.168.160.1 - 000d.65c8.787f ARPA Vlan160c4506(config-if)#do sh int po45
Port-channel45 is up, line protocol is up (connected)
Hardware is EtherChannel, address is 000d.ed0a.ac2c (bia 000d.ed0a.ac2c)
MTU 1500 bytes, BW 2000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is N/A
input flow-control is on, output flow-control is unsupported
Members in this channel: Gi2/45 Gi2/46
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1695000 bits/sec, 158 packets/sec
5 minute output rate 60000 bits/sec, 87 packets/sec
47514206 packets input, 31299672513 bytes, 0 no buffer
Received 48757 broadcasts (20806 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 input packets with dribble condition detected
42130817 packets output, 28239898431 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped outinterface Port-channel45
switchport
switchport trunk encapsulation dot1q
switchport trunk native vlan 666
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk
!
interface GigabitEthernet2/45
switchport trunk encapsulation dot1q
switchport trunk native vlan 666
switchport mode trunk
switchport nonegotiate
channel-protocol lacp
channel-group 45 mode passive
!
interface GigabitEthernet2/46
switchport trunk encapsulation dot1q
switchport trunk native vlan 666
switchport mode trunk
switchport nonegotiate
channel-protocol lacp
channel-group 45 mode passive
endsh etherchannel 45 de
Group state = L2
Ports: 2 Maxports = 8
Port-channels: 1 Max Port-channels = 1
Protocol: LACP
Minimum Links: 0
Ports in the group:
–-----------------
Port: Gi2/45Port state = Up Mstr Assoc In-Bndl
Channel group = 45 Mode = Passive Gcchange = -
Port-channel = Po45 GC = - Pseudo port-channel = Po45
Port index = 0 Load = 0x00 Protocol = LACPFlags: S - Device is sending Slow LACPDUs F - Device is sending fast LACPDUs.
A - Device is in active mode. P - Device is in passive mode.Local information:
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Gi2/45 SP bndl 32768 0x2D 0x2D 0x31 0x3CPartner's information:
LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi2/45 SA 32768 000d.60f3.15cc 20s 0x0 0x170 0x5 0x3DAge of the port in the current state: 0d:00h:00m:48s
Port: Gi2/46
Port state = Up Mstr Assoc In-Bndl
Channel group = 45 Mode = Passive Gcchange = -
Port-channel = Po45 GC = - Pseudo port-channel = Po45
Port index = 1 Load = 0x00 Protocol = LACPFlags: S - Device is sending Slow LACPDUs F - Device is sending fast LACPDUs.
A - Device is in active mode. P - Device is in passive mode.Local information:
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Gi2/46 SP bndl 32768 0x2D 0x2D 0x32 0x3CPartner's information:
LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi2/46 SA 32768 000d.60f3.15cc 19s 0x0 0x170 0x6 0x3DAge of the port in the current state: 0d:00h:00m:47s
Port-channels in the group:
---------------------------Port-channel: Po45 (Primary Aggregator)
Age of the Port-channel = 4d:01h:54m:18s
Logical slot/port = 11/45 Number of ports = 2
Port state = Port-channel Ag-Inuse
Protocol = LACP
Port security = DisabledPorts in the Port-channel:
Index Load Port EC state No of bits
------+------+------+------------------+-----------
0 00 Gi2/45 Passive 0
1 00 Gi2/46 Passive 0Time since last port bundled: 0d:00h:00m:50s Gi2/46
Time since last port Un-bundled: 0d:00h:03m:39s Gi2/46#sh vlan brief
VLAN Name Status Ports
120 pfsense (internet) active
160 external active
666 untagged-traffic active#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static routeGateway of last resort is 172.16.120.2 to network 0.0.0.0
C 192.168.160.0/24 is directly connected, Vlan160
172.16.0.0/29 is subnetted, 1 subnets
C 172.16.120.0 is directly connected, Vlan120
S* 0.0.0.0/0 [1/0] via 172.16.120.2Can't ping either of the 129.168.160.0 interfaces across the trunk yet both the 172.16.160.0/29 links function as expected.
It looks like vlan 160 is not being trunked on lagg0 even though it all looks correct (except the arp tables).
I have now reached the end of my world and entering uncharted territory.any ideas if configure error or possible bug?
2.0-RC2 (i386)
built on Tue May 24 18:59:38 EDT 2011Mark</performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast,ipfw_filter></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,promisc,simplex,multicast></active,collecting,distributing></active,collecting,distributing></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate></up,broadcast,running,promisc,simplex,multicast>
-
tcpdump on lagg0 with -e, make sure you're getting traffic tagged correctly. If not, it's a switch config issue.
-
tcpdump -e -i lagg0 -G15
yes I can see bootp/dhcp and arp traffic, and ipv6 traffic, thats a bit odd isn't it?
Using SVI and access lists I have had to route everything down lagg_vlan120 to pfsense until I get the 2nd vlan working.
tcpdump -e -i lagg0 vlan 160
tcpdump: WARNING: lagg0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lagg0, link-type EN10MB (Ethernet), capture size 96 bytes
17:30:14.953683 70:1a:0401:78 (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5.65113 > ff02::c.1900: UDP, length 146
17:30:14.956316 70:1a:0401:78 (oui Unknown) > 33:33:ff:60:91:a8 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5 > ff02::1:ff60:91a8: ICMP6, neighbor solicitation, who has fe80::d452:5831:f560:91a8, length 32
17:30:15.184303 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8.55059 > ff02::c.1900: UDP, length 146
17:30:15.724600 00:23:4d:ae:f9:0a (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 254: vlan 160, p 0, ethertype IPv4, Miya-PC.non-stop.com.au.netbios-dgm > 192.168.160.255.netbios-dgm: NBT UDP PACKET(138)
17:30:15.855039 70:1a:0401:78 (oui Unknown) > 33:33:ff:60:91:a8 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5 > ff02::1:ff60:91a8: ICMP6, neighbor solicitation, who has fe80::d452:5831:f560:91a8, length 32
17:30:16.855254 70:1a:0401:78 (oui Unknown) > 33:33:ff:60:91:a8 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5 > ff02::1:ff60:91a8: ICMP6, neighbor solicitation, who has fe80::d452:5831:f560:91a8, length 32
17:30:17.954656 70:1a:0401:78 (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5.65113 > ff02::c.1900: UDP, length 146
17:30:18.184423 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8.55059 > ff02::c.1900: UDP, length 146
17:30:20.751020 00:17:08:50:16:70 (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 359: vlan 160, p 0, ethertype IPv4, vlan160.non-stop.com.au.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:17:08:50:16:70 (oui Unknown), length 313
17:30:20.955044 70:1a:0401:78 (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5.65113 > ff02::c.1900: UDP, length 146
17:30:22.185326 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8.55059 > ff02::c.1900: UDP, length 146
17:30:22.187515 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:ff:80:cd:b5 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8 > ff02::1:ff80:cdb5: ICMP6, neighbor solicitation, who has fe80::51f7:693c:2380:cdb5, length 32
17:30:22.733767 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:ff:80:cd:b5 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8 > ff02::1:ff80:cdb5: ICMP6, neighbor solicitation, who has fe80::51f7:693c:2380:cdb5, length 32
17:30:23.733979 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:ff:80:cd:b5 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8 > ff02::1:ff80:cdb5: ICMP6, neighbor solicitation, who has fe80::51f7:693c:2380:cdb5, length 32
17:30:23.747248 00:17:08:50:16:70 (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 359: vlan 160, p 0, ethertype IPv4, vlan160.non-stop.com.au.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:17:08:50:16:70 (oui Unknown), length 313
17:30:24.955630 70:1a:0401:78 (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5.65113 > ff02::c.1900: UDP, length 146
17:30:24.957029 70:1a:0401:78 (oui Unknown) > 33:33:ff:60:91:a8 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5 > ff02::1:ff60:91a8: ICMP6, neighbor solicitation, who has fe80::d452:5831:f560:91a8, length 32
17:30:25.185255 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8.55059 > ff02::c.1900: UDP, length 146
17:30:26.355313 70:1a:0401:78 (oui Unknown) > 33:33:ff:60:91:a8 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5 > ff02::1:ff60:91a8: ICMP6, neighbor solicitation, who has fe80::d452:5831:f560:91a8, length 32
17:30:27.355360 70:1a:0401:78 (oui Unknown) > 33:33:ff:60:91:a8 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5 > ff02::1:ff60:91a8: ICMP6, neighbor solicitation, who has fe80::d452:5831:f560:91a8, length 32
17:30:27.955850 70:1a:0401:78 (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5.65113 > ff02::c.1900: UDP, length 146
17:30:28.184886 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8.55059 > ff02::c.1900: UDP, length 146
^C
22 packets captured
2700 packets received by filter
0 packets dropped by kernel17:34:18.260155 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8.55059 > ff02::c.1900: UDP, length 146
17:34:18.759256 00:17:08:50:16:70 (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 359: vlan 160, p 0, ethertype IPv4, vlan160.non-stop.com.au.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:17:08:50:16:70 (oui Unknown), length 313
17:34:21.220921 70:1a:0401:78 (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::51f7:693c:2380:cdb5.65113 > ff02::c.1900: UDP, length 146
17:34:22.261472 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:00:00:00:0c (oui Unknown), ethertype 802.1Q (0x8100), length 212: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8.55059 > ff02::c.1900: UDP, length 146
17:34:22.263368 00:23:4d:ae:f9:0a (oui Unknown) > 33:33:ff:80:cd:b5 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, fe80::d452:5831:f560:91a8 > ff02::1:ff80:cdb5: ICMP6, neighbor solicitation, who has fe80::51f7:693c:2380:cdb5, length 3217:17:12.060268 00:0d:60:f3:15:cc (oui Unknown) > 00:0d:65:c8:78:7f (oui Unknown), ethertype 802.1Q (0x8100), length 58: vlan 120, p 0, ethertype IPv4, (tos 0x0, ttl 51, id 22879, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->3e1a)!)
mail.ycwua.org.34491 > 192.168.5.77.36912: Flags [.], cksum 0xa326 (correct), ack 95, win 92, length 0
17:17:12.060744 00:0d:65:c8:78:7f (oui Unknown) > 00:0d:60:f3:15:cc (oui Unknown), ethertype 802.1Q (0x8100), length 134: vlan 120, p 0, ethertype IPv4, (tos 0x0, ttl 63, id 4198, offset 0, flags [DF], proto TCP (6), length 116)
192.168.5.77.36912 > mail.ycwua.org.34491: Flags [P.], ack 520, win 64207, length 76
17:17:12.065563 70:1a:0401:78 (oui Unknown) > 33:33:00:00:00:16 (oui Unknown), ethertype 802.1Q (0x8100), length 94: vlan 160, p 0, ethertype IPv6, (hlim 1, next-header Options (0) payload length: 36) fe80::51f7:693c:2380:cdb5 > ff02::16: HBH (rtalert: 0x0000) (padn)[icmp6 sum ok] ICMP6, multicast listener report v2, length 28, 1 group record(s) [gaddr ff02::1:3 to_in, 0 source(s)]
17:17:12.067744 00:0d:60:f3:15:cc (oui Unknown) > 00:0d:65:c8:78:7f (oui Unknown), ethertype 802.1Q (0x8100), length 58: vlan 120, p 0, ethertype IPv4, (tos 0x0, ttl 43, id 24146, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->13bf)!)
ool-ad036ac6.dyn.optonline.net.51006 > 192.168.5.77.23206: Flags [.], cksum 0x33a7 (correct), ack 1235279, win 65535, length 0
.
.
.
17:17:15.827760 70:1a:0401:78 (oui Unknown) > 33:33:ff:60:91:a8 (oui Unknown), ethertype 802.1Q (0x8100), length 90: vlan 160, p 0, ethertype IPv6, (hlim 255, next-header ICMPv6 (58) payload length: 32) fe80::51f7:693c:2380:cdb5 > ff02::1:ff60:91a8: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has fe80::d452:5831:f560:91a8
source link-address option (1), length 8 (1): 70:1a:0401:78
17:17:15.832500 00:0d:60:f3:15:cc (oui Unknown) > 00:0d:65:c8:78:7f (oui Unknown), ethertype 802.1Q (0x8100), length 66: vlan 120, p 0, ethertype IPv4, (tos 0x0, ttl 115, id 1958, offset 0, flags [none], proto UDP (17), length 48, bad cksum 0 (->e28e)!)
111.Red-83-36-68.dynamicIP.rima-tde.net.52592 > 192.168.5.77.36912: UDP, length 20 -
You are sure that you have allowed traffic with firewall ruleS?
-
Thanks for your reply, I have firewall rules for both interfaces, pass all for each relevant network.
update
I switched the interface from the lagg trunk to a standard switchport access interface and all works as expected.
The only other difference on the new interface is a intel driver (dc).
So that is, No Tagging, No LACP, it works.
Will introduce the do1q next and see how it goes.
Many thanks,
mark -
Well, that was a interesting ride, changed opt1 back to the lagg_vlan160 and it all worked.
I did, however learn heaps so its not a waste of time, aye :)