New Fiber install, fresh Pfsense install, only getting 20Mbps up/down

stephenw10

@jddoxtator said in New Fiber install, fresh Pfsense install, only getting 20Mbps up/down:

Cisco CBS220-24FP-4X

That switch does port mirroring. Remove the VLANs from the switch. It needs to pass the tagged traffic from the ISP router so put two ports in port-vlan mode or whatever Cisco has renamed that.

Then mirror one of those ports and capture on it.

Steve

jddoxtator

@stephenw10 Ok, I deleted all VLAN's and my multicast's. Only problem is I'm not sure where port mirroring is in this switch. I'm guessing that this is more on the physical level under Port configuration. The only thing I see there that involves multiple ports is Link Aggregation. Could this be what I am looking for?

Edit: Found the manual online. I was looking in the wrong spot apparently. They put SPAN which is their port mirroring under the statistics tab.... strange choice but OK. Now the SPAN interface says I require a VLAN to define the mirror, so I'm guessing this is where I go to VLAN and select the two ports to have in the same VLAN group.

stephenw10

Nope not LAG. Looks like Cisco are using some combination of the terms mirroring, port monitoring and span port.

https://www.cisco.com/c/en/us/td/docs/switches/lan/csbss/CBS220/Adminstration-Guide/cbs-220-admin-guide/status-and-statistics.html?bookSearch=true#Cisco_Concept.dita_86e4dbba-7744-408d-b5e2-c55428a982b6
or
https://www.cisco.com/c/en/us/td/docs/switches/lan/csbss/CBS220/CLI-Guide/b_220CLI/port_monitor_commands.html

Steve

jddoxtator

@stephenw10 Think I almost have this figured out.

I have to list the two data ports as source then the listening port as destination all under the same session ID. Then they have to be in the same VLAN group and I think that should work. I hope, lets see.

jddoxtator

Ok, so this setup gives me a bunch of local network ARP requests

1	0.000000	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.121
2	0.106019	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
3	0.320230	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.121
4	0.609572	ASUSTekC_8c:16:e1	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.116
5	0.814689	ASUSTekC_f5:1f:a0	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.154
6	1.013517	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.121
7	1.105998	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
8	1.330691	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.121
9	1.622972	ASUSTekC_8c:16:e1	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.116
10	1.828057	ASUSTekC_f5:1f:a0	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.154
11	2.346824	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.121
12	2.636239	ASUSTekC_8c:16:e1	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.116
13	3.357319	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.121
14	3.649568	ASUSTekC_8c:16:e1	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.116
15	4.106170	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
16	4.370852	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.121
17	4.874859	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.121
18	5.105890	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
19	5.384213	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.121
20	5.809578	ASUSTekC_8c:16:e1	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.116
21	5.881565	ASUSTekC_f5:1f:a0	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.154
22	5.893358	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.121
23	6.105790	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
24	6.397349	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.121
25	6.822939	ASUSTekC_8c:16:e1	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.116
26	6.876218	ASUSTekC_8c:16:e1	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.116
27	6.894748	ASUSTekC_f5:1f:a0	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.154
28	6.904138	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.121
29	7.411112	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.121
30	7.836240	ASUSTekC_8c:16:e1	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.116
31	7.889569	ASUSTekC_8c:16:e1	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.116
32	7.908081	ASUSTekC_f5:1f:a0	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.154
33	8.423936	RivetNet_c8:5f:5d	Broadcast	ARP	60	Who has 192.168.1.134? Tell 192.168.1.121
34	8.453629	Calix_0c:ae:2c	Broadcast	ARP	60	Who has 172.31.16.1? Tell 172.31.17.249
35	8.453633	Calix_0c:ae:2c	Broadcast	ARP	60	Who has 172.31.16.1? Tell 172.31.17.249

ISP router still would not connect through this method. I think the VLAN isolation is not working as I am getting all my network devices.

stephenw10

I've never tried that on a Cisco switch but....
It seems like you just need to set a session destination and choose a session ID and a local port.
Then set session source using the same session ID and set it to Rx and Tx.
Then as long as the ISP router traffic is passing the session source port you should see it on the destination port.

jddoxtator

@stephenw10 My mistake, I had some other ports still trunked into the VLAN. I have it isolated now and got much more useful information

1	0.000000	0.0.0.0	255.255.255.255	DHCP	342	DHCP Discover - Transaction ID 0x163b8778
2	3.070015	0.0.0.0	255.255.255.255	DHCP	342	DHCP Discover - Transaction ID 0x163b8778
3	6.130010	0.0.0.0	255.255.255.255	DHCP	342	DHCP Discover - Transaction ID 0x163b8778
4	9.200069	0.0.0.0	255.255.255.255	DHCP	342	DHCP Discover - Transaction ID 0x163b8778
5	12.260068	0.0.0.0	255.255.255.255	DHCP	342	DHCP Discover - Transaction ID 0x163b8778
6	15.560215	0.0.0.0	255.255.255.255	DHCP	342	DHCP Discover - Transaction ID 0x163b8778
7	15.560238	0.0.0.0	255.255.255.255	DHCP	342	DHCP Discover - Transaction ID 0x163b8778
8	15.584241	172.31.16.1	172.31.17.42	DHCP	398	DHCP Offer    - Transaction ID 0x163b8778
9	15.584268	172.31.16.1	172.31.17.42	DHCP	398	DHCP Offer    - Transaction ID 0x163b8778
10	15.584554	172.31.16.1	172.31.17.42	DHCP	398	DHCP Offer    - Transaction ID 0x163b8778
11	15.584578	172.31.16.1	172.31.17.42	DHCP	398	DHCP Offer    - Transaction ID 0x163b8778
12	15.630099	0.0.0.0	255.255.255.255	DHCP	342	DHCP Request  - Transaction ID 0x163b8778
13	15.630121	0.0.0.0	255.255.255.255	DHCP	342	DHCP Request  - Transaction ID 0x163b8778
14	15.633695	172.31.16.1	172.31.17.42	DHCP	398	DHCP ACK      - Transaction ID 0x163b8778
15	15.633706	172.31.16.1	172.31.17.42	DHCP	398	DHCP ACK      - Transaction ID 0x163b8778
16	15.634197	172.31.16.1	172.31.17.42	DHCP	398	DHCP ACK      - Transaction ID 0x163b8778
17	15.634208	172.31.16.1	172.31.17.42	DHCP	398	DHCP ACK      - Transaction ID 0x163b8778
18	15.951848	Calix_1c:4f:67	Broadcast	ARP	60	Who has 172.31.16.1? Tell 172.31.19.199
19	15.951853	Calix_1c:4f:67	Broadcast	ARP	60	Who has 172.31.16.1? Tell 172.31.19.199
20	18.610414	Calix_6b:e8:f7	Broadcast	ARP	60	Who has 172.31.16.1? Tell 172.31.17.42
21	18.610417	Calix_6b:e8:f7	Broadcast	ARP	60	Who has 172.31.16.1? Tell 172.31.17.42
22	18.613395	Cisco_f2:da:7f	Calix_6b:e8:f7	ARP	60	172.31.16.1 is at 7c:69:f6:f2:da:7f
23	18.613400	Cisco_f2:da:7f	Calix_6b:e8:f7	ARP	60	172.31.16.1 is at 7c:69:f6:f2:da:7f
24	18.618696	64.235.98.226	172.31.17.42	DNS	93	Standard query response 0x5f81 A stun-ca.calix.com A 99.79.144.131
25	18.618739	64.235.98.226	172.31.17.42	DNS	93	Standard query response 0x5f81 A stun-ca.calix.com A 99.79.144.131
26	18.635840	8.8.8.8	172.31.17.42	DNS	93	Standard query response 0x5f81 A stun-ca.calix.com A 99.79.144.131
27	18.635870	8.8.8.8	172.31.17.42	DNS	93	Standard query response 0x5f81 A stun-ca.calix.com A 99.79.144.131
28	18.645352	99.79.144.131	172.31.17.42	CLASSIC-STUN	86	Message: Binding Response
29	18.645384	99.79.144.131	172.31.17.42	CLASSIC-STUN	86	Message: Binding Response
30	21.478185	Calix_07:31:f7	Broadcast	ARP	60	Who has 172.31.16.1? Tell 172.31.17.28
31	21.478190	Calix_07:31:f7	Broadcast	ARP	60	Who has 172.31.16.1? Tell 172.31.17.28
32	28.670541	64.235.98.226	172.31.17.42	DNS	141	Standard query response 0x8b10 A 0.ca.pool.ntp.org A 198.27.76.102 A 162.159.200.1 A 205.206.70.40 A 217.180.209.214
33	28.670575	64.235.98.226	172.31.17.42	DNS	141	Standard query response 0x8b10 A 0.ca.pool.ntp.org A 198.27.76.102 A 162.159.200.1 A 205.206.70.40 A 217.180.209.214
34	28.670688	64.235.98.226	172.31.17.42	DNS	132	Standard query response 0xbbba AAAA 0.ca.pool.ntp.org SOA b.ntpns.org
35	28.670717	64.235.98.226	172.31.17.42	DNS	132	Standard query response 0xbbba AAAA 0.ca.pool.ntp.org SOA b.ntpns.org
36	28.677733	64.235.98.226	172.31.17.42	DNS	141	Standard query response 0xe1d8 A 1.ca.pool.ntp.org A 205.206.70.42 A 216.232.132.102 A 149.56.37.32 A 207.210.46.249
37	28.677747	64.235.98.226	172.31.17.42	DNS	141	Standard query response 0xe1d8 A 1.ca.pool.ntp.org A 205.206.70.42 A 216.232.132.102 A 149.56.37.32 A 207.210.46.249
38	28.677932	64.235.98.226	172.31.17.42	DNS	132	Standard query response 0xe409 AAAA 1.ca.pool.ntp.org SOA b.ntpns.org
39	28.677946	64.235.98.226	172.31.17.42	DNS	132	Standard query response 0xe409 AAAA 1.ca.pool.ntp.org SOA b.ntpns.org
40	28.684459	64.235.98.226	172.31.17.42	DNS	152	Standard query response 0x3d28 A 0.north-america.pool.ntp.org A 65.108.76.171 A 206.108.0.132 A 216.6.2.70 A 198.199.14.18
41	28.684489	64.235.98.226	172.31.17.42	DNS	152	Standard query response 0x3d28 A 0.north-america.pool.ntp.org A 65.108.76.171 A 206.108.0.132 A 216.6.2.70 A 198.199.14.18
42	28.684607	64.235.98.226	172.31.17.42	DNS	143	Standard query response 0x5359 AAAA 0.north-america.pool.ntp.org SOA b.ntpns.org
43	28.684648	64.235.98.226	172.31.17.42	DNS	143	Standard query response 0x5359 AAAA 0.north-america.pool.ntp.org SOA b.ntpns.org
44	28.691310	64.235.98.226	172.31.17.42	DNS	141	Standard query response 0x7977 A 0.us.pool.ntp.org A 50.205.244.24 A 162.159.200.1 A 162.159.200.123 A 162.248.241.94
45	28.691340	64.235.98.226	172.31.17.42	DNS	141	Standard query response 0x7977 A 0.us.pool.ntp.org A 50.205.244.24 A 162.159.200.1 A 162.159.200.123 A 162.248.241.94
46	28.691468	64.235.98.226	172.31.17.42	DNS	132	Standard query response 0x6d87 AAAA 0.us.pool.ntp.org SOA b.ntpns.org
47	28.691498	64.235.98.226	172.31.17.42	DNS	132	Standard query response 0x6d87 AAAA 0.us.pool.ntp.org SOA b.ntpns.org
48	28.693444	8.8.8.8	172.31.17.42	DNS	141	Standard query response 0x8b10 A 0.ca.pool.ntp.org A 199.182.221.110 A 205.206.70.42 A 192.95.27.155 A 194.0.5.123
49	28.693478	8.8.8.8	172.31.17.42	DNS	141	Standard query response 0x8b10 A 0.ca.pool.ntp.org A 199.182.221.110 A 205.206.70.42 A 192.95.27.155 A 194.0.5.123
50	28.693535	8.8.8.8	172.31.17.42	DNS	132	Standard query response 0xbbba AAAA 0.ca.pool.ntp.org SOA c.ntpns.org
51	28.693550	8.8.8.8	172.31.17.42	DNS	132	Standard query response 0xbbba AAAA 0.ca.pool.ntp.org SOA c.ntpns.org
52	28.700935	8.8.8.8	172.31.17.42	DNS	141	Standard query response 0xe1d8 A 1.ca.pool.ntp.org A 216.197.156.83 A 162.159.200.123 A 217.180.209.214 A 208.81.1.244
53	28.700967	8.8.8.8	172.31.17.42	DNS	141	Standard query response 0xe1d8 A 1.ca.pool.ntp.org A 216.197.156.83 A 162.159.200.123 A 217.180.209.214 A 208.81.1.244
54	28.701152	8.8.8.8	172.31.17.42	DNS	132	Standard query response 0xe409 AAAA 1.ca.pool.ntp.org SOA i.ntpns.org
55	28.701182	8.8.8.8	172.31.17.42	DNS	132	Standard query response 0xe409 AAAA 1.ca.pool.ntp.org SOA i.ntpns.org
56	28.701736	8.8.8.8	172.31.17.42	DNS	143	Standard query response 0x5359 AAAA 0.north-america.pool.ntp.org SOA e.ntpns.org
57	28.701767	8.8.8.8	172.31.17.42	DNS	143	Standard query response 0x5359 AAAA 0.north-america.pool.ntp.org SOA e.ntpns.org
58	28.708145	8.8.8.8	172.31.17.42	DNS	141	Standard query response 0x7977 A 0.us.pool.ntp.org A 64.251.10.152 A 50.205.244.113 A 204.93.207.12 A 69.164.213.136
59	28.708147	8.8.8.8	172.31.17.42	DNS	152	Standard query response 0x3d28 A 0.north-america.pool.ntp.org A 38.229.62.9 A 66.220.9.122 A 144.172.118.20 A 129.250.35.251
60	28.708160	8.8.8.8	172.31.17.42	DNS	141	Standard query response 0x7977 A 0.us.pool.ntp.org A 64.251.10.152 A 50.205.244.113 A 204.93.207.12 A 69.164.213.136
61	28.708165	8.8.8.8	172.31.17.42	DNS	152	Standard query response 0x3d28 A 0.north-america.pool.ntp.org A 38.229.62.9 A 66.220.9.122 A 144.172.118.20 A 129.250.35.251
62	28.714399	8.8.8.8	172.31.17.42	DNS	132	Standard query response 0x6d87 AAAA 0.us.pool.ntp.org SOA f.ntpns.org
63	28.714429	8.8.8.8	172.31.17.42	DNS	132	Standard query response 0x6d87 AAAA 0.us.pool.ntp.org SOA f.ntpns.org
64	37.872289	64.235.98.226	172.31.17.42	DNS	93	Standard query response 0xa7dc A gcs6-ca.calix.com A 52.60.181.28
65	37.872307	64.235.98.226	172.31.17.42	DNS	93	Standard query response 0xa7dc A gcs6-ca.calix.com A 52.60.181.28
66	37.896074	8.8.8.8	172.31.17.42	DNS	93	Standard query response 0xa7dc A gcs6-ca.calix.com A 52.60.181.28
67	37.896104	8.8.8.8	172.31.17.42	DNS	93	Standard query response 0xa7dc A gcs6-ca.calix.com A 52.60.181.28
68	38.030652	52.60.181.28	172.31.17.42	TCP	74	8443 → 39182 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=236259536 TSecr=5140577 WS=128
69	38.030679	52.60.181.28	172.31.17.42	TCP	74	[TCP Out-Of-Order] 8443 → 39182 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=236259536 TSecr=5140577 WS=128
70	38.056085	52.60.181.28	172.31.17.42	TLSv1.2	203	Server Hello, Change Cipher Spec, Encrypted Handshake Message
71	38.056112	52.60.181.28	172.31.17.42	TCP	203	[TCP Retransmission] 8443 → 39182 [PSH, ACK] Seq=1 Ack=518 Win=62208 Len=137 TSval=236259561 TSecr=5140580
72	38.083558	52.60.181.28	172.31.17.42	TCP	66	8443 → 39182 [ACK] Seq=138 Ack=822 Win=61952 Len=0 TSval=236259589 TSecr=5140582
73	38.083584	52.60.181.28	172.31.17.42	TCP	66	[TCP Dup ACK 72#1] 8443 → 39182 [ACK] Seq=138 Ack=822 Win=61952 Len=0 TSval=236259589 TSecr=5140582
74	38.086084	52.60.181.28	172.31.17.42	TCP	66	8443 → 39182 [ACK] Seq=138 Ack=2270 Win=60544 Len=0 TSval=236259591 TSecr=5140583
75	38.086096	52.60.181.28	172.31.17.42	TCP	66	[TCP Dup ACK 74#1] 8443 → 39182 [ACK] Seq=138 Ack=2270 Win=60544 Len=0 TSval=236259591 TSecr=5140583
76	38.086141	52.60.181.28	172.31.17.42	TCP	66	8443 → 39182 [ACK] Seq=138 Ack=3718 Win=59136 Len=0 TSval=236259591 TSecr=5140583
77	38.086143	52.60.181.28	172.31.17.42	TCP	66	[TCP Dup ACK 76#1] 8443 → 39182 [ACK] Seq=138 Ack=3718 Win=59136 Len=0 TSval=236259591 TSecr=5140583
78	38.086540	52.60.181.28	172.31.17.42	TCP	66	8443 → 39182 [ACK] Seq=138 Ack=4326 Win=58624 Len=0 TSval=236259592 TSecr=5140583
79	38.086543	52.60.181.28	172.31.17.42	TCP	66	[TCP Dup ACK 78#1] 8443 → 39182 [ACK] Seq=138 Ack=4326 Win=58624 Len=0 TSval=236259592 TSecr=5140583
80	38.087740	52.60.181.28	172.31.17.42	TLSv1.2	340	Application Data
81	38.087767	52.60.181.28	172.31.17.42	TCP	340	[TCP Retransmission] 8443 → 39182 [PSH, ACK] Seq=138 Ack=4326 Win=58624 Len=274 TSval=236259593 TSecr=5140583
82	38.120573	52.60.181.28	172.31.17.42	TCP	66	8443 → 39182 [ACK] Seq=412 Ack=4357 Win=58624 Len=0 TSval=236259626 TSecr=5140586
83	38.120599	52.60.181.28	172.31.17.42	TCP	66	[TCP Dup ACK 82#1] 8443 → 39182 [ACK] Seq=412 Ack=4357 Win=58624 Len=0 TSval=236259626 TSecr=5140586
84	38.120613	52.60.181.28	172.31.17.42	TLSv1.2	97	Encrypted Alert
85	38.120620	52.60.181.28	172.31.17.42	TCP	97	[TCP Retransmission] 8443 → 39182 [PSH, ACK] Seq=412 Ack=4357 Win=58624 Len=31 TSval=236259626 TSecr=5140586
86	38.120625	52.60.181.28	172.31.17.42	TCP	66	8443 → 39182 [FIN, ACK] Seq=443 Ack=4357 Win=58624 Len=0 TSval=236259626 TSecr=5140586
87	38.120629	52.60.181.28	172.31.17.42	TCP	66	[TCP Out-Of-Order] 8443 → 39182 [FIN, ACK] Seq=443 Ack=4357 Win=58624 Len=0 TSval=236259626 TSecr=5140586
88	38.121583	52.60.181.28	172.31.17.42	TCP	66	8443 → 39182 [ACK] Seq=444 Ack=4358 Win=58624 Len=0 TSval=236259627 TSecr=5140586
89	38.121595	52.60.181.28	172.31.17.42	TCP	66	[TCP Dup ACK 88#1] 8443 → 39182 [ACK] Seq=444 Ack=4358 Win=58624 Len=0 TSval=236259627 TSecr=5140586
90	38.146682	52.60.181.28	172.31.17.42	TCP	74	8443 → 33372 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=236259652 TSecr=5140589 WS=128
91	38.146708	52.60.181.28	172.31.17.42	TCP	74	[TCP Out-Of-Order] 8443 → 33372 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=236259652 TSecr=5140589 WS=128
92	38.172161	52.60.181.28	172.31.17.42	TLSv1.2	203	Server Hello, Change Cipher Spec, Encrypted Handshake Message
93	38.172188	52.60.181.28	172.31.17.42	TCP	203	[TCP Retransmission] 8443 → 33372 [PSH, ACK] Seq=1 Ack=518 Win=62208 Len=137 TSval=236259678 TSecr=5140591
94	38.199727	52.60.181.28	172.31.17.42	TCP	66	8443 → 33372 [ACK] Seq=138 Ack=1101 Win=61696 Len=0 TSval=236259705 TSecr=5140594
95	38.199753	52.60.181.28	172.31.17.42	TCP	66	[TCP Dup ACK 94#1] 8443 → 33372 [ACK] Seq=138 Ack=1101 Win=61696 Len=0 TSval=236259705 TSecr=5140594
96	38.202039	52.60.181.28	172.31.17.42	TCP	66	8443 → 33372 [ACK] Seq=138 Ack=2549 Win=60288 Len=0 TSval=236259707 TSecr=5140594
97	38.202065	52.60.181.28	172.31.17.42	TCP	66	[TCP Dup ACK 96#1] 8443 → 33372 [ACK] Seq=138 Ack=2549 Win=60288 Len=0 TSval=236259707 TSecr=5140594
98	38.202077	52.60.181.28	172.31.17.42	TCP	66	8443 → 33372 [ACK] Seq=138 Ack=3997 Win=58880 Len=0 TSval=236259707 TSecr=5140594
99	38.202084	52.60.181.28	172.31.17.42	TCP	66	[TCP Dup ACK 98#1] 8443 → 33372 [ACK] Seq=138 Ack=3997 Win=58880 Len=0 TSval=236259707 TSecr=5140594
100	38.202619	52.60.181.28	172.31.17.42	TCP	66	8443 → 33372 [ACK] Seq=138 Ack=4560 Win=58368 Len=0 TSval=236259708 TSecr=5140594

the ISP router still didn't connect, but at least we have relevant info about it.

stephenw10

If you have the source port in an 802.1q VLAN though it will not pass anything but that VLAN so whatever VLAN tagging the ISP router may or may not be using would get dropped and not appear there.
The ports the ISP router traffic is using need to pass all tagged traffic.

jddoxtator

@stephenw10 ok, so maybe RSPAN VLAN is not required for this then. Let me try without it.

jddoxtator

Alright same procedure without RSPAN VLAN.

34	21.469965	0.0.0.0	255.255.255.255	DHCP	342	DHCP Discover - Transaction ID 0x6b391354
35	21.469991	0.0.0.0	255.255.255.255	DHCP	342	DHCP Discover - Transaction ID 0x6b391354
36	21.477901	172.31.16.1	172.31.17.42	DHCP	398	DHCP Offer    - Transaction ID 0x6b391354
37	21.477929	172.31.16.1	172.31.17.42	DHCP	398	DHCP Offer    - Transaction ID 0x6b391354
38	21.610081	0.0.0.0	255.255.255.255	DHCP	342	DHCP Request  - Transaction ID 0x6b391354
39	21.610104	0.0.0.0	255.255.255.255	DHCP	342	DHCP Request  - Transaction ID 0x6b391354
40	21.619714	172.31.16.1	172.31.17.42	DHCP	398	DHCP ACK      - Transaction ID 0x6b391354
41	21.619814	172.31.16.1	172.31.17.42	DHCP	398	DHCP ACK      - Transaction ID 0x6b391354
42	23.471686	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
43	23.471688	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
44	24.470907	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
45	24.470913	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
46	25.470906	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
47	25.470911	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
48	28.471041	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
49	28.471046	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
50	29.306140	Calix_6b:e8:f7	Broadcast	ARP	60	Who has 172.31.16.1? Tell 172.31.17.42
51	29.306145	Calix_6b:e8:f7	Broadcast	ARP	60	Who has 172.31.16.1? Tell 172.31.17.42
52	29.306893	Cisco_f2:da:7f	Calix_6b:e8:f7	ARP	60	172.31.16.1 is at 7c:69:f6:f2:da:7f
53	29.312275	64.235.98.226	172.31.17.42	DNS	93	Standard query response 0x0d4a A gcs6-ca.calix.com A 52.60.181.28
54	29.336235	8.8.8.8	172.31.17.42	DNS	93	Standard query response 0x0d4a A gcs6-ca.calix.com A 52.60.181.28
55	29.338204	52.60.181.28	172.31.17.42	TCP	74	8443 → 40880 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=237229642 TSecr=5237588 WS=128
56	29.363891	52.60.181.28	172.31.17.42	TLSv1.2	203	Server Hello, Change Cipher Spec, Encrypted Handshake Message
57	29.391025	52.60.181.28	172.31.17.42	TCP	66	8443 → 40880 [ACK] Seq=138 Ack=822 Win=61952 Len=0 TSval=237229695 TSecr=5237593
58	29.393327	52.60.181.28	172.31.17.42	TCP	66	8443 → 40880 [ACK] Seq=138 Ack=2270 Win=60544 Len=0 TSval=237229697 TSecr=5237593
59	29.393353	52.60.181.28	172.31.17.42	TCP	66	8443 → 40880 [ACK] Seq=138 Ack=3718 Win=59136 Len=0 TSval=237229697 TSecr=5237593
60	29.393884	52.60.181.28	172.31.17.42	TCP	66	8443 → 40880 [ACK] Seq=138 Ack=4326 Win=58624 Len=0 TSval=237229698 TSecr=5237593
61	29.394462	52.60.181.28	172.31.17.42	TLSv1.2	340	Application Data
62	29.426693	52.60.181.28	172.31.17.42	TCP	66	8443 → 40880 [ACK] Seq=412 Ack=4357 Win=58624 Len=0 TSval=237229730 TSecr=5237597
63	29.426706	52.60.181.28	172.31.17.42	TLSv1.2	97	Encrypted Alert
64	29.426752	52.60.181.28	172.31.17.42	TCP	66	8443 → 40880 [FIN, ACK] Seq=443 Ack=4357 Win=58624 Len=0 TSval=237229730 TSecr=5237597
65	29.426923	52.60.181.28	172.31.17.42	TCP	66	8443 → 40880 [ACK] Seq=444 Ack=4358 Win=58624 Len=0 TSval=237229731 TSecr=5237597
66	29.452715	52.60.181.28	172.31.17.42	TCP	74	8443 → 36033 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=237229756 TSecr=5237599 WS=128
67	29.470799	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
68	29.470804	Cisco_f4:83:3a	Broadcast	ARP	60	Who has 192.168.1.1? Tell 192.168.1.155
69	29.478182	52.60.181.28	172.31.17.42	TLSv1.2	203	Server Hello, Change Cipher Spec, Encrypted Handshake Message
70	29.505481	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=138 Ack=1101 Win=61696 Len=0 TSval=237229809 TSecr=5237604
71	29.507689	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=138 Ack=2549 Win=60288 Len=0 TSval=237229811 TSecr=5237605
72	29.507715	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=138 Ack=3997 Win=58880 Len=0 TSval=237229811 TSecr=5237605
73	29.507948	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=138 Ack=4560 Win=58368 Len=0 TSval=237229812 TSecr=5237605
74	29.511811	52.60.181.28	172.31.17.42	TLSv1.2	571	Application Data
75	29.576321	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=643 Ack=5156 Win=57856 Len=0 TSval=237229880 TSecr=5237612
76	29.577110	52.60.181.28	172.31.17.42	TLSv1.2	895	Application Data
77	29.610470	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=1472 Ack=5808 Win=57216 Len=0 TSval=237229914 TSecr=5237615
78	29.611045	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=1472 Ack=6921 Win=56576 Len=0 TSval=237229915 TSecr=5237615
79	29.613017	52.60.181.28	172.31.17.42	TLSv1.2	1021	Application Data
80	29.644515	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=2427 Ack=7572 Win=56576 Len=0 TSval=237229948 TSecr=5237618
81	29.644790	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=2427 Ack=8126 Win=56576 Len=0 TSval=237229949 TSecr=5237619
82	29.645791	52.60.181.28	172.31.17.42	TLSv1.2	771	Application Data
83	30.341325	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=8779 Win=56576 Len=0 TSval=237230645 TSecr=5237688
84	30.345607	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=10227 Win=56576 Len=0 TSval=237230649 TSecr=5237689
85	30.345633	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=11675 Win=56576 Len=0 TSval=237230649 TSecr=5237689
86	30.345647	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=14571 Win=56576 Len=0 TSval=237230649 TSecr=5237689
87	30.345653	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=17467 Win=53760 Len=0 TSval=237230649 TSecr=5237689
88	30.345914	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=18915 Win=56576 Len=0 TSval=237230650 TSecr=5237689
89	30.345926	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=21811 Win=56576 Len=0 TSval=237230650 TSecr=5237689
90	30.366516	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=24707 Win=56576 Len=0 TSval=237230670 TSecr=5237691
91	30.370821	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=26155 Win=56576 Len=0 TSval=237230675 TSecr=5237691
92	30.370833	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=30499 Win=56576 Len=0 TSval=237230675 TSecr=5237691
93	30.370877	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=33395 Win=56576 Len=0 TSval=237230675 TSecr=5237691
94	30.371050	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=34843 Win=56576 Len=0 TSval=237230675 TSecr=5237691
95	30.371064	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=36291 Win=56576 Len=0 TSval=237230675 TSecr=5237691
96	30.371882	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=37739 Win=56576 Len=0 TSval=237230676 TSecr=5237691
97	30.371894	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=39187 Win=56576 Len=0 TSval=237230676 TSecr=5237691
98	30.372797	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=3132 Ack=39527 Win=56576 Len=0 TSval=237230677 TSecr=5237691
99	30.376856	52.60.181.28	172.31.17.42	TLSv1.2	1370	Application Data
100	30.403991	52.60.181.28	172.31.17.42	TCP	66	8443 → 36033 [ACK] Seq=4436 Ack=40179 Win=56576 Len=0 TSval=237230708 TSecr=5237694

Different messages, results much the same as before. ISP router no connection.

stephenw10

Might need to test it with something else then. Make sure you can send tagged traffic between the ports and capture it.

You might need to use 'Dot1q-Tunnel' mode:
https://www.cisco.com/c/en/us/td/docs/switches/lan/csbss/CBS220/Adminstration-Guide/cbs-220-admin-guide/vlan-management.html#ID-0000320b

Since I can see nothing about port based VLANs there.

Steve

jddoxtator

@stephenw10 Ok, some mild progres. Enabeling Dot1q-Tunnel on the ports I was using allowed the ISP router to full connect. Problem is the only packets I am snooping on the Pfsense box are DNS, DHCP and ARP. Nothing with VLAN information.

Edit: Fanstatic! I can now eliminate the SFP converter and directly connect the fiber to the switch with the Dot1q-Tunnel trick.

Edit2: Routing at full speed on ISP router. We are making progress here. Now we just need to get Pfsense to do the same.

Edit3: yep still capped at 20Mb/s on the Pfsense box.

stephenw10

Ok, so you are still not seeing all the traffic to/from the ISP router?

But you can see it using DHCP to pull an address in the correct subnet from upstream?

That should show any special dhcp client options it's using.

Steve

keyser

@stephenw10 said in New Fiber install, fresh Pfsense install, only getting 20Mbps up/down:

Ok, so you are still not seeing all the traffic to/from the ISP router?

But you can see it using DHCP to pull an address in the correct subnet from upstream?

That should show any special dhcp client options it's using.

Steve

Perhaps not true. If he is only seeing dhcp and other broadcasts, he’s either not capturing in promiscious mode, or he’s only seeing the broadcasts in the native VLAN which might be used for ISP management and has nothing to do with the user/internet VLAN

keyser

@jddoxtator the major issue with using a managed switch is it will by default discard any VLAN tagged frames if that VLAN is not enabled in the switch. Thats why i suggested you used a dumb non managed switch. It makes it infinitely much easier to learn VLAN tags from as Long as you can see the broadcasts.

You are likely capturing data on a port that has stripped unknown VLANs or forgot to capture i promiscious mode.
If you are capturing on mvneta1 in the SG-2100, remember that is a uplink port to the internal 4 port switch. If dot1.q mode is enabled on that switch to create “discrete” interfaces, then the 2100 is the switch that is stripping unknown VLAN’s

jddoxtator

@keyser Yes, I forgot promiscuous mode. Good catch.

Recaptured and I am seeing the PVST+

Now the VLAN is type PVID, so I am wondering if that is the issue. I see option in the switch for PVID but not in Pfsense.

here is the full details:

Originating VLAN (PVID): 85
Type: Originating VLAN (0x0000)
Length: 2
Originating VLAN: 85

stephenw10

There is no PVID setting in pfSense (except those with built in switches) because that only applies to assigning VLAN tags to untagged traffic and that only happens in a switch.
pfSense either sends and receives tagged traffic on a VLAN interface or untagged traffic on a regular interface.

To be clear you are now seeing VLAN tagged traffic in your pcaps on the SPAN port?

jddoxtator

@stephenw10 Yes, however I have tried the VLAN 85 before and Pfsense just fails to even connect.

There is a possibility I am not implementing the VLAN correctly in Pfsense.

What i am doing is creating the VLAN on the bare port ID then assigning that VLAN to WAN instead of having it the bare port ID.

stephenw10

Which is fine if only VLAN85 is required. Looks like something more is so maybe priority tags. And/or custom dhclient values. Something else...

jddoxtator

Well this is interesting.... ISP router has completely locked my out of the WebGUI but passes the internet. Tried reboots hooking back up to converter, might have to reset the damned thing.