New Fiber install, fresh Pfsense install, only getting 20Mbps up/down
-
That's after setting VLAN1? It looks like dhcp requests from pfSense tagged as that.
You might try switching the ISP router in and back out before the pcap to try to get some tagged traffic from the ISP as you did before with the ARP packet.
Ultimately the only way to know for sure is to setup a switch with a mirror port so you can capture exactly what the ISP router is doing.
The other thing is that you are almost certainly not the first person trying this. Someone else may have documented what's required for that ISP. Somewhere.
Steve
-
@jddoxtator To make this easy on yourself, try and connect a switch between the media converter and the original ISP router.
Connect your pfSense to one switchport and set its WAN port to either no IP address, or a fixed random private IP address.
Start a packet capture on WAN, and connect the ISP router to the switch.When the ISP router is connected it will attempt to get a IP address via DHCP frames which is broadcasted - and include the VLAN tag the ISP router is using.
Those broadcasts should also reach your pfSense if the switch is a dumb non-managed Layer2 switch. If it is a smart VLAN capable/managed switch, this will not work, and you will have to setup a mirrorport/spanport on the switch which mirrors the ISP router port to your pfsense port. -
@stephenw10 Before setting VLAN 1
I set the pfsense router back to stock and switched the WAN port from the ISP router to the pfsense router after the ISP router had connected.
-
Update: My old switch died, would not resolve DHCP anymore for some reason. Long story short, got a Cisco CBS220-24FP-4X capable of VLAN's and specifically PVST+
Some interesting behavior after getting this switch installed and running.
I set up a VLAN across one of the SFP 10Gbe ports and one of the copper 1Gbe ports. I then connected the fiber line directly to the switch SFP port and routers to the copper port.
Pfsense picked up the WAN signal and did it's usual thing connecting at 20Mb/s. However, the ISP router would not connect at all with the switch routing the fiber to the copper port on the VLAN.
There is something fundamentally different in how these two routers are connecting and I have no idea what.
-
@jddoxtator I still suspect putting a managed switch with port mirroring on the WAN line of your ISP router would be the most efficient way of finding out what works.
-
@patch I tried multiple setups and most gave me nothing at all on packet capture.
I used multicast on 3 ports across a single VLAN and only once I captured the ISP router sending an ARP request for the same gateway that Pfsense uses, but it could not connect.
Every other time there was no traffic to record.
This was all the packets I got:
1 0.000000 Calix_7a:06:4a Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.83 2 1.917011 172.31.16.23 224.0.0.1 ICMP 60 Mobile IP Advertisement (Normal router advertisement) 3 10.450749 Calix_4c:f9:11 Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.19.81Just a reminder, when the ISP router is connected directly, it connects with Gateway 192.24.57.1 not 172.31.16.1
-
@jddoxtator said in New Fiber install, fresh Pfsense install, only getting 20Mbps up/down:
Cisco CBS220-24FP-4X
That switch does port mirroring. Remove the VLANs from the switch. It needs to pass the tagged traffic from the ISP router so put two ports in port-vlan mode or whatever Cisco has renamed that.
Then mirror one of those ports and capture on it.
Steve
-
@stephenw10 Ok, I deleted all VLAN's and my multicast's. Only problem is I'm not sure where port mirroring is in this switch. I'm guessing that this is more on the physical level under Port configuration. The only thing I see there that involves multiple ports is Link Aggregation. Could this be what I am looking for?
Edit: Found the manual online. I was looking in the wrong spot apparently. They put SPAN which is their port mirroring under the statistics tab.... strange choice but OK. Now the SPAN interface says I require a VLAN to define the mirror, so I'm guessing this is where I go to VLAN and select the two ports to have in the same VLAN group.
-
Nope not LAG. Looks like Cisco are using some combination of the terms mirroring, port monitoring and span port.
https://www.cisco.com/c/en/us/td/docs/switches/lan/csbss/CBS220/Adminstration-Guide/cbs-220-admin-guide/status-and-statistics.html?bookSearch=true#Cisco_Concept.dita_86e4dbba-7744-408d-b5e2-c55428a982b6
or
https://www.cisco.com/c/en/us/td/docs/switches/lan/csbss/CBS220/CLI-Guide/b_220CLI/port_monitor_commands.htmlSteve
-
@stephenw10 Think I almost have this figured out.
I have to list the two data ports as source then the listening port as destination all under the same session ID. Then they have to be in the same VLAN group and I think that should work. I hope, lets see.
-
Ok, so this setup gives me a bunch of local network ARP requests
1 0.000000 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.121 2 0.106019 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 3 0.320230 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 4 0.609572 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 5 0.814689 ASUSTekC_f5:1f:a0 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.154 6 1.013517 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.121 7 1.105998 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 8 1.330691 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 9 1.622972 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 10 1.828057 ASUSTekC_f5:1f:a0 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.154 11 2.346824 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 12 2.636239 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 13 3.357319 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 14 3.649568 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 15 4.106170 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 16 4.370852 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 17 4.874859 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.121 18 5.105890 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 19 5.384213 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 20 5.809578 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 21 5.881565 ASUSTekC_f5:1f:a0 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.154 22 5.893358 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.121 23 6.105790 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 24 6.397349 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 25 6.822939 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 26 6.876218 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.116 27 6.894748 ASUSTekC_f5:1f:a0 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.154 28 6.904138 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.121 29 7.411112 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 30 7.836240 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 31 7.889569 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.116 32 7.908081 ASUSTekC_f5:1f:a0 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.154 33 8.423936 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 34 8.453629 Calix_0c:ae:2c Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.249 35 8.453633 Calix_0c:ae:2c Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.249ISP router still would not connect through this method. I think the VLAN isolation is not working as I am getting all my network devices.
-
I've never tried that on a Cisco switch but....
It seems like you just need to set a session destination and choose a session ID and a local port.
Then set session source using the same session ID and set it to Rx and Tx.
Then as long as the ISP router traffic is passing the session source port you should see it on the destination port. -
@stephenw10 My mistake, I had some other ports still trunked into the VLAN. I have it isolated now and got much more useful information
1 0.000000 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x163b8778 2 3.070015 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x163b8778 3 6.130010 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x163b8778 4 9.200069 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x163b8778 5 12.260068 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x163b8778 6 15.560215 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x163b8778 7 15.560238 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x163b8778 8 15.584241 172.31.16.1 172.31.17.42 DHCP 398 DHCP Offer - Transaction ID 0x163b8778 9 15.584268 172.31.16.1 172.31.17.42 DHCP 398 DHCP Offer - Transaction ID 0x163b8778 10 15.584554 172.31.16.1 172.31.17.42 DHCP 398 DHCP Offer - Transaction ID 0x163b8778 11 15.584578 172.31.16.1 172.31.17.42 DHCP 398 DHCP Offer - Transaction ID 0x163b8778 12 15.630099 0.0.0.0 255.255.255.255 DHCP 342 DHCP Request - Transaction ID 0x163b8778 13 15.630121 0.0.0.0 255.255.255.255 DHCP 342 DHCP Request - Transaction ID 0x163b8778 14 15.633695 172.31.16.1 172.31.17.42 DHCP 398 DHCP ACK - Transaction ID 0x163b8778 15 15.633706 172.31.16.1 172.31.17.42 DHCP 398 DHCP ACK - Transaction ID 0x163b8778 16 15.634197 172.31.16.1 172.31.17.42 DHCP 398 DHCP ACK - Transaction ID 0x163b8778 17 15.634208 172.31.16.1 172.31.17.42 DHCP 398 DHCP ACK - Transaction ID 0x163b8778 18 15.951848 Calix_1c:4f:67 Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.19.199 19 15.951853 Calix_1c:4f:67 Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.19.199 20 18.610414 Calix_6b:e8:f7 Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.42 21 18.610417 Calix_6b:e8:f7 Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.42 22 18.613395 Cisco_f2:da:7f Calix_6b:e8:f7 ARP 60 172.31.16.1 is at 7c:69:f6:f2:da:7f 23 18.613400 Cisco_f2:da:7f Calix_6b:e8:f7 ARP 60 172.31.16.1 is at 7c:69:f6:f2:da:7f 24 18.618696 64.235.98.226 172.31.17.42 DNS 93 Standard query response 0x5f81 A stun-ca.calix.com A 99.79.144.131 25 18.618739 64.235.98.226 172.31.17.42 DNS 93 Standard query response 0x5f81 A stun-ca.calix.com A 99.79.144.131 26 18.635840 8.8.8.8 172.31.17.42 DNS 93 Standard query response 0x5f81 A stun-ca.calix.com A 99.79.144.131 27 18.635870 8.8.8.8 172.31.17.42 DNS 93 Standard query response 0x5f81 A stun-ca.calix.com A 99.79.144.131 28 18.645352 99.79.144.131 172.31.17.42 CLASSIC-STUN 86 Message: Binding Response 29 18.645384 99.79.144.131 172.31.17.42 CLASSIC-STUN 86 Message: Binding Response 30 21.478185 Calix_07:31:f7 Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.28 31 21.478190 Calix_07:31:f7 Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.28 32 28.670541 64.235.98.226 172.31.17.42 DNS 141 Standard query response 0x8b10 A 0.ca.pool.ntp.org A 198.27.76.102 A 162.159.200.1 A 205.206.70.40 A 217.180.209.214 33 28.670575 64.235.98.226 172.31.17.42 DNS 141 Standard query response 0x8b10 A 0.ca.pool.ntp.org A 198.27.76.102 A 162.159.200.1 A 205.206.70.40 A 217.180.209.214 34 28.670688 64.235.98.226 172.31.17.42 DNS 132 Standard query response 0xbbba AAAA 0.ca.pool.ntp.org SOA b.ntpns.org 35 28.670717 64.235.98.226 172.31.17.42 DNS 132 Standard query response 0xbbba AAAA 0.ca.pool.ntp.org SOA b.ntpns.org 36 28.677733 64.235.98.226 172.31.17.42 DNS 141 Standard query response 0xe1d8 A 1.ca.pool.ntp.org A 205.206.70.42 A 216.232.132.102 A 149.56.37.32 A 207.210.46.249 37 28.677747 64.235.98.226 172.31.17.42 DNS 141 Standard query response 0xe1d8 A 1.ca.pool.ntp.org A 205.206.70.42 A 216.232.132.102 A 149.56.37.32 A 207.210.46.249 38 28.677932 64.235.98.226 172.31.17.42 DNS 132 Standard query response 0xe409 AAAA 1.ca.pool.ntp.org SOA b.ntpns.org 39 28.677946 64.235.98.226 172.31.17.42 DNS 132 Standard query response 0xe409 AAAA 1.ca.pool.ntp.org SOA b.ntpns.org 40 28.684459 64.235.98.226 172.31.17.42 DNS 152 Standard query response 0x3d28 A 0.north-america.pool.ntp.org A 65.108.76.171 A 206.108.0.132 A 216.6.2.70 A 198.199.14.18 41 28.684489 64.235.98.226 172.31.17.42 DNS 152 Standard query response 0x3d28 A 0.north-america.pool.ntp.org A 65.108.76.171 A 206.108.0.132 A 216.6.2.70 A 198.199.14.18 42 28.684607 64.235.98.226 172.31.17.42 DNS 143 Standard query response 0x5359 AAAA 0.north-america.pool.ntp.org SOA b.ntpns.org 43 28.684648 64.235.98.226 172.31.17.42 DNS 143 Standard query response 0x5359 AAAA 0.north-america.pool.ntp.org SOA b.ntpns.org 44 28.691310 64.235.98.226 172.31.17.42 DNS 141 Standard query response 0x7977 A 0.us.pool.ntp.org A 50.205.244.24 A 162.159.200.1 A 162.159.200.123 A 162.248.241.94 45 28.691340 64.235.98.226 172.31.17.42 DNS 141 Standard query response 0x7977 A 0.us.pool.ntp.org A 50.205.244.24 A 162.159.200.1 A 162.159.200.123 A 162.248.241.94 46 28.691468 64.235.98.226 172.31.17.42 DNS 132 Standard query response 0x6d87 AAAA 0.us.pool.ntp.org SOA b.ntpns.org 47 28.691498 64.235.98.226 172.31.17.42 DNS 132 Standard query response 0x6d87 AAAA 0.us.pool.ntp.org SOA b.ntpns.org 48 28.693444 8.8.8.8 172.31.17.42 DNS 141 Standard query response 0x8b10 A 0.ca.pool.ntp.org A 199.182.221.110 A 205.206.70.42 A 192.95.27.155 A 194.0.5.123 49 28.693478 8.8.8.8 172.31.17.42 DNS 141 Standard query response 0x8b10 A 0.ca.pool.ntp.org A 199.182.221.110 A 205.206.70.42 A 192.95.27.155 A 194.0.5.123 50 28.693535 8.8.8.8 172.31.17.42 DNS 132 Standard query response 0xbbba AAAA 0.ca.pool.ntp.org SOA c.ntpns.org 51 28.693550 8.8.8.8 172.31.17.42 DNS 132 Standard query response 0xbbba AAAA 0.ca.pool.ntp.org SOA c.ntpns.org 52 28.700935 8.8.8.8 172.31.17.42 DNS 141 Standard query response 0xe1d8 A 1.ca.pool.ntp.org A 216.197.156.83 A 162.159.200.123 A 217.180.209.214 A 208.81.1.244 53 28.700967 8.8.8.8 172.31.17.42 DNS 141 Standard query response 0xe1d8 A 1.ca.pool.ntp.org A 216.197.156.83 A 162.159.200.123 A 217.180.209.214 A 208.81.1.244 54 28.701152 8.8.8.8 172.31.17.42 DNS 132 Standard query response 0xe409 AAAA 1.ca.pool.ntp.org SOA i.ntpns.org 55 28.701182 8.8.8.8 172.31.17.42 DNS 132 Standard query response 0xe409 AAAA 1.ca.pool.ntp.org SOA i.ntpns.org 56 28.701736 8.8.8.8 172.31.17.42 DNS 143 Standard query response 0x5359 AAAA 0.north-america.pool.ntp.org SOA e.ntpns.org 57 28.701767 8.8.8.8 172.31.17.42 DNS 143 Standard query response 0x5359 AAAA 0.north-america.pool.ntp.org SOA e.ntpns.org 58 28.708145 8.8.8.8 172.31.17.42 DNS 141 Standard query response 0x7977 A 0.us.pool.ntp.org A 64.251.10.152 A 50.205.244.113 A 204.93.207.12 A 69.164.213.136 59 28.708147 8.8.8.8 172.31.17.42 DNS 152 Standard query response 0x3d28 A 0.north-america.pool.ntp.org A 38.229.62.9 A 66.220.9.122 A 144.172.118.20 A 129.250.35.251 60 28.708160 8.8.8.8 172.31.17.42 DNS 141 Standard query response 0x7977 A 0.us.pool.ntp.org A 64.251.10.152 A 50.205.244.113 A 204.93.207.12 A 69.164.213.136 61 28.708165 8.8.8.8 172.31.17.42 DNS 152 Standard query response 0x3d28 A 0.north-america.pool.ntp.org A 38.229.62.9 A 66.220.9.122 A 144.172.118.20 A 129.250.35.251 62 28.714399 8.8.8.8 172.31.17.42 DNS 132 Standard query response 0x6d87 AAAA 0.us.pool.ntp.org SOA f.ntpns.org 63 28.714429 8.8.8.8 172.31.17.42 DNS 132 Standard query response 0x6d87 AAAA 0.us.pool.ntp.org SOA f.ntpns.org 64 37.872289 64.235.98.226 172.31.17.42 DNS 93 Standard query response 0xa7dc A gcs6-ca.calix.com A 52.60.181.28 65 37.872307 64.235.98.226 172.31.17.42 DNS 93 Standard query response 0xa7dc A gcs6-ca.calix.com A 52.60.181.28 66 37.896074 8.8.8.8 172.31.17.42 DNS 93 Standard query response 0xa7dc A gcs6-ca.calix.com A 52.60.181.28 67 37.896104 8.8.8.8 172.31.17.42 DNS 93 Standard query response 0xa7dc A gcs6-ca.calix.com A 52.60.181.28 68 38.030652 52.60.181.28 172.31.17.42 TCP 74 8443 → 39182 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=236259536 TSecr=5140577 WS=128 69 38.030679 52.60.181.28 172.31.17.42 TCP 74 [TCP Out-Of-Order] 8443 → 39182 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=236259536 TSecr=5140577 WS=128 70 38.056085 52.60.181.28 172.31.17.42 TLSv1.2 203 Server Hello, Change Cipher Spec, Encrypted Handshake Message 71 38.056112 52.60.181.28 172.31.17.42 TCP 203 [TCP Retransmission] 8443 → 39182 [PSH, ACK] Seq=1 Ack=518 Win=62208 Len=137 TSval=236259561 TSecr=5140580 72 38.083558 52.60.181.28 172.31.17.42 TCP 66 8443 → 39182 [ACK] Seq=138 Ack=822 Win=61952 Len=0 TSval=236259589 TSecr=5140582 73 38.083584 52.60.181.28 172.31.17.42 TCP 66 [TCP Dup ACK 72#1] 8443 → 39182 [ACK] Seq=138 Ack=822 Win=61952 Len=0 TSval=236259589 TSecr=5140582 74 38.086084 52.60.181.28 172.31.17.42 TCP 66 8443 → 39182 [ACK] Seq=138 Ack=2270 Win=60544 Len=0 TSval=236259591 TSecr=5140583 75 38.086096 52.60.181.28 172.31.17.42 TCP 66 [TCP Dup ACK 74#1] 8443 → 39182 [ACK] Seq=138 Ack=2270 Win=60544 Len=0 TSval=236259591 TSecr=5140583 76 38.086141 52.60.181.28 172.31.17.42 TCP 66 8443 → 39182 [ACK] Seq=138 Ack=3718 Win=59136 Len=0 TSval=236259591 TSecr=5140583 77 38.086143 52.60.181.28 172.31.17.42 TCP 66 [TCP Dup ACK 76#1] 8443 → 39182 [ACK] Seq=138 Ack=3718 Win=59136 Len=0 TSval=236259591 TSecr=5140583 78 38.086540 52.60.181.28 172.31.17.42 TCP 66 8443 → 39182 [ACK] Seq=138 Ack=4326 Win=58624 Len=0 TSval=236259592 TSecr=5140583 79 38.086543 52.60.181.28 172.31.17.42 TCP 66 [TCP Dup ACK 78#1] 8443 → 39182 [ACK] Seq=138 Ack=4326 Win=58624 Len=0 TSval=236259592 TSecr=5140583 80 38.087740 52.60.181.28 172.31.17.42 TLSv1.2 340 Application Data 81 38.087767 52.60.181.28 172.31.17.42 TCP 340 [TCP Retransmission] 8443 → 39182 [PSH, ACK] Seq=138 Ack=4326 Win=58624 Len=274 TSval=236259593 TSecr=5140583 82 38.120573 52.60.181.28 172.31.17.42 TCP 66 8443 → 39182 [ACK] Seq=412 Ack=4357 Win=58624 Len=0 TSval=236259626 TSecr=5140586 83 38.120599 52.60.181.28 172.31.17.42 TCP 66 [TCP Dup ACK 82#1] 8443 → 39182 [ACK] Seq=412 Ack=4357 Win=58624 Len=0 TSval=236259626 TSecr=5140586 84 38.120613 52.60.181.28 172.31.17.42 TLSv1.2 97 Encrypted Alert 85 38.120620 52.60.181.28 172.31.17.42 TCP 97 [TCP Retransmission] 8443 → 39182 [PSH, ACK] Seq=412 Ack=4357 Win=58624 Len=31 TSval=236259626 TSecr=5140586 86 38.120625 52.60.181.28 172.31.17.42 TCP 66 8443 → 39182 [FIN, ACK] Seq=443 Ack=4357 Win=58624 Len=0 TSval=236259626 TSecr=5140586 87 38.120629 52.60.181.28 172.31.17.42 TCP 66 [TCP Out-Of-Order] 8443 → 39182 [FIN, ACK] Seq=443 Ack=4357 Win=58624 Len=0 TSval=236259626 TSecr=5140586 88 38.121583 52.60.181.28 172.31.17.42 TCP 66 8443 → 39182 [ACK] Seq=444 Ack=4358 Win=58624 Len=0 TSval=236259627 TSecr=5140586 89 38.121595 52.60.181.28 172.31.17.42 TCP 66 [TCP Dup ACK 88#1] 8443 → 39182 [ACK] Seq=444 Ack=4358 Win=58624 Len=0 TSval=236259627 TSecr=5140586 90 38.146682 52.60.181.28 172.31.17.42 TCP 74 8443 → 33372 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=236259652 TSecr=5140589 WS=128 91 38.146708 52.60.181.28 172.31.17.42 TCP 74 [TCP Out-Of-Order] 8443 → 33372 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=236259652 TSecr=5140589 WS=128 92 38.172161 52.60.181.28 172.31.17.42 TLSv1.2 203 Server Hello, Change Cipher Spec, Encrypted Handshake Message 93 38.172188 52.60.181.28 172.31.17.42 TCP 203 [TCP Retransmission] 8443 → 33372 [PSH, ACK] Seq=1 Ack=518 Win=62208 Len=137 TSval=236259678 TSecr=5140591 94 38.199727 52.60.181.28 172.31.17.42 TCP 66 8443 → 33372 [ACK] Seq=138 Ack=1101 Win=61696 Len=0 TSval=236259705 TSecr=5140594 95 38.199753 52.60.181.28 172.31.17.42 TCP 66 [TCP Dup ACK 94#1] 8443 → 33372 [ACK] Seq=138 Ack=1101 Win=61696 Len=0 TSval=236259705 TSecr=5140594 96 38.202039 52.60.181.28 172.31.17.42 TCP 66 8443 → 33372 [ACK] Seq=138 Ack=2549 Win=60288 Len=0 TSval=236259707 TSecr=5140594 97 38.202065 52.60.181.28 172.31.17.42 TCP 66 [TCP Dup ACK 96#1] 8443 → 33372 [ACK] Seq=138 Ack=2549 Win=60288 Len=0 TSval=236259707 TSecr=5140594 98 38.202077 52.60.181.28 172.31.17.42 TCP 66 8443 → 33372 [ACK] Seq=138 Ack=3997 Win=58880 Len=0 TSval=236259707 TSecr=5140594 99 38.202084 52.60.181.28 172.31.17.42 TCP 66 [TCP Dup ACK 98#1] 8443 → 33372 [ACK] Seq=138 Ack=3997 Win=58880 Len=0 TSval=236259707 TSecr=5140594 100 38.202619 52.60.181.28 172.31.17.42 TCP 66 8443 → 33372 [ACK] Seq=138 Ack=4560 Win=58368 Len=0 TSval=236259708 TSecr=5140594the ISP router still didn't connect, but at least we have relevant info about it.
-
If you have the source port in an 802.1q VLAN though it will not pass anything but that VLAN so whatever VLAN tagging the ISP router may or may not be using would get dropped and not appear there.
The ports the ISP router traffic is using need to pass all tagged traffic. -
@stephenw10 ok, so maybe RSPAN VLAN is not required for this then. Let me try without it.
-
Alright same procedure without RSPAN VLAN.
34 21.469965 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x6b391354 35 21.469991 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x6b391354 36 21.477901 172.31.16.1 172.31.17.42 DHCP 398 DHCP Offer - Transaction ID 0x6b391354 37 21.477929 172.31.16.1 172.31.17.42 DHCP 398 DHCP Offer - Transaction ID 0x6b391354 38 21.610081 0.0.0.0 255.255.255.255 DHCP 342 DHCP Request - Transaction ID 0x6b391354 39 21.610104 0.0.0.0 255.255.255.255 DHCP 342 DHCP Request - Transaction ID 0x6b391354 40 21.619714 172.31.16.1 172.31.17.42 DHCP 398 DHCP ACK - Transaction ID 0x6b391354 41 21.619814 172.31.16.1 172.31.17.42 DHCP 398 DHCP ACK - Transaction ID 0x6b391354 42 23.471686 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 43 23.471688 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 44 24.470907 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 45 24.470913 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 46 25.470906 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 47 25.470911 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 48 28.471041 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 49 28.471046 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 50 29.306140 Calix_6b:e8:f7 Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.42 51 29.306145 Calix_6b:e8:f7 Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.42 52 29.306893 Cisco_f2:da:7f Calix_6b:e8:f7 ARP 60 172.31.16.1 is at 7c:69:f6:f2:da:7f 53 29.312275 64.235.98.226 172.31.17.42 DNS 93 Standard query response 0x0d4a A gcs6-ca.calix.com A 52.60.181.28 54 29.336235 8.8.8.8 172.31.17.42 DNS 93 Standard query response 0x0d4a A gcs6-ca.calix.com A 52.60.181.28 55 29.338204 52.60.181.28 172.31.17.42 TCP 74 8443 → 40880 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=237229642 TSecr=5237588 WS=128 56 29.363891 52.60.181.28 172.31.17.42 TLSv1.2 203 Server Hello, Change Cipher Spec, Encrypted Handshake Message 57 29.391025 52.60.181.28 172.31.17.42 TCP 66 8443 → 40880 [ACK] Seq=138 Ack=822 Win=61952 Len=0 TSval=237229695 TSecr=5237593 58 29.393327 52.60.181.28 172.31.17.42 TCP 66 8443 → 40880 [ACK] Seq=138 Ack=2270 Win=60544 Len=0 TSval=237229697 TSecr=5237593 59 29.393353 52.60.181.28 172.31.17.42 TCP 66 8443 → 40880 [ACK] Seq=138 Ack=3718 Win=59136 Len=0 TSval=237229697 TSecr=5237593 60 29.393884 52.60.181.28 172.31.17.42 TCP 66 8443 → 40880 [ACK] Seq=138 Ack=4326 Win=58624 Len=0 TSval=237229698 TSecr=5237593 61 29.394462 52.60.181.28 172.31.17.42 TLSv1.2 340 Application Data 62 29.426693 52.60.181.28 172.31.17.42 TCP 66 8443 → 40880 [ACK] Seq=412 Ack=4357 Win=58624 Len=0 TSval=237229730 TSecr=5237597 63 29.426706 52.60.181.28 172.31.17.42 TLSv1.2 97 Encrypted Alert 64 29.426752 52.60.181.28 172.31.17.42 TCP 66 8443 → 40880 [FIN, ACK] Seq=443 Ack=4357 Win=58624 Len=0 TSval=237229730 TSecr=5237597 65 29.426923 52.60.181.28 172.31.17.42 TCP 66 8443 → 40880 [ACK] Seq=444 Ack=4358 Win=58624 Len=0 TSval=237229731 TSecr=5237597 66 29.452715 52.60.181.28 172.31.17.42 TCP 74 8443 → 36033 [SYN, ACK] Seq=0 Ack=1 Win=62643 Len=0 MSS=1460 SACK_PERM=1 TSval=237229756 TSecr=5237599 WS=128 67 29.470799 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 68 29.470804 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 69 29.478182 52.60.181.28 172.31.17.42 TLSv1.2 203 Server Hello, Change Cipher Spec, Encrypted Handshake Message 70 29.505481 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=138 Ack=1101 Win=61696 Len=0 TSval=237229809 TSecr=5237604 71 29.507689 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=138 Ack=2549 Win=60288 Len=0 TSval=237229811 TSecr=5237605 72 29.507715 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=138 Ack=3997 Win=58880 Len=0 TSval=237229811 TSecr=5237605 73 29.507948 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=138 Ack=4560 Win=58368 Len=0 TSval=237229812 TSecr=5237605 74 29.511811 52.60.181.28 172.31.17.42 TLSv1.2 571 Application Data 75 29.576321 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=643 Ack=5156 Win=57856 Len=0 TSval=237229880 TSecr=5237612 76 29.577110 52.60.181.28 172.31.17.42 TLSv1.2 895 Application Data 77 29.610470 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=1472 Ack=5808 Win=57216 Len=0 TSval=237229914 TSecr=5237615 78 29.611045 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=1472 Ack=6921 Win=56576 Len=0 TSval=237229915 TSecr=5237615 79 29.613017 52.60.181.28 172.31.17.42 TLSv1.2 1021 Application Data 80 29.644515 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=2427 Ack=7572 Win=56576 Len=0 TSval=237229948 TSecr=5237618 81 29.644790 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=2427 Ack=8126 Win=56576 Len=0 TSval=237229949 TSecr=5237619 82 29.645791 52.60.181.28 172.31.17.42 TLSv1.2 771 Application Data 83 30.341325 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=8779 Win=56576 Len=0 TSval=237230645 TSecr=5237688 84 30.345607 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=10227 Win=56576 Len=0 TSval=237230649 TSecr=5237689 85 30.345633 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=11675 Win=56576 Len=0 TSval=237230649 TSecr=5237689 86 30.345647 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=14571 Win=56576 Len=0 TSval=237230649 TSecr=5237689 87 30.345653 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=17467 Win=53760 Len=0 TSval=237230649 TSecr=5237689 88 30.345914 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=18915 Win=56576 Len=0 TSval=237230650 TSecr=5237689 89 30.345926 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=21811 Win=56576 Len=0 TSval=237230650 TSecr=5237689 90 30.366516 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=24707 Win=56576 Len=0 TSval=237230670 TSecr=5237691 91 30.370821 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=26155 Win=56576 Len=0 TSval=237230675 TSecr=5237691 92 30.370833 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=30499 Win=56576 Len=0 TSval=237230675 TSecr=5237691 93 30.370877 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=33395 Win=56576 Len=0 TSval=237230675 TSecr=5237691 94 30.371050 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=34843 Win=56576 Len=0 TSval=237230675 TSecr=5237691 95 30.371064 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=36291 Win=56576 Len=0 TSval=237230675 TSecr=5237691 96 30.371882 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=37739 Win=56576 Len=0 TSval=237230676 TSecr=5237691 97 30.371894 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=39187 Win=56576 Len=0 TSval=237230676 TSecr=5237691 98 30.372797 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=3132 Ack=39527 Win=56576 Len=0 TSval=237230677 TSecr=5237691 99 30.376856 52.60.181.28 172.31.17.42 TLSv1.2 1370 Application Data 100 30.403991 52.60.181.28 172.31.17.42 TCP 66 8443 → 36033 [ACK] Seq=4436 Ack=40179 Win=56576 Len=0 TSval=237230708 TSecr=5237694Different messages, results much the same as before. ISP router no connection.
-
Might need to test it with something else then. Make sure you can send tagged traffic between the ports and capture it.
You might need to use 'Dot1q-Tunnel' mode:
https://www.cisco.com/c/en/us/td/docs/switches/lan/csbss/CBS220/Adminstration-Guide/cbs-220-admin-guide/vlan-management.html#ID-0000320bSince I can see nothing about port based VLANs there.
Steve
-
@stephenw10 Ok, some mild progres. Enabeling Dot1q-Tunnel on the ports I was using allowed the ISP router to full connect. Problem is the only packets I am snooping on the Pfsense box are DNS, DHCP and ARP. Nothing with VLAN information.
Edit: Fanstatic! I can now eliminate the SFP converter and directly connect the fiber to the switch with the Dot1q-Tunnel trick.
Edit2: Routing at full speed on ISP router. We are making progress here. Now we just need to get Pfsense to do the same.
Edit3: yep still capped at 20Mb/s on the Pfsense box.
-
Ok, so you are still not seeing all the traffic to/from the ISP router?
But you can see it using DHCP to pull an address in the correct subnet from upstream?
That should show any special dhcp client options it's using.
Steve
-
@stephenw10 said in New Fiber install, fresh Pfsense install, only getting 20Mbps up/down:
Ok, so you are still not seeing all the traffic to/from the ISP router?
But you can see it using DHCP to pull an address in the correct subnet from upstream?
That should show any special dhcp client options it's using.
Steve
Perhaps not true. If he is only seeing dhcp and other broadcasts, he’s either not capturing in promiscious mode, or he’s only seeing the broadcasts in the native VLAN which might be used for ISP management and has nothing to do with the user/internet VLAN
