New Fiber install, fresh Pfsense install, only getting 20Mbps up/down
-
@jddoxtator Ahhh, think I just figured your strange packet capture. You had the ISP router connected to a switch, and the switch to the media converter right?
You then disconnected the ISP router from the switch the same time you connected your pfSense right?Then the first ARP frame is a broadcast from your ISP router because you had them both connected for a brief split second. And all the Spanning tree frames are from your switch…. :-)
AND: if that’s the case then the ARP frame should have your needed VLAN tag attached. So download and install Wireshark on your machine. Download the packet capture from your pfSense and open it in Wireshark. Inspect the ARP frame, and look at the Ethernet VLAN tag on that frame.
You then need to create that VLAN number on your pfSense, and reassign your WAN interface to that VLAN number on the NIC connected to the switch/media converter.
-
The ISP's router has to be within the address range you get. You can't just change the router address and expect it to work.
-
@keyser
Ah, This is very helpful.I am assuming that the ARP IP address is related to the fact that I have set the gateway manually to the same one as the ISP router and forced static IP for the gateway.
There was no switch between the converter and the ISP router before the WAN port was switched over to the WAN port on the pfsense device.
I will download this Wireshark you speak of and see the details.
Edit: Slight problem... I run Linux on all my devices, is there a wireshark for linux?
Edit2: nvm found it. -
My mistake, the ARP request came from some Calix device.
Here is the expanded log:
1 0.000000 Cisco_89:a0:f6 CDP/VTP/DTP/PAgP/UDLD DTP 60 Dynamic Trunk Protocol
2 0.967229 Calix_6b:e8:f7 Broadcast ARP 42 Who has 192.24.57.1? Tell 192.24.57.117
3 0.998761 Cisco_89:a0:f6 CDP/VTP/DTP/PAgP/UDLD DTP 60 Dynamic Trunk Protocol
4 1.696854 0.0.0.0 255.255.255.255 DHCP 342 DHCP Request - Transaction ID 0xa4d00549
5 2.001227 Cisco_89:a0:f6 CDP/VTP/DTP/PAgP/UDLD DTP 60 Dynamic Trunk Protocol
6 2.531984 0.0.0.0 255.255.255.255 DHCP 342 DHCP Request - Transaction ID 0xa4d00549
7 3.063618 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
8 3.569753 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x1c1ffc0e
9 4.041146 fe80::3eec:efff:fe70:1cf5 ff02::1:2 DHCPv6 98 Information-request XID: 0x0163ec CID: 0001000129f61dd33cecef701cf5
10 4.639807 0.0.0.0 255.255.255.255 DHCP 342 DHCP Request - Transaction ID 0xa4d00549
11 4.950686 fe80::3eec:efff:fe70:1cf5 ff02::1:2 DHCPv6 98 Information-request XID: 0x0163ec CID: 0001000129f61dd33cecef701cf5
12 5.083875 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
13 6.700670 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x1c1ffc0e
14 6.740756 fe80::3eec:efff:fe70:1cf5 ff02::1:2 DHCPv6 98 Information-request XID: 0x0163ec CID: 0001000129f61dd33cecef701cf5
15 7.089523 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
16 9.105414 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
17 9.422423 0.0.0.0 255.255.255.255 DHCP 342 DHCP Request - Transaction ID 0xa4d00549
18 9.461966 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0x1c1ffc0e
19 10.270858 fe80::3eec:efff:fe70:1cf5 ff02::1:2 DHCPv6 98 Information-request XID: 0x0163ec CID: 0001000129f61dd33cecef701cf5
20 11.122321 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
21 13.150367 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
22 15.168436 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
23 17.243260 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
24 17.290174 fe80::3eec:efff:fe70:1cf5 ff02::1:2 DHCPv6 98 Information-request XID: 0x0163ec CID: 0001000129f61dd33cecef701cf5
25 19.265960 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
26 20.521444 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0xa45704e7
27 21.269650 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
28 22.628473 0.0.0.0 255.255.255.255 DHCP 342 DHCP Discover - Transaction ID 0xa45704e7
29 23.297708 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814f
30 25.317116 Cisco_89:a0:f6 PVST+ STP 64 Conf. TC + Root = 24576/85/7c:69:f6:f2:da:40 Cost = 2 Port = 0x814fNot a network engineer, so not exactly sure what I am looking at, but I don't see anything that specifically references VLAN. Unless PVST+ is some kind of VLAN like protocol.
Editr: Search is your friend. It appears that PVST+ is a cisco brand Per VLAN Spanning Tree Plus. Though I don't see anything but a MAC address and ports, no IP to configure a VLAN from.
-
@keyser said in New Fiber install, fresh Pfsense install, only getting 20Mbps up/down:
@jddoxtator said in New Fiber install, fresh Pfsense install, only getting 20Mbps up/down:
Ok, I captured packets from WAN with nothing attached to make sure there was no activity, then started a new capture and unplugged the WAN from the ISP router and directly plugged it into the WAN on the Pfsense router
This is what I got after 30 seconds of capture:
02:28:01.732611 DTPv1, length 38
02:28:02.699840 ARP, Request who-has 192.24.57.1 tell 192.24.57.117, length 28
02:28:02.731372 DTPv1, length 38
02:28:03.429465 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300
02:28:03.733838 DTPv1, length 38
02:28:04.264595 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300
02:28:04.796229 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:05.302364 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300
02:28:05.773757 IP6 fe80::3eec:efff:fe70:1cf5.546 > ff02::1:2.547: UDP, length 36
02:28:06.372418 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300
02:28:06.683297 IP6 fe80::3eec:efff:fe70:1cf5.546 > ff02::1:2.547: UDP, length 36
02:28:06.816486 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:08.433281 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300
02:28:08.473367 IP6 fe80::3eec:efff:fe70:1cf5.546 > ff02::1:2.547: UDP, length 36
02:28:08.822134 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:10.838025 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:11.155034 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300
02:28:11.194577 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300
02:28:12.003469 IP6 fe80::3eec:efff:fe70:1cf5.546 > ff02::1:2.547: UDP, length 36
02:28:12.854932 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:14.882978 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:16.901047 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:18.975871 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:19.022785 IP6 fe80::3eec:efff:fe70:1cf5.546 > ff02::1:2.547: UDP, length 36
02:28:20.998571 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:22.254055 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300
02:28:23.002261 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:24.361084 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300
02:28:25.030319 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42
02:28:27.049727 STP 802.1d, Config, Flags [Topology change], bridge-id 8055.e0:2f:6d:a5:16:80.814f, length 42It looks like a bunch of spam of IP 0.0.0.68 complaining about topology change. What is interesting is the bridge ID. Is that Pfsense or the ISP gateway?
Well we can’t decode everything from this as that is only a summary “overview” of the capture. You need to open it in Wireshark or another pcap decoder application.
However, a few things is obvious. Your ISP is not your average setup since they run Spanning Tree to the client edge - that’s a new for me - never seen that before :-)
But there is also Cisco dynamic trunking protocol frames on the wire, so it seems your ISP is running some VLANs on the wire.The funny thing though… all the 0.0.0.0:68 frames is your pfSense trying to aqquire a IP address via DHCP - it doesn’t get any. So there is no Internet available to it - how on earth are you testing with success albeit very slow speed?
Forgot to address the connection with no IP. It does get one, but it seems it is the wrong gateway. They are currently still building out the network in my area, so there may be some insecure patch devices in the line for workers to access? That's my only thought...
-
Mmm, this does seem like either a VLAN is required or maybe priority tagging. Or possibly some DHCP client options.
A pcap of the ISP router connecting would show it either way.Steve
-
So if I am correct in my understanding.... It sounds like I just need to make a VLAN based around the IP address in that ARP request.
I have two IP's
Sender IP address: 192.24.57.117
Target IP address: 192.24.57.1
Target has to be the gatway VLAN and I have to apply this to WAN device?
Oh, and the Calix device is the ISP router, so this was a captured broadcast from the ISP router. I'm guessing I caught the echo off the gateway because it wasn't plugged into any switch. It was a really fast port swap and I had the recorder going when I did it.
Did some more research, and the VLAN tag should be 57 based on the IP addresses I think.
-
Easy enough to test. Create a VLAN interface with ID 57 on the current WAN interface (ix3?). Then reassign WAN to be that new VLAN (ix3.57).
Steve
-
@stephenw10
Tried that, but I think I am extrapolating the Tag ID wrong. as 57 did not work.I think it is the 802.1Q number I am after. Which is 0xa5 or a5 hex / 165 decimal
-
Try that then. Where are you reading that from?
-
That was from the sub menus of DTP. It did not work unfortunately.
I also found Originating VLAN: 85 in PVST+ , but that did not work as well.
I am about to try PID: PVSTP+ (0x010b) or 267 dec.
-
I had the same issues on FTTH here in Switzerland.
It was the SFP+ when mounted in a switch. In a converter it didnt get an address and exposed th routers MAC to the ISP and everything worked perfectly.
-
From what I understand by reading a description of Cisco's implementation of PVST+, DTP is part of the trunk that the routers use in their network. So we can safely ignore that.
STP seems to be the client side of the VLAN. This being the most important information I can find in STP protocol:
Originating VLAN (PVID): 85
Type: Originating VLAN (0x0000)
Length: 2
Originating VLAN: 85By my understanding that should make the VLAN 85, but that doesn't work. So there is still something missing.
-
what brand are the ISP router?
-
@cool_corona Calix
-
Have you told the support that you want to use your own router?
So they will release the MAC and let you do that?
-
@cool_corona Yes, they wont allow it.
-
@jddoxtator Have you tried to spoof the mac of the org router?
-
@cool_corona Yes, the spoof has been enabled since the start.
-
Are there any dip switches in the converter?
