New Fiber install, fresh Pfsense install, only getting 20Mbps up/down
-
what brand are the ISP router?
-
@cool_corona Calix
-
Have you told the support that you want to use your own router?
So they will release the MAC and let you do that?
-
@cool_corona Yes, they wont allow it.
-
@jddoxtator Have you tried to spoof the mac of the org router?
-
@cool_corona Yes, the spoof has been enabled since the start.
-
Are there any dip switches in the converter?
-
Ok, testing this locally I expect to be able to see the tagged traffic in the GUI packet capture if the view detail is set to full however there is some oddness there. I'm digging into that but it will show there if you do not filter like:
19:36:07.585799 90:ec:77:1f:8a:5f > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 229, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.229.5.10 tell 10.229.5.1, length 28There is no question of which VLAN is in use there.
You can also run at the CLI something like:
tcpdump -nvve -i ix0And you will see all the traffic on the interface including vlan tags.
Steve
-
@stephenw10 Alright tried the console code and got a different VLAN again
15:21:32.364086 3c:ec:ef:70:19:a6 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 346: vlan 1, p 0, ethertype IPv4, (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 3c:ec:ef:70:19:a6, length 300, xid 0xa6981c02, Flags [none] (0x0000) Client-Ethernet-Address 3c:ec:ef:70:19:a6 Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: Discover Client-ID Option 61, length 7: ether 3c:ec:ef:70:19:a6 Hostname Option 12, length 7: "pfSense" Parameter-Request Option 55, length 10: Subnet-Mask, BR, Time-Zone, Classless-Static-Route Default-Gateway, Domain-Name, Domain-Name-Server, Hostname Option 119, MTU 15:21:32.865804 10:f9:20:89:a0:f6 > 01:00:0c:cc:cc:cc, 802.3, length 40: LLC, dsap SNAP (0xaa) Individual, ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid DTP (0x2004), length 38: DTPv1, length 38 Domain TLV (0x0001) TLV, length 11, Packet Status TLV (0x0002) TLV, length 5, 0x81 DTP type TLV (0x0003) TLV, length 5, 0xa5 Neighbor TLV (0x0004) TLV, length 10, 10:f9:20:89:a0:f6 15:21:33.395704 3c:ec:ef:70:19:a6 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 346: vlan 1, p 0, ethertype IPv4, (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 3c:ec:ef:70:19:a6, length 300, xid 0xa6981c02, secs 1, Flags [none] (0x0000) Client-Ethernet-Address 3c:ec:ef:70:19:a6 Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: Discover Client-ID Option 61, length 7: ether 3c:ec:ef:70:19:a6 Hostname Option 12, length 7: "pfSense" Parameter-Request Option 55, length 10: Subnet-Mask, BR, Time-Zone, Classless-Static-Route Default-Gateway, Domain-Name, Domain-Name-Server, Hostname Option 119, MTU 15:21:33.865863 10:f9:20:89:a0:f6 > 01:00:0c:cc:cc:cc, 802.3, length 40: LLC, dsap SNAP (0xaa) Individual, ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid DTP (0x2004), length 38: DTPv1, length 38 Domain TLV (0x0001) TLV, length 11, Packet Status TLV (0x0002) TLV, length 5, 0x81 DTP type TLV (0x0003) TLV, length 5, 0xa5 Neighbor TLV (0x0004) TLV, length 10, 10:f9:20:89:a0:f6 15:21:34.410039 3c:ec:ef:70:19:a6 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 346: vlan 1, p 0, ethertype IPv4, (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 3c:ec:ef:70:19:a6, length 300, xid 0xa6981c02, secs 2, Flags [none] (0x0000) Client-Ethernet-Address 3c:ec:ef:70:19:a6 Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: Discover Client-ID Option 61, length 7: ether 3c:ec:ef:70:19:a6 Hostname Option 12, length 7: "pfSense" Parameter-Request Option 55, length 10: Subnet-Mask, BR, Time-Zone, Classless-Static-Route Default-Gateway, Domain-Name, Domain-Name-Server, Hostname Option 119, MTU 15:21:35.057589 3c:ec:ef:70:1c:f5 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 3c:ec:ef:70:1c:f5, length 300, xid 0xc9c42930, Flags [none] (0x0000) Client-Ethernet-Address 3c:ec:ef:70:1c:f5 Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: Discover Client-ID Option 61, length 7: ether 3c:ec:ef:70:1c:f5 MSZ Option 57, length 2: 576 Parameter-Request Option 55, length 7: Subnet-Mask, Default-Gateway, Domain-Name-Server, Hostname Domain-Name, BR, NTP Vendor-Class Option 60, length 12: "udhcp 1.23.1" 15:21:35.108688 10:f9:20:89:a0:f6 > 01:00:0c:cc:cc:cc, ethertype 802.1Q (0x8100), length 560: vlan 1, p 7, LLC, dsap SNAP (0xaa) Individual, ssap SNAP (0xaa) Command, ctrl 0x03: oui Cisco (0x00000c), pid CDP (0x2000), length 534: CDPv2, ttl: 180s, checksum: 0x72f9 (unverified), length 534 Device-ID (0x01), value length: 32 bytes: 'MtBrydges-4507-2.nftctelecom.com' Version String (0x05), value length: 285 bytes: Cisco IOS Software, IOS-XE Software, Catalyst 4500 L3 Switch Software (cat4500es8-UNIVERSALK9-M), Version 03.09.00.E RELEASE SOFTWARE (fc1) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2016 by Cisco Systems, Inc. Compiled Tue 19-Jul-16 12:34 by prod_rel_team Platform (0x06), value length: 17 bytes: 'cisco WS-C4507R+E' Address (0x02), value length: 13 bytes: IPv4 (1) 172.31.16.2 Port-ID (0x03), value length: 19 bytes: 'GigabitEthernet6/15' Capability (0x04), value length: 4 bytes: (0x00000029): Router, L2 Switch, IGMP snooping Prefixes (0x07), value length: 10 bytes: IPv4 Prefixes (2): 172.31.16.0/22 192.168.3.0/24 VTP Management Domain (0x09), value length: 6 bytes: 'Packet' Native VLAN ID (0x0a), value length: 2 bytes: 85 Duplex (0x0b), value length: 1 byte: full AVVID trust bitmap (0x12), value length: 1 byte: 0x00 AVVID untrusted ports CoS (0x13), value length: 1 byte: 0x00 Management Addresses (0x16), value length: 13 bytes: IPv4 (1) 172.31.16.2 unknown field type (0x1a), value length: 12 bytes: 0x0000: 0000 0001 0000 0000 ffff ffff unknown field type (0x1b), value length: 1 byte: 0x0000: 00 unknown field type (0x1f), value length: 1 byte: 0x0000: 00 unknown field type (0x1005), value length: 20 bytes: 0x0000: 5753 2d58 3435 2d53 5550 382d 4500 2830 0x0010: 2972 3f7c unknown field type (0x1004), value length: 15 bytes: 0x0000: 6530 3266 2e36 6461 352e 3136 3830 00 unknown field type (0x1003), value length: 1 byte: 0x0000: 31I copied everything from connection until response from a cisco router. I see VLAN 1 but I tried that and it gives me no IP. Same as any other VLAN I have tried.
-
That's after setting VLAN1? It looks like dhcp requests from pfSense tagged as that.
You might try switching the ISP router in and back out before the pcap to try to get some tagged traffic from the ISP as you did before with the ARP packet.
Ultimately the only way to know for sure is to setup a switch with a mirror port so you can capture exactly what the ISP router is doing.
The other thing is that you are almost certainly not the first person trying this. Someone else may have documented what's required for that ISP. Somewhere.
Steve
-
@jddoxtator To make this easy on yourself, try and connect a switch between the media converter and the original ISP router.
Connect your pfSense to one switchport and set its WAN port to either no IP address, or a fixed random private IP address.
Start a packet capture on WAN, and connect the ISP router to the switch.When the ISP router is connected it will attempt to get a IP address via DHCP frames which is broadcasted - and include the VLAN tag the ISP router is using.
Those broadcasts should also reach your pfSense if the switch is a dumb non-managed Layer2 switch. If it is a smart VLAN capable/managed switch, this will not work, and you will have to setup a mirrorport/spanport on the switch which mirrors the ISP router port to your pfsense port. -
@stephenw10 Before setting VLAN 1
I set the pfsense router back to stock and switched the WAN port from the ISP router to the pfsense router after the ISP router had connected.
-
Update: My old switch died, would not resolve DHCP anymore for some reason. Long story short, got a Cisco CBS220-24FP-4X capable of VLAN's and specifically PVST+
Some interesting behavior after getting this switch installed and running.
I set up a VLAN across one of the SFP 10Gbe ports and one of the copper 1Gbe ports. I then connected the fiber line directly to the switch SFP port and routers to the copper port.
Pfsense picked up the WAN signal and did it's usual thing connecting at 20Mb/s. However, the ISP router would not connect at all with the switch routing the fiber to the copper port on the VLAN.
There is something fundamentally different in how these two routers are connecting and I have no idea what.
-
@jddoxtator I still suspect putting a managed switch with port mirroring on the WAN line of your ISP router would be the most efficient way of finding out what works.
-
@patch I tried multiple setups and most gave me nothing at all on packet capture.
I used multicast on 3 ports across a single VLAN and only once I captured the ISP router sending an ARP request for the same gateway that Pfsense uses, but it could not connect.
Every other time there was no traffic to record.
This was all the packets I got:
1 0.000000 Calix_7a:06:4a Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.83 2 1.917011 172.31.16.23 224.0.0.1 ICMP 60 Mobile IP Advertisement (Normal router advertisement) 3 10.450749 Calix_4c:f9:11 Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.19.81Just a reminder, when the ISP router is connected directly, it connects with Gateway 192.24.57.1 not 172.31.16.1
-
@jddoxtator said in New Fiber install, fresh Pfsense install, only getting 20Mbps up/down:
Cisco CBS220-24FP-4X
That switch does port mirroring. Remove the VLANs from the switch. It needs to pass the tagged traffic from the ISP router so put two ports in port-vlan mode or whatever Cisco has renamed that.
Then mirror one of those ports and capture on it.
Steve
-
@stephenw10 Ok, I deleted all VLAN's and my multicast's. Only problem is I'm not sure where port mirroring is in this switch. I'm guessing that this is more on the physical level under Port configuration. The only thing I see there that involves multiple ports is Link Aggregation. Could this be what I am looking for?
Edit: Found the manual online. I was looking in the wrong spot apparently. They put SPAN which is their port mirroring under the statistics tab.... strange choice but OK. Now the SPAN interface says I require a VLAN to define the mirror, so I'm guessing this is where I go to VLAN and select the two ports to have in the same VLAN group.
-
Nope not LAG. Looks like Cisco are using some combination of the terms mirroring, port monitoring and span port.
https://www.cisco.com/c/en/us/td/docs/switches/lan/csbss/CBS220/Adminstration-Guide/cbs-220-admin-guide/status-and-statistics.html?bookSearch=true#Cisco_Concept.dita_86e4dbba-7744-408d-b5e2-c55428a982b6
or
https://www.cisco.com/c/en/us/td/docs/switches/lan/csbss/CBS220/CLI-Guide/b_220CLI/port_monitor_commands.htmlSteve
-
@stephenw10 Think I almost have this figured out.
I have to list the two data ports as source then the listening port as destination all under the same session ID. Then they have to be in the same VLAN group and I think that should work. I hope, lets see.
-
Ok, so this setup gives me a bunch of local network ARP requests
1 0.000000 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.121 2 0.106019 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 3 0.320230 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 4 0.609572 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 5 0.814689 ASUSTekC_f5:1f:a0 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.154 6 1.013517 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.121 7 1.105998 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 8 1.330691 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 9 1.622972 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 10 1.828057 ASUSTekC_f5:1f:a0 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.154 11 2.346824 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 12 2.636239 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 13 3.357319 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 14 3.649568 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 15 4.106170 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 16 4.370852 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 17 4.874859 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.121 18 5.105890 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 19 5.384213 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 20 5.809578 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 21 5.881565 ASUSTekC_f5:1f:a0 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.154 22 5.893358 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.121 23 6.105790 Cisco_f4:83:3a Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.155 24 6.397349 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 25 6.822939 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 26 6.876218 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.116 27 6.894748 ASUSTekC_f5:1f:a0 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.154 28 6.904138 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.121 29 7.411112 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 30 7.836240 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.116 31 7.889569 ASUSTekC_8c:16:e1 Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.116 32 7.908081 ASUSTekC_f5:1f:a0 Broadcast ARP 60 Who has 192.168.1.1? Tell 192.168.1.154 33 8.423936 RivetNet_c8:5f:5d Broadcast ARP 60 Who has 192.168.1.134? Tell 192.168.1.121 34 8.453629 Calix_0c:ae:2c Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.249 35 8.453633 Calix_0c:ae:2c Broadcast ARP 60 Who has 172.31.16.1? Tell 172.31.17.249ISP router still would not connect through this method. I think the VLAN isolation is not working as I am getting all my network devices.
