Orange Fibre: DHCP gives a public ip address, internet is fast, then after a few minutes it slows, and eventually unable to ping anything on the internet

sloopbun

Hello,

I am on PfSense 2.7.0-RELEASE (amd64) and especially with the help of this post https://forum.netgate.com/post/1105083 from @keyser - I was able to get DHCP with Orange Fibre in France to give me a public ip address.

My problem is that while speed it initially good, after a few minutes it slows to a crawl and then stops responding. No ping to 8.8.8.8 or the gateway etc. Turning off the fibre box and turning it on again restores it for a few minutes.

I cannot see anything in the logs to indicate a problem, except for sendmsg errors out when it cannot ping the gateway for monitoring.

I am wondering if this is due to my configuration or something else? No such problem with the LiveBox5 in DMZ mode.

Here is a long explanation of my configuration:

I am new to trying to shift from the LiveBox5 to direct connection using PfSense. I can connect and DHCP gives me a 92.132.X.X address and I get good speeed internet for a few minutes, then it becomes very slow and eventually stops responding. The WAN still shows as up, but impossible to ping the gateway or any DNS like 8.8.8.8. If I unplug the fibre and replug it, then it reconnects and the cycle continues.

Has this behaviour been seen before? Is it something with my configuration?

I have a LiveBox5 that I want to replace with with a MC220L and a Huawei MA5671A.

On the MA7561A I set the config to match my LiveBox5.

/etc/init.d/sys.sh

	oem-generic)
		uci set sys.target.name=oem-generic
		uci set sys.features.mib_customized=1
		uci set sys.features.igmp_fast_leave=1
		uci set sys.features.dying_gasp_enabled=0
		uci set sys.features.suppress_power_level=0
		uci set sys.features.eeprom_sync_option='1'
		uci set sys.mib.vendor_id='ARLT'
		uci set sys.mib.ont_version='ARLTARLBF121\0\0'
		uci set sys.mib.equipment_id='ARLTXXXXXXX\0\0\0'

and

/etc/mibs/data_1g_8q.ini

# ONT-G
256 0 ARLT ARLTARLBF121\0\0 00000000 2 0 0 0 0 #0

fw_setenv target oem-generic
fw_setenv ont_serial XXXXXXXXXX
fw_setenv image0_version XXXXXXXXXX
fw_setenv image1_version XXXXXXXXXX
fw_setenv sgmii_mode 5

After reboot, I see onu ploamsg = 5 and the VLANs:

gtop

Nom :        ONU_GPE_VLAN_TABLE
Traitement du VLAN GPE

Nom : onu_gpe_vlan_table        ONU_GPE_VLAN_TREATMENT_TABLE
ID : 43
;;;;tagb;tagb;tagb;taga;taga;taga
no;inner not generate;outer not generate;discard enable;tpid;vid;treatment;tpid;vid;treatment
0;1;1 ; ;4 ; 1 ; 1;4 ; ;15
1;1;1 ; ; ; 851 ; 9 ; ; ;15
2;1;1 ; ; ; 840 ; 9 ; ; ;15
3;1;1 ; ; ; 838 ; 9 ; ; ;15
4;1;1 ; ; ; 835 ; 9 ; ; ;15
5;1;1 ; ; ; 832 ; 9 ; ; ;15
6;1;1;1 ; ; ;15 ; ; ;15
7 ; ; ;1 ; ; ;15 ; ; ;15
64;1;1 ; ; ; ;15 ; ; ;15
65;1;1 ; ;6 ; 851 ; 9 ; ; ;15
66;1;1 ; ;6 ; 840 ; 9 ; ; ;15
67;1;1 ; ;6 ; 838 ; 9 ; ; ;15
68;1;1 ; ;6 ; 835 ; 9 ; ; ;15
69;1;1 ; ;6 ; 832 ; 9 ; ; ;15
128;1;1 ; ; ; ;15 ; ; ;15
129;1;1 ; ; ; ;15 ; ; ;15
130 ; ;1 ; ; ; ;15 ; ; ;15

In pfsense I set the parent interface, re0 to match the MAC of the LiveBox5. Note that I did not get it to work unless specifiying Livebox4....

WAN is set to interface re0.832 with the following configuration(fti password is confirmed by Orange directly):

VLAN tagging 802.1p = 6

dhcp-class-identifier "sagem",dhcp-client-identifier 01:XX:XX:XX,user-class "+FSVDSL_livebox.Internet.softathome.Livebox4",option-90 00:00:00:00:00:00:00:00:00:00:00:1A:XX:XX:XX:XX
subnet-mask,broadcast-address,dhcp-lease-time,dhcp-renewal-time,dhcp-rebinding-time,domain-search,routers,domain-name-servers,option-90,option-125
vlan-pcp 6

The result is:

re0.832 : flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description : WAN
	ether xx:xx:xx
	inet6 fe80::e675:dcff:fe03:41b0%re0.832 prefixlen 64 scopeid 0xf
	inet 92.132.x.x netmask 0xfffff800 broadcast 92.132.191.255
	groups : vlan
	vlan : 832 vlanproto : 802.1q vlanpcp : 6 parent interface : re0
	media : Ethernet autoselect (1000baseT <full-duplex>)
	status : active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

dhclient.leases.re0.832
lease {
  interface "re0.832";
  fixed-address 92.132.X.X;
  next-server 80.10.233.37;
  option subnet-mask 255.255.248.0;
  option routers 92.132.184.1;
  option domain-name-servers 80.10.246.134,81.253.149.5;
  option host-name "pfSense";
  option broadcast-address 92.132.191.255;
  option dhcp-lease-time 604800;
  option dhcp-message-type 5;
  option dhcp-server-identifier 80.10.233.37;
  option dhcp-renewal-time 83744;
  option dhcp-rebinding-time 483840;
  option dhcp-client-identifier 1:e4:75:dc:3:41:b0;
  option option-90 0:0:0:0:0:0:0:0:0:0:0:X:X:X:XX;
  option domain-search "ANN.access.orange-multimedia.net.";
  option option-125 0:0:5:58:c:1:a:0:1:0:0:0:0:0:0:0:0;
  renew 3 2023/11/1 17:59:27;
  rebind 4 2023/11/2 11:26:15;
  expire 2 2023/11/7 18:43:43;
}

stephenw10

Hmm, that's quite specialist!

Is it the same length of time ever time?

Does the ARP entry for the gateway expire?

When it stop responding is there anything coming in from a packet capture?

This might require someone else with a similar setup to solve though.

Steve

keyser

@sloopbun said in Orange Fibre: DHCP gives a public ip address, internet is fast, then after a few minutes it slows, and eventually unable to ping anything on the internet:

MA5671A

Hi Sloopbun

I'm the author of the posts that got you this far, and I'd just like to comment that my setup - since fixing the DHCP renew issue - has be running flawlessly ever since.
So you should definitively be able to get it going.

What I did want to say is: I'm using the FS ONT SFP module, and since uptime is rather important for my french connection, I decided that the ONT SFP might be a bit of a "riskfactor" if it broke and would take weeks to find again. So I ordered another one as a backup, and while testing that, I had more or less the exact same symptoms you have. Initial connection is okay and onu ploamsg went to 5, but within a minute things would slow to a halt and nothing worked. After a LOT of troubleshooting it turned out it was the actual ONT SFP module that was defective, and after replacing it, I had no issues.
FS suggested I might have recieved a module where the CPU i the ONT did not have proper contact with the metal (for passive cooling) and it simply overheated.

Since you do get a DHCP address succesfully, and you can transfer traffic successfully for short while I'm pretty certain your problem is not configuration, but rather a hardware issues - loss of sync or overheating or xxx. Does your "onu ploamsg" state remain in O5 even when there is no traffic going? If it is loss of sync there might be a chance the module actually reports it by loosing O5 and returning to O3

sloopbun

I had to put this project to the side for a while and actually did not see your message until now.

The idea from FS is very logical. The ONU does get quite hot. I’d guess about 50° on the outside. Any way to monitor the temperature on the CPU of the module? An idea how hot yours gets?

onu ploamsg remains at O5 even when internet connectivity is completely stalled.

@keyser said in Orange Fibre: DHCP gives a public ip address, internet is fast, then after a few minutes it slows, and eventually unable to ping anything on the internet:

@sloopbun said in Orange Fibre: DHCP gives a public ip address, internet is fast, then after a few minutes it slows, and eventually unable to ping anything on the internet:

MA5671A

Hi Sloopbun

I'm the author of the posts that got you this far, and I'd just like to comment that my setup - since fixing the DHCP renew issue - has be running flawlessly ever since.
So you should definitively be able to get it going.

What I did want to say is: I'm using the FS ONT SFP module, and since uptime is rather important for my french connection, I decided that the ONT SFP might be a bit of a "riskfactor" if it broke and would take weeks to find again. So I ordered another one as a backup, and while testing that, I had more or less the exact same symptoms you have. Initial connection is okay and onu ploamsg went to 5, but within a minute things would slow to a halt and nothing worked. After a LOT of troubleshooting it turned out it was the actual ONT SFP module that was defective, and after replacing it, I had no issues.
FS suggested I might have recieved a module where the CPU i the ONT did not have proper contact with the metal (for passive cooling) and it simply overheated.

Since you do get a DHCP address succesfully, and you can transfer traffic successfully for short while I'm pretty certain your problem is not configuration, but rather a hardware issues - loss of sync or overheating or xxx. Does your "onu ploamsg" state remain in O5 even when there is no traffic going? If it is loss of sync there might be a chance the module actually reports it by loosing O5 and returning to O3

keyser

@sloopbun I haven’t found a way to have the ONU CPU report CPU or chassis temperature.
The FS ONU I use also gets unpleasantly hot to the touch - id say around 45Cish, but my original as well as the replaced backup ONU have been working flawlessly at this temperature for a couple of years now.

With the ONU becoming so hot to the touch, it would seem quite logical, that you too have a overheating issue because of a non-ideal contact between the CPU and the ONU chassis (plays the role of CPU cooler). The defective one I had also remained responsive on SSH when it stopped passing traffic, and it also remained in ONU 5

sloopbun

@keyser Thanks for help!

After some testing, I found that overheating was definitely a problem. I put a fan blowing over the module and with a media converter I can consistently maintain 500mbps down, which is progress.

However when I move the MA5671A sfp module to a BCM57810S pcie card, I only get 20-50mbps down in pfsense or opnsense…

ifconfig shows 2500mbps connection

Any idea on what might cause that?

keyser

@sloopbun Hmm, that does sound strange. I have no suggestions for how to troubleshoot that. It could be that the NIC PCIe card needs different drivers or some optimisations are needed for the current driver.
But it could also be that the SFP does not play nice with that NIC.