How to get Bell Fibe in Quebec/Ontario (Internet and IPTV) working with pfSense

[Chase84]

@sylvain613 How so? I have been using a Asus BRT828AC (VLAN capable) for years and decided to go with something I would have a bit more control over (pfsense). I was doing some custom scripting on the 828 for greater control but it was getting to be a pain and everytime I rebooted it I had to have a script that I set up run on it to configure it all again via ssh. I thought about installing opnsense and trying it but after installing pfsense Friday, an ubiquiti AP and spending nearly 40 hours fighting with Internet, IPTV, VLANs, etc, I am not excited at the thought of giving myself another challenge quite yet.

[clearrtc]

Just wanted to say thanks to the OP.

I have the "New" Bell Fixed Wireless Internet over LTE LOS. 25Mbps and always having issues sharing with Family and lagging while gamming etc. so wanted pfSense to figure out what users/apps where the offenders to more equitably distribute the internet around the house.

I found that no VLAN gave me a 192.x.x.x address oddly enough. I knew of this thread and looked it up. Configured VLAN35 with PPPoE and it did not connect. After some troubleshooting I found just VLAN35 and DHCP solved my issue over this LTE service. I have no IPTV service, but I would not likely have found the VLAN 35 solution without your help many years later.

Thanks

ClearRTC

[Patian]

@sylvain613
Hi, I am in Quebec and using Bell fibre 1G plan with max 940M Downloead and 750M Upload.
Originally, I had an ONT to Bell hub then to home network connection.
I do not use Bell fibre TV service, so my setup should be simple.
Currently, I have ONT to pfsense with VLAN35 and PPPoE setup for internet.
The internet is working fine, but I am not getting a full internet speed, tested at the pfsense LAN port, ie only 1 device on the network, with around 750M UP and 570M Down.

I would like to know if you really get a full speed at the LAN port. I posted the query on the forum and the reply was such that it was the way FreeBS implement the PPPoE causing the issue.

Any suggestion?

[techanalyst]

@patian I had to switch to a linux based firewall, in this case testing untangle but with the CC Provider changed to BBR, that alone solved all my issues. No matter the kind of egress limiter.shaper whatever, it didnt matter, the second I went over 300Mbit for example with things outside of Quebec or Ontario my speeds would die to 20-30, bbr seemed to solve the issue completely

[stephenw10]

@patian
If you are seeing a partial reduction like 750Mbps down from 940Mbps it probably is the PPPoE limitation you're hitting. What hardware are you running on there?
Have you tried the suggested sysctl tweak?
https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#pppoe-with-multi-queue-nics

Steve

[techanalyst]

@patian Getting every drop out of that line.....make sure the device you test with have all the power saving crap disabled on the nic, use speedtest cli to run your performance tests (they have it for every os). Also check your egress and make sure you have it set, for years I usually use 925-930Mbit, when i lived in Guelph this was enough to give me perfect uploads.

[stephenw10]

When testing throughput be sure to do so from a client behind the firewall. speedtest-cli can be run on pfSense but it uses significant CPU cycles to do so. You will always see a lower result on the firewall itself.

Steve

[techanalyst]

@stephenw10 This is true, its like installing iperf on the pfsense box, you get an idea but connect a device as the next hop and you can adjust for far better performance vs taking the pfsense boxes word for it

[Patian]

@techanalyst
Hi ,
Thank you for all the inputs. I have reported the issue with bell, they will send me new equipment to replace. I will keep everyone inform on the outcome in a few days after I receive the new equipment. It is because i am getting slow speed (better say inconsistent speed) on the Bell modem as well.

I have a Netgate SG-5100. I think using the netgate device will not have hardware compatibility issue with pfsense.

My test setup is very simple. There is only one laptop ethernet connected to the LAN1 port of Bell modem or igb 1 LAN port of SG-5100 and nothing else. Running Firefox browser with speedtest.net as a testing medium. I just swap the Bell modem with SG-5100 for direct comparison on the speedtest.

ISP only guarantee speed at their modem LAN port.

Should I stop the pfbockerNG and Suricata for the speed test on SG-5100?

All firmware and OS are up to date.

[Patian]

@techanalyst
I received the new bell modem today and used the build-in speedtest within the modem portal to do the test. At the modem interface, I constantly got 927-932M Download and 700M upload.

With iMac alone connected to the bell modem, the speed test was 932M Download and 654-700M upload.

With iMac connected to SG-5100, the speed test was 757-790M Download and 613-668M Upload.
The PPPoE seems to be the issue on the speed. There is no solution on it.

Should I keep the bell modem in the connection, use Double NAT/DMZ, SG-5100 in DHCP?

Any recommendation?

[josh256]

@patian If you don't need voice and your network supports VLANs I would create a dedicated interface on PfSense [and VLAN on the LAN] to contain the internal multicast (unless you have Cisco and/or Meraki who's IGMP snooping actually works in which case a single internal interface would suffice).

Edit: I am recommending you ditch the HH!

So, in total four interfaces:

Interfaces-->Assignments:
WAN_PPPoE [tagged VLAN 35, PPPoE]
WAN_IPTV [tagged VLAN 36, DHCP]
LAN [192.168.X/24]
LAN_IPTV. [192.168.2/24]

System-->RoutingStatic-->Routes
10.0.0.0/8 to WAN_IPTV_DHCP

Firewall-->Rules-->LAN_IPTV
permit any any etc... expand Advanced Options: Enable IP Options

Firewall-->Rules-->WAN_IPTV
permit any any etc... expand Advanced Options: Enable IP Options

Services-->IGMP Proxy:
WAN_IPTV upstream 224.0.0.0/4,10.0.0.0/8,192.168.1.0/24,192.168.2.0/24
LAN_IPTV downstream 192.168.2.0/24

Services-->DNS Forwarder: Custom Options
rebind-domain-ok=bell.ca
rebind-domain-ok=bell.com
server=/bell.ca/10.2.127.228
server=/bell.ca/10.2.127.196
server=/bell.com/10.2.127.228
server=/bell.com/10.2.127.196

[Patian]

Hi Josh256,
Thank you for your suggestion.

I already have wan_PPP0e(tagged VLAN 35, PPPoE) setup. It is working fine.

I do not use Bell Fibre TV, I watch most of the program on streaming. I do not have setup for IPTV. The phone line is branched out from the ONT.

My network setup is simple, Mostly internet traffic, streaming, using VLAN for security camera, IOT and guest network.

The issue I have is, I am not able to get the full internet bandwidth from the netgate device using PPPoE.

I subscribe to a 1GB plan (Max 940down, 750up), the best is 780down and 690 with netgate device.

If I use the Bell modem , i can get a full internet speed using the modem build-in speed test portal.

I am a bit disappointed with the money invested into the netgate device. Apart from that, everything seems to be working well.

I wonder if I should use double NAT and put back the Bell modem into the WAN interface and using DMZ on Bell modem for Netgate.

[josh256]

@patian

I'm on 1G Fibe FTTH vs. the 1.5G and am 830+ down, 720+ up -- I'm running PfSense virtualized (ESXi) on an old Core i3 (Gigabyte mini-itx Z77-WIFI)...

[Patian]

@josh256

You also do not get the full internet bandwidth using pfsense with PPPoE.

[josh256]

@patian PPPoE overhead is mainly irrelevant (8 bytes overhead on a 1500byte frame) Also not really even a PfSense constraint so much as a PHY+MAC constraint (NIC and SFP) --> certain Broadcom adaptors support >1Gbps by way of a firmware reflash and you should be able to get PfSense running 1.4Gbps+ per TX/RX directions with a the Bell ONT/GPON in one of those NICs..

[Patian]

@josh256
Hmm, Before the pfsense, I was getting close to 900down and 700up. I do not really mean to get full internet bandwidth from the Bell advertised speed.

I have a Netgate SG-5100 with the latest 21.02.2 version of pfsense installed. Currently, I get 750down and 620up. It is a significant decrease in speed.

With reference from another post, I was told it was due to how FreeBSD implement PPPoE. I guess most pfsense user's WAN setup is auto IP.

If someone can get a close to max internet speed from fibre Bell, I am very interested to know it is done.

I wonder if I let Bell modem does the PPPoE, with DMZ setting, auto DHCP to Netgate device, double NAT, auto IP on pfsense will get better speed.

[stephenw10]

Did you add the values recommended here yet?
https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#pppoe-with-multi-queue-nics

Steve

[Patian]

@stephenw10
Thank you for your advise.
I am not aware of such setting. I will study it and give it a try.

[techanalyst]

@patian The link that stephenw10 gave has most of the things I followed when I used pfsense but I switched to untangle, mostly because its linux, and I use BBR, the speed differences for folks was significant, any speed test across the contry on bell I get almost my max speeds or better. login-to-view

[stephenw10]

Nice, I can only dream of a connection like that.

Yeah, if you have a PPPoE connection that fast you're unlikely to fill it with anything FreeBSD based while that single thread restriction exists. You would need something very fast for single thread use. Up to 1Gbps is possible on reasonable hardware though.

Steve