Can PFsense handle 10/10 Gbe Internet?

[remi_imer]

@Dobby_ , hi there,

Thanks for your input.

Do you reckon if I put PFsense on a modern PC could have an impact? Perhaps I could also do this, I have another PC running Ryzen 5 6 core with 16GB ram. I was meant to configure this as Proxmox Server but I have not time to do it due to busy time at work.

Going to your MTU comment. My pfsense has it blank at present, which I believe defaults to 1500. Do I have to change this value if running 10 Gbit?

As for the PPPoE, I think I am not using this one. I don't remember selecting PPPoE when I installed Pfsense.

WIll also try other speed test servers and see what result I get. But i think Ookla is by far the most reliable one.

[remi_imer]

@RobbieTT

Thanks a bunch for your input and also for sharing the article. Perhaps that was true at the time of publication, however, things have significantly changed at present.

Speedtest.net definitely made some improvements to its server, such that it can give you close to 25 gigabit download and upload.

Check out this guy's article on how he achieved this. Link below. PS: he is using a Linux Router which he wrote himself called router 7

Router Build Blog

Test Results

This guy is using the same provider as I am, but he opted for the fastest tier available, which is 25 Gigabit. I am eligible for 25 Gigabit also, but I wanted to test the 10 Gigabit first since the upgrade path was for free

So yeah I am definitely convinced that there are some hardware limitations of some sort.

[remi_imer]

@stephenw10 hi there, I will test with just my PC and see how it goes. Will keep everyone posted.

[stephenw10]

@remi_imer said in Can PFsense handle 10/10 Gbe Internet?:

Does it mean if it's rated at 3.6 Ghz it should be reaching that during testing?

Yes. We have seen systems that didn't enable turbo mode or default to the lowest CPU speed for example.
The CPU speed is usually shown on the dashboard but you can also check:
sysctl dev.cpu.0

But check the per-core usage with top, make sure no core is at 100%

Steve

[NollipfSense]

Dang, I am moving to where ever this place is...
login-to-view

[RobbieTT]

@NollipfSense said in Can PFsense handle 10/10 Gbe Internet?:

Dang, I am moving to where ever this place is...
login-to-view

Well at least in the UK I can glance over and check the season with Stonehenge. None of that Swiss Watch precision needed; that tech will never catch on.

Anyway, I am sporting a new imported abacus. You won't believe how thin and light it is.

[Dobby_]

@remi_imer said in Can PFsense handle 10/10 Gbe Internet?:

Do you reckon if I put PFsense on a modern PC could have an impact?

For sure but if I am in your situation I would try out that
with the actual given hardware once more.

Perhaps I could also do this, I have another PC running Ryzen 5 6 core with 16GB ram. I was meant to configure this as Proxmox Server but I have not time to do it due to busy time at work.

Could be nice or also not!

Going to your MTU comment. My pfsense has it blank at present, which I believe defaults to 1500. Do I have to change this value if running 10 Gbit?

If the MTU is not the same on all device in that row,
you may be seeing other numbers as a result then!

As for the PPPoE, I think I am not using this one. I don't remember selecting PPPoE when I installed Pfsense.

If so you will be nailed to one CPU core "only"!
And that means also only one queue for the entire
wan traffic, would good to be knowing about.

Will also try other speed test servers and see what result I get. But i think Ookla is by far the most reliable one.

Ok, but perhaps you may be trying it out at other
or different time frames!

[remi_imer]

@Dobby_

Am curious about this one. How do I ensure I move away from this?

If so you will be nailed to one CPU core "only"!
And that means also only one queue for the entire
wan traffic, would good to be knowing about.

[stephenw10]

I'd be amazed if you're using PPPoE with 10G.

But it's caused by the fact that PPPoE is not IP and cannot take advantage of hardware/driver hashing to divide traffic across queues:
https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#pppoe-with-multi-queue-nics

Very unlikely to apply here but you should check the per-core usage to be sure.

Steve

[remi_imer]

@stephenw10 hi there,

I did exactly as you said and this is what I found during testing. I dont know what am i suppose to be looking at here

last pid: 52273;  load averages:  0.24,  0.23,  0.18                                                                up 3+12:58:31  01:27:23
204 threads:   7 running, 162 sleeping, 35 waiting
CPU 0:  0.0% user,  0.0% nice,  0.0% system, 58.8% interrupt, 41.2% idle
CPU 1:  0.0% user,  0.0% nice,  0.0% system, 50.0% interrupt, 50.0% idle
CPU 2:  0.0% user,  0.0% nice,  0.4% system, 12.2% interrupt, 87.4% idle
CPU 3:  0.0% user,  0.0% nice,  0.0% system,  6.1% interrupt, 93.9% idle
Mem: 21M Active, 47M Inact, 404M Wired, 82M Buf, 3363M Free
Swap: 3656M Total, 3656M Free

[remi_imer]

@stephenw10

I also did what you suggested here.

/root: sysctl dev.cpu.0
dev.cpu.0.temperature: 57.0C
dev.cpu.0.coretemp.throttle_log: 0
dev.cpu.0.coretemp.tjmax: 100.0C
dev.cpu.0.coretemp.resolution: 1
dev.cpu.0.coretemp.delta: 43
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc
dev.cpu.0.cx_usage_counters: 43323265 0
dev.cpu.0.cx_usage: 100.00% 0.00% last 277us
dev.cpu.0.cx_lowest: C1
dev.cpu.0.cx_supported: C1/1/1 C2/2/148
dev.cpu.0.freq_levels: 3201/84000 3200/84000 3000/76349 2900/73036 2700/66595 2500/60408 2300/53689 2200/50804 2000/45218 1800/39849 1700/37248 1500/31535 1300/26734 1100/22147 1000/19936 800/15661
dev.cpu.0.freq: 3201
dev.cpu.0.%parent: acpi0
dev.cpu.0.%pnpinfo: _HID=none _UID=0 _CID=none
dev.cpu.0.%location: handle=\_PR_.CPU0
dev.cpu.0.%driver: cpu
dev.cpu.0.%desc: ACPI CPU

[remi_imer]

Here's an update.

I connected my PC directly to the FTTH OTO Socket without pfsense router and perform the test.

Unfortunately the result is the same with or without pfsense router. Perhaps there is an issue with my 10 Gb NIC. Or there is an issue at the fiber cable somewhere.. or at my ISP. Will call them tomorrow to find out.

[stephenw10]

Ah, well that's mostly good then.

No CPU core is at 0% idle (100% used). The loading could be spread better.

The CPU is running at 3201MHz. The extra 1 there implies turbo is enabled.

So it should be capable of more if there is more to be had.

You might try a local test between two 10G interfaces to confirm that.

Steve

[remi_imer]

Latest Update.

Goal of closer to 10 Gigabit reached So to answer my own question. Yes PFsense can indeed handle 10 Gigabit internet.

Thank you all for your inputs. I highly appreciate it.

Next goal is the 25 Gigabit. I hope TNSR is polished and have GUI interface by then

[stephenw10]

Ooo, nice. I can only dream of such connectivity!

What sort of CPU loading do you see at that throughput?

[remi_imer]

@stephenw10 , hi there.

26-29% CPU utilization and 11% Memory Usage.

[stephenw10]

Wow, that's much lower than I would have expected. Maybe still spread unevenly?

[remi_imer]

@stephenw10

I am quite surprised myself. I was actually planning to swap the CPU to i7 4970 (that's the max the motherboard supports) since I saw one in FB market place for 40 CHF but it seems I don't need to do that at all which is good.

But I am not running any other stuff in my PFsense Box. Just plain routing.

[stephenw10]

With pf enabled though? It's also filtering?

[remi_imer]

@stephenw10

Yes, i performed the test using Pfsense - Not direct.