Very slow upload on pfSense in KVM

gusto

I'm sorry, but my English is very poor

I have a new connection GPON 250/25 Mbps
Now I use an old RB450 router, which has a maximum throughput of 100 Mbit (real 80 Mbit)
I reach the current topology 80/27 Mbps.
https://www.speedtest.net/result/10098285484.png
Old topology network

Now I want to change the router/firewall to pfsense, but I want to use Proxmox VE KVM inside
I just installed proxmox 6.2-4 on RAID1 ZFS, Odroid H2 (RAM 32 GB), which has 2x NICs

Then I installed pfSense in KVM according to these instructions
https://docs.netgate.com/pfsense/en/latest/virtualization/virtualizing-pfsense-with-proxmox.html
Topology with pfSense

Everything seems to be working, but I see one problem. Upload is very slow.
https://www.speedtest.net/result/10097340434.png

However, when I tested the speed on an old Dell D610 laptop via the old Dlink DIR300 wifi router, the upload was fine.
Of course, the download is on wifi b/g slowly
(see screenshot)
https://www.speedtest.net/result/10097393325.png

System info
System activity
Proxmox KVM
pfSense KVMt
HW virtual

I have no idea where the problem may be
Can anyone help me please?

DaddyGo

@gusto said in Very slow upload on pfSense in KVM:

Dell D610 laptop via the old Dlink DIR300 wifi router, the upload was fine.

Hi,

Something happens on DESKTOP, because Proxmox delivers the right speed, as evidenced by your Dell measurement.

Dell has a wired NIC option(?) I would test on the switch to see what speed it shows

+++edit:
what is the type of NIC in DESKTOP?

gusto

Hi

04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 01)

DaddyGo

@gusto

This is not good news (Realtek), but you can also see that Dell has a good up speed, ergo somewhere the "desktop" is problematic...
The Realtek driver should be the latest...
and anyway, even if Realtek..... can’t reduce the speed to 10%

(otherwise it is better that Realtek is at the endpoint and not in a proxmox "iron")

Have you tested on another switch port?
test with Dell directly on the switch, if you have a wired (RJ45) option

see what happens

btw:
don't forget to hide the public IP in the sys log - "dpinger" 95.103.etc.....

gusto

BTW Mainboard Odroid H2 have 2 x GbE LAN ports Realtek RTL8111G.
However, someone writes here that pfsense (opnsense) works in KVM on Odroid H2
https://forum.odroid.com/viewtopic.php?p=267860#p267860

At night I will try to connect the desktop directly to the LAN port without a switch

DaddyGo

@gusto said in Very slow upload on pfSense in KVM:

Mainboard Odroid H2

There's nothing wrong with your pfSense, it's not a good thing to use Realtek stuff, but it's not impossible ...
The suggestion is always to avoid Realtek..

This is definitely some other network problem that can be narrowed down with the suggested tests.

Remember, if you had a problem with pfSense, Dell wouldn't know 20 (Mbps) up either

+++edit:
read this, is useful in terms of orientation:
https://forum.netgate.com/topic/137015/odroid-h2-sbc/2?_=1600350418401

gusto

When I create VMs in Proxmox, all HW is virtualized and therefore also network cards. Pfsense then doesn't see the real realtek NIC, but paravirtualized VirtIO, or am I wrong?

DaddyGo

@gusto said in Very slow upload on pfSense in KVM:

Pfsense then doesn't see the real realtek NIC, but paravirtualized VirtIO, or am I wrong?

Yes you are wrong, because the end is still the physical hardware as the traffic there goes in and out
(the driver is indeed virtual, but it does not help maintain the physical parameters of the network correctly)

the fact that you are virtualizing only complicates things further, especially on a weak piece of hardware.

I don't really familiar with Proxmox,I reviewed it a long time ago, but I don't like it.....

I’m more of a Xen, VMWare fan, but it doesn’t matter because pfsense is well virtualizable

Another question is that, I would never do, because the baremetal - is - baremetal and this is an NGFW.

I never suffer from intermediate problems like NIC pass-through...

not to mention that there is no redundancy if the VM dies, everything dies

+++edit:
I think it doesn't matter that, Proxmox or pfSense feels like Realtek stuff they are like shi.....t / end is the same
Or am I wrong

gusto

Last year I tested vmware esxi 6.7 and I was not satisfied, so I stayed on proxmox. I am very satisfied with proxmox.
But let's get to the point
This morning I replaced the old RB450 again with Odroid H2 (proxmox (pfsense)) and tested the speed.
I found this post before and set it up that way

System –>Advanced -->Networking i tick Disable hardware checksum offload

Now everything works very well
https://www.speedtest.net/result/10102747532.png

I also read this long thread and people complained about the instability of realtek drivers.
Very often they had LAN / WAN crashes. A link to the realtek driver is also published in this thread. It seems to someone that it works well with this driver.
But I did not use this driver. I have no reason yet when the upload and download is OK.
Now I need to see if it will be stable. It must be very stable otherwise it will not make sense to operate.

DaddyGo

@gusto said in Very slow upload on pfSense in KVM:

Now I need to see if it will be stable.

On a router, LRO, TSO and hardware checksum offload must always be disabled.
These features are good for endpoint devices but not for a router.

There are millions of posts on this forum about this theme:
https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.html

but even better if you disable it in loader.conf.local, where the other unnecessary functions include EEE, flow control, etc. (so stay protected from FW upgrades)

And these about Realtek:

https://forum.netgate.com/topic/135850/official-realtek-driver-binary-1-95-for-2-4-4-release/76

(A lot of people use this driver and if I know well it's really just that good and / or better solution.)

https://forum.netgate.com/topic/133536/official-realtek-driver-v1-95-binary?_=1600417473785

It's not easy to get it to work well.
You can almost forget about using Suricata and Snort with Realtek.

For me, ESXi and Xen (for web server / VPS) remain eternal love