Slow routing speeds
-
The original APU, that we sold as the VK-T40E, has a G-T40E CPU and Realtek NICs. That is good for ~350Mbps between two ports, or was last time I tested it.
The G-T40N appears to differ from this only in the intergrated GPU so I would expect it to perform almost identically.
Is this using a Realtek NIC? Is it single queue? (check the boot logs)If so ~300Mbps is probably not much lower than expected with a router on a stick config.
Steve
-
@hngaminguk said in Slow routing speeds:
hopefully @stephenw10 will have some ideas.
Told you he would ;)
-
@stephenw10 said in Slow routing speeds:
The original APU, that we sold as the VK-T40E, has a G-T40E CPU and Realtek NICs. That is good for ~350Mbps between two ports, or was last time I tested it.
The G-T40N appears to differ from this only in the intergrated GPU so I would expect it to perform almost identically.
Is this using a Realtek NIC? Is it single queue? (check the boot logs)If so ~300Mbps is probably not much lower than expected with a router on a stick config.
Steve
Thanks for the information Steve, when I ordered this device (eBay) it claimed to have an Intel card (I never opened the unit to confirm). But I have just done so now and I can confirm it has a PCI card (NC7170).
Looking at the card, the processor is an Intel one and looking on google it also confirms it's an Intel based card. So not sure if there is a different limitation in play here?
In relation to the boot logs I looked in the "OS Boot" section of the logs but not sure if that's the location you are referring to? Secondly I don't know what exactly I need to look for if that is the correct log section.
-
What are the NICs assigned as? em0, em1, igb0?
-
@stephenw10 said in Slow routing speeds:
What are the NICs assigned as? em0, em1, igb0?
WAN is em0
LAN is em1Hope that helps.
-
It does yes. OK so the router-on-a-stick part here is that you're testing between two VLANs both on the LAN NIC?
em is also single queue though so, whilst a lot better than re, you will still be limited.
The later APU2 devices had quad core CPUs and igb NICs and with that and some heavy tweaking it was possible to get close to 1Gbps. Here you are using effectively quarter of that in any one direction. It is interesting looking at your top output that almost all the use is on one of the queues, probably the receive queue.That NIC looks like is actually PCI-X. I haven't seen one of those is a very long time! If you can use a igb based NIC there you probably would see more throughput as it can use both CPU cores for transmit and receive. Not sure there are any PCI/PCI-X igb NICs though.
Ultimately you're unlikely to see more tan 500Mbps with that CPU.
Steve
-
@stephenw10 said in Slow routing speeds:
you're testing between two VLANs both on the LAN NIC?
I think he was testing through wan and lan (router on a stick on the lan).. So if I understand what he was testing he did end up testing through the 2 different nics, or alteast different ports on the same nic. If that makes any difference?
So what your saying in a nutshell - is not some setting he turn off to see a boost, and if he wants to see full wirespeed he needs better hardware.
-
@stephenw10 said in Slow routing speeds:
It does yes. OK so the router-on-a-stick part here is that you're testing between two VLANs both on the LAN NIC?
em is also single queue though so, whilst a lot better than re, you will still be limited.
The later APU2 devices had quad core CPUs and igb NICs and with that and some heavy tweaking it was possible to get close to 1Gbps. Here you are using effectively quarter of that in any one direction. It is interesting looking at your top output that almost all the use is on one of the queues, probably the receive queue.That NIC looks like is actually PCI-X. I haven't seen one of those is a very long time! If you can use a igb based NIC there you probably would see more throughput as it can use both CPU cores for transmit and receive. Not sure there are any PCI/PCI-X igb NICs though.
Ultimately you're unlikely to see more tan 500Mbps with that CPU.
Steve
Originally I was testing throughput from the UDM/home system to the pf/lab system which was also only getting ~250 - 300Mbps.
So even with an igb NIC (if I can get one) I am unlikely to see over 500Mbps routing through the WAN side and out the LAN as my original network diagram shows?
If that is the case to get full 1Gbps (or more) routing what CPU spec would be the minimum? Also out of the current netgate devices which one would be the best option? I may in the future want to do 10Gbps but if it costs more than £500 I would rather go for a lower option and upgrade down the line when 10Gbps is cheaper.
-
Mmm, I would expect to see at least 350Mbps there as the APU could do that even with Realtek NICs.
IIRC there were some determined users who fitted mPCIe igb NICs to it and got closer to 500Mbps. You will never see Gigabit 'wire speed' through that though.I used to have a test box with a Core2Duo E4500 and that could just do gigabit with em NICs. If you look at a synthetic benchmark you can see why the G-T40 won't:
https://www.cpubenchmark.net/compare/AMD-G-T40E-vs-Intel-Core2-Duo-E4500-vs-Intel-Atom-C3558/264vs936vs3129
That doesn't give the full story but as a basic guide it's useful.The C3558 is what we have in the 5100/6100/7100/.
Steve
-
@stephenw10 said in Slow routing speeds:
Mmm, I would expect to see at least 350Mbps there as the APU could do that even with Realtek NICs.
IIRC there were some determined users who fitted mPCIe igb NICs to it and got closer to 500Mbps. You will never see Gigabit 'wire speed' through that though.I used to have a test box with a Core2Duo E4500 and that could just do gigabit with em NICs. If you look at a synthetic benchmark you can see why the G-T40 won't:
https://www.cpubenchmark.net/compare/AMD-G-T40E-vs-Intel-Core2-Duo-E4500-vs-Intel-Atom-C3558/264vs936vs3129
That doesn't give the full story but as a basic guide it's useful.The C3558 is what we have in the 5100/6100/7100/.
Steve
Okay great thanks for the confirmation and the quick replies!
Looking at the CPU Mark scores alone I am not surprised it is only able to do ~300Mbps.
I will look into upgrading my router at some point then.
I will mark this as solved! Thanks @johnpoz & @stephenw10
-
Marking this as solved, basically the processor in use is not enough processing power to handle full gigabit routing. My solution options are either a) Upgrade the network card to an igb NIC and complete some tweaking to maybe get 500Mbps or b) Upgrade the router as a whole to a better CPU.
Myself I am likely going for option 2, just need to decide if I am going for a netgate device or custom built.
-
@hngaminguk I may be a bit biased, but I love my sg4860.. And when its time to retire it I will most certainly be sticking with netgate box.
If mine blew up today - I would prob go with the 6100 base.. It would allow for 10 in the future and I could leverage the 2.5ge ports today.. I am not a fan of switch ports on a router..
And like my 4860 its overkill for my current internet speeds.. But never no what tomorrow might bring, etc.
-
@johnpoz said in Slow routing speeds:
@hngaminguk I may be a bit biased, but I love my sg4860.. And when its time to retire it I will most certainly be sticking with netgate box.
If mine blew up today - I would prob go with the 6100 base.. It would allow for 10 in the future and I could leverage the 2.5ge ports today.. I am not a fan of switch ports on a router..
And like my 4860 its overkill for my current internet speeds.. But never no what tomorrow might bring, etc.
Thanks for your input
With a max budget of £500 ($650 approx) I think the highest option I could go with is the 2100 since the 3100 is EOS.
Only annoyance being that the 2100 states a max of 881Mbps for Firewall (10k ACLs) I am not well versed into knowing how many ACLs I am using but I assume my setup currently has less? So I could likely hit 1Gbps?
-
@hngaminguk well to be honest, there was some threads recently about actual speeds not meeting those numbers via testing.. I don't recall exactly what model that was - might of been related to 10g, not sure..
-
@hngaminguk said in Slow routing speeds:
Only annoyance being that the 2100 states a max of 881Mbps for Firewall (10k ACLs) I am not well versed into knowing how many ACLs I am using but I assume my setup currently has less? So I could likely hit 1Gbps?
No, using fewer firewall rules will not allow it to hit 1Gbps between two subnets. Enabling pf to set any number of rules will introduce that overhead.
Steve
-
@stephenw10 said in Slow routing speeds:
@hngaminguk said in Slow routing speeds:
Only annoyance being that the 2100 states a max of 881Mbps for Firewall (10k ACLs) I am not well versed into knowing how many ACLs I am using but I assume my setup currently has less? So I could likely hit 1Gbps?
No, using fewer firewall rules will not allow it to hit 1Gbps between two subnets. Enabling pf to set any number of rules will introduce that overhead.
Steve
Okay thanks for the confirmation, in that case I will have to go for a 3rd party option such as https://www.ebay.co.uk/itm/Intel-Atom-E3845-4-LAN-3G-4G-4G-RAM-64G-SSD-Fanless-pfSense-Firewall-AES-NI-/114644549859?mkcid=16&mkevt=1&_trksid=p2349624.m46890.l49286&mkrid=710-127635-2958-0
