Unable to get 1 Gb NAT throughput with new Jetway NUC build
-
Also the AT&T RG is not just a modem it is a gateway itself too. The RG also does NAT too and is able to achieve 936/912 directly connected to the RG.
This ISP sponsored boxes often are do their job in silicon (FPGA/ASIC) and pfSense is a pure software firewall
and pending on the double NAT or router cascade you have created it might be not going faster as you want
but more less fast owed to the circumstance of the double NAT situation and 2 * 5% you may loose now.
And so this ISP box might be faster with only doing one times NAT.So I don't agree with your logic that normal NAT will drop things down to 800-850 to be normal.
You might be not agreeing with me but spending 200 € is delevering you also only for 200 € speed!
I found and enabled PowerD using hiadaptive and that instantly made thing jump up from 780/780 to 900/920.
- minus double NAT
- minus overhead
- minus packetfilter (pf) firewall rules
One thing I was forgetting in my first post to count on top of all of this, the pfSense is at this time, surely
not for ever and they are working hard on it, but actual if will be only using one single CPU core at the WAN
interface if PPPoE will done there. This might be also narrow down the entire WAN speed, but as I see it
right your consumer router is not doing and passing any firewall rules, that are also slowing down the
entire WAN throughput you got what you payed for and with 900/920 the full maximum related to your
hardware is given here in this game. So I promise you, you will get lower speed if you turn on more
firewall rules or installing some packets.The weird thing after many tests is upload consistently gets 920, 20 mbits more than download. Also interesting thing is when I am directly connected to the RG the max upload i can get is 912 but via pfSense I can get 920.
Speed test done with iPerf or NetIO would sometimes different from each other, but if you only do a speed test
on some of the Internet based websites you might be also counting in the time that was needed to do so.I tried increasing mbuf to 131072 but that made performance inconsistent and erratic with tests. It just seemed overall much worse and I removed the setting from System Tunables and performance went back to 900/920.
4 COU cores * 2 LAN ports = 8 queues that is not to high and perhaps here in this case it is not really needed
to high up the mbuf size to an equal value.Is there a way to enable TRIM for the SSD via the GUI or does that have to be done via the command line?
You should boot from an usb pen drive into single user mode and then activating TRIM right there
and reboot from the mSATA or SSD. Here is a thread that explanes the entire rest.Also any other ideas for how to increase throughput to get the full 936 mbits for the downstream connection?
Please trust me you will never see this number really, but more pending on the single core cpu usage
then more pointed to other circumstances. if you will have a C2758, D-15x8 or Xeon E3 based pfSense box
and using a static IP at the WAN interface you will be seeing this number with ease, but not with the
hardware ypu are using. Your consumer router is a pure router and pfSense will be a firewall that can
be turned into a full featured UTM device and if you compare this prices on the global market you will
be also knowing that ~200 € is not very much for a 1 GBit/s Internet connection. -
"Also any other ideas for how to increase throughput to get the full 936 mbits for the downstream connection?"
Sure: eliminate the packet loss
Assuming that you're seeing 1538 byte packets onto the wire (1500 + 7 + 1 + 6 + 6 + 2 + 4 + 12)
These are 12304 bits long. (x 8)1,000,000,000/12304 = 81273 packets/second
936,000,000/12304 = 76072 packets /second
920,000,000/12304 74771 packets/secondSo you've got something like 1.7% packet loss along the route, or in the end application (the AT&T Speedtest application), or in the ability of your 82538V NICs, or the driver thereof to actually deal with those packet rates.
The 82538Vs don't support RSS or any hw queues.
The i210/i211/i35x (As used on the C2758, RCC-VE and RCC-DFF), do.BlueKobold assumes (above) that "4 COU cores * 2 LAN ports = 8 queues" but your NICs have one.
(The math is really that you want the queue count to match the core count. it doesn't matter how many NICs you have.) -
it doesn't matter how many NICs you have.)
Happy new year to all!
Yes you are right, at the beginning here in the forum I was read something about that each CPU core
would create one queue for each NIC on the board or inside of pfSense, if this is not so, it is my fault!
Sorry then, about this behavior. -
Well thanks for everyone's help. Its real disappointing to see that this hardware can't push 1 Gb NAT after all and i'll be returning the equipment.
My original intent was to buy something very small, compact and lower power draw but can definitely push 1 Gb NAT. Before I purchased my Jetway I saw the pfSense SG-2220 but that model did not state it would be able to do 1 Gb NAT like other models. Also based upon previous forum posts I found it sounded questionable that it would be able to push 1 Gb NAT based upon people's real world experience.
Does anyone have any recommendations for hardware that would fit that bill?
-
well, Nephi (born of goodly parents?)
As I said, you'll need to eliminate the packet loss.
-
@jwt:
well, Nephi (born of goodly parents?)
As I said, you'll need to eliminate the packet loss.
Yes, I understand which is why I am asking about things from a hardware perspective. You guys know more about that than me. Previously you said.
@jwt:
The 82538Vs don't support RSS or any hw queues.
The i210/i211/i35x (As used on the C2758, RCC-VE and RCC-DFF), do.BlueKobold assumes (above) that "4 COU cores * 2 LAN ports = 8 queues" but your NICs have one.
(The math is really that you want the queue count to match the core count. it doesn't matter how many NICs you have.)I tried out my Jetway build with Sophos UTM and was able to get full 1 Gb NAT performance. So apparently Sophos has better drivers or optimizations to take advantage of the Jetway hardware. However, I like pfSense more from what I have seen so far. That is why I am asking more questions about what pfSense hardware like SG-2220 can handle.
I only have a moderate knowledge of networking experience unlike you guys who are experts. I know the basics of how NAT works and have done plenty of wireshark captures to troubleshoot issues at work. Years ago when I was in college I did tier 2 VPN support for example. So I know enough to get around. But I am completely new to pfSense and especially the ins and outs of network hardware that is anything above regular consumer hardware.
Before all this research I had never known about NIC RSS or AES-NI. But now you guys are helping me out learning and I appreciate that.
So going back to the SG-2220 I noticed today when I looked at the pfSense store product page it now says under the "Best For" section heading "Anyone with High-Speed Gigabit Connections". I am pretty sure it didn't say that a couple weeks ago when I was first researching hardware for a pfSense firewall. I also learned about the Intel Atom Rangeley series which from I have briefly read today is a server series class of Intel Atom processors.
So based upon at least with Sophos on my Jetway I could get full gigabit NAT when testing with speedtest.net and att.com/speedtest, do you think I would I be able to get full gigabit NAT with the SG-2220?
If so, would I be severely performance constrained with the Intel Atom C2338 to add packages later when I want to become more adventurous with pfSense?
Are there other hardware acceleration benefits other than RSS and AES-NI that I would get with the SG-2220?
Thanks in advance!
-
Well thanks for everyone's help.
Happy new year!
Its real disappointing to see that this hardware can't push 1 Gb NAT after all and i'll be returning the equipment.
900/920 MBit/s + overhead + firewall rules + NAT is for me nearly 1 GBit/s, and please don´t forget
it is done with one CPU core only or alone! The N2930 is a 4 core CPU, if you get from your ISP a static public
IP address and don´t need PPPoE, the WAN part will be worked out by all 4 CPU cores and not by only one!
And for sure this will be not the problem from the vendor Jetway or pfSense.Also based upon previous forum posts I found it sounded questionable that it would be able to push 1 Gb
NAT based upon people's real world experience.Are they using PPPoE and will be also using only a single CPU core at the WAN part or did they own
their own static public IP address from their ISP? And what is a real world experience for you?
if I get 900/920 MBit/s with a ~200 € device likes you I would be glad to count on top of this
NAT + overhead + firewall rules and then I am at nearly to above 1 GBit/s. So no problems are
really there as I see it right. If you get 100% of 1 GBit/s throughput, where is the time to perform, NAT,
passing the firewall rules and on top counting the overhead? This is not done in 0.0 seconds by using a
lower end CPU based appliance!!! If you are using an Intel Atom C2000 SoC, Xeon D-15x8 or Xeon E3-1200
based appliance I am on your side and with you, but spending 200 bucks and then starting a thread why not
all is given to you, but offered by pfSense might be another thing only you should think about.You can not buy a small car that is saving fuel and think then why the hell this is not fast as a Porsche Cayenne!
Please have a look at this device here Jetway N2930 it comes with 4 x Intel 211AT LAN ports and is pushing something
around ~950/970 MBit/s, but only pending on the LAN ports and more RAM to high up the mbuf size??? Could this bring
up something more WAN speed?Why not saving money and go with a SG-4860 unit that is capable to deliver this speed?
Together with an pre-tuned ADI Image you would be on the save side as I see it right. -
I know there is some over head and when I say I want to get full 1 gigabit NAT, which by the way I got on the same Jetway with Sophos UTM, I mean getting the full 936/936 directly from the AT&T RG.
if I get 900/920 MBit/s with a ~200 € device likes you I would be glad to count on top of this
NAT + overhead + firewall rules and then I am at nearly to above 1 GBit/sThis is not done in 0.0 seconds by using a
lower end CPU based appliance!!!I never insinuated that I would expect it to take zero time to do NAT processing. However, if my AT&T RG and a Jetway Sophos build can do it, surely it isn't unreasonable to think it isn't possible to do with the same Jetway hardware but with pfSense.
I know it isn't a Porsche, I am not asking to push 1 Gb via VPN.
Also I think you need to calm down some, at the time when I was doing my research it did not seem completely unreasonable for me think the Jetway could do 1 Gb NAT since according to CPU benchmarks the CPU in the Jetway was over 2x more powerful then the Intel Atom CPU in the SG-2220. I saw comments in similar forum posts basically saying "Oh yeah, Intel Celeron and Intel NIC will definitely get you 1 Gb NAT." At the time I didn't know about the hardware accelerated features in the SG-2220.
I just didn't know any better and now I do. So please cut me some slack. I am barely learning about pfSense.
So back to my questions again…
So based upon at least with Sophos on my Jetway I could get full gigabit NAT when testing with speedtest.net and att.com/speedtest, do you think I would I be able to get full gigabit NAT with the SG-2220?
If so, would I be severely performance constrained with the Intel Atom C2338 to add packages later when I want to become more adventurous with pfSense?
Are there other hardware acceleration benefits other than RSS and AES-NI that I would get with the SG-2220?
I am not using PPPoE and I have a dynamic IP from my ISP. But it is basically static since it never changes.
-
You have narrowed down your bottleneck to pfSense. If the difference between 936/936 and 900/920 is a dealbreaker than use Sophos. If your nitpicking over 3% difference you should consider yourself lucky to have such minor problems…
-
@Phishfry:
You have narrowed down your bottleneck to pfSense. If the difference between 936/936 and 900/920 is a dealbreaker than use Sophos. If your nitpicking over 3% difference you should consider yourself lucky to have such minor problems…
No kidding. I'll trade you my 4/1.2 connection for your meager 900/920 any day. :o
But I'd still try to eek every last bit of performance out it myself too….......
-
@Phishfry:
You have narrowed down your bottleneck to pfSense. If the difference between 936/936 and 900/920 is a dealbreaker than use Sophos. If your nitpicking over 3% difference you should consider yourself lucky to have such minor problems…
Yes but that is with me adding no packages on at all. What I am setting up I want to last me for years to come with breathing room to grow for the future.
Just like Jailer said but i'll put in full size text
But I'd still try to eek every last bit of performance out it myself too….......
That is exactly what I am trying to do.
-
I am new but interested.
I am having a hard time deciding whether the OP is running the AT&T modem in bridge mode when he tests the pfSense build? It is easy to tell. If the pfSense build is getting an outside IP address. Otherwise he has double NAT working against him.
I just got TWC 300 megabit connection. I am trying figure out how to maximize my connection speed.
-
I am new but interested.
I am having a hard time deciding whether the OP is running the AT&T modem in bridge mode when he tests the pfSense build? It is easy to tell. If the pfSense build is getting an outside IP address. Otherwise he has double NAT working against him.
I just got TWC 300 megabit connection. I am trying figure out how to maximize my connection speed.
AT&T does not offer the ability to do a bridge mode, only the crappy IP Passthrough.