2.0-RELEASE: Performance oddity?
-
And I just tried setting the WAN MTU to 1500 and trying again, same results… What a mystery. So I reset it to 9000.
-
And another interesting factoid to consider…. Whenever I try to rsync that file outbound to the internet from inside the firewall, it seems the firewall state table shoots way up. I don't know why this would be since the rsync is a single socket connection, I can't imagine why it would generate 200+ states in the state table... Screenshot of the state graph attached. The activity on the far right of the graph was generated by me copying the file outbound.
-
Sorry, forgot to answer an earlier question… Yes, this is the amd64 version. And all the hosts and switches are dialed for jumbo frames. Tested and verified.
-
What is your provided speed (up and down) by your ISP?
http://cable-dsl.navasgroup.com/#Asymmetry
-
The outbound and inbound speed is 10Gb/s (full duplex synchronous) via CENIC in California.
-
Hmm, I don't think I have the NIC in promiscuous mode, at least I didn't do it on purpose. That's mostly used when doing packet tracing on the NIC right? I haven't been doing any of that.
Well, after consulting with a very knowledgeable friend of mine, it appears I do have the WAN interface in promiscuous mode, but it appears that it is that way because of pfsync/pflog because I'm using virtual floating IPs via CARP or something. Perhaps the pfSense devs know more on that. But I don't know if that is the root of the problem? Maybe… But I need CARP and virtual IPs so I hope it isn't the problem... ;)
-
Well, it is generally a bad idea to put that on WAN or LAN. It is better to have that on a dedicated NIC. As a test, disable pfsync. Promiscuous is not enabled by CARP VIPs only.
-
Hmm.. I do have a dedicated NIC for pfSync/CARP already? And it's not the WAN one. ;) I'm just using RFC1918 space for the dedicated CARP interface. I attached 2 screenshots showing my CARP config. It should be noted that I'm syncing everything available however.
But maybe I did something incorrectly there, let me know if so!
-
Is CARP a vLAN on on the WAN or LAN interface?
-
No, I'm not using VLANs in this model. The CARP interface uses a dedicated CAT5e cable connected in a crossover fashion from one physical interface of the "left" firewall to one physical interface on the "right" firewall, on a uniquely defined network that does not overlap any other nearby network.
-
Hmm.. I do have a dedicated NIC for pfSync/CARP already? And it's not the WAN one. ;) I'm just using RFC1918 space for the dedicated CARP interface. I attached 2 screenshots showing my CARP config. It should be noted that I'm syncing everything available however.
But maybe I did something incorrectly there, let me know if so!
CARP DOC says that it needs public ip to function
-
It needs 3 Internet IPs
1 for Physical Connection on Master
1 for Physical Connection on Backup
1 that is shared between the 2 (CARP Interface)It also needs 3 IPs per LAN interface for the same purposes.
It is highly recommended that you have dedicated NICs for pfsync and settings sync. This interface does not need internet route-able addresses. It is only to sync settings and and states.
-
My summary of interfaces are as follows:
firewall #1:
WAN - public IP 199.22.33.4/24
LAN - private IP 172.16.0.2/16
CARP - private IP 192.168.100.1/24 (connected directly to CARP interface on firewall #2, dedicated)firewall #2:
WAN - public IP 199.22.33.5/24
LAN - private IP 172.16.0.3/16
CARP - private IP 192.168.100.2/24 (connected directly to CARP interface on firewall #1, dedicated)Again, the CARP cable is a dedicated crossover cable at 1Gb/s ethernet. It is on a network that does not overlap with either the WAN or LAN networks. I am telling CARP/pfSync to use the dedicated CARP interface only.
One of the things CARP is doing is managing the virtual public IPs on the WAN interfaces. Such that if firewall #1 dies, firewall #2 would bring over the virtual IPs (on the WAN interface). Is that what is causing my WAN interfaces to be operating in promiscuous mode?
-
Another thing that is weird is that while I can see 2Gb/s on the live bandwidth graph, the RRD graphs don't show anything that high (maybe 20Mb/s or something). Is it possible the RRD graphs have upper limits and my traffic is above those limits, and therefore being ignored?
