Slow speed between 2 pfSense routers
-
Hello,
I have 2 pfSense firewalls, each is configured to it's own wan, and there is a cable directly connecting the two ("glink" because its the gigabit link between routers)Using autoselect, it chooses 1000baseT Full-duplex, but when I try to send data across the glink (either by browsing from the opposite gateway, or sending data across networks), it goes very very slow. always < 10Mbps. The link speed is identified as 1000baseT which is weird. I watched via traffic graph and rrd data
I also tried manually specifing the link speed on both ends, with the same results.
Any ideas?
-
Example:
10.0.0.10 is on 1ST pfSense box 300Mbps wan
10.20.0.10 is on 2ND pfSense box (100Mbps wan)
10.40.0.1 - 10.40.0.2 = link between them
1ST - 2ND
Speedtests from 10.0.0.10 network show high results, hundreds of mbps, speed tests on 10.20.0.10 also show high 80s - 90s.If I set up a computer on 10.20.0.50 to use the gateway 10.40.0.1 and its going so slow, a crawl, 480 Kbps.
/e oops sry for not modifying
-
New poll, maybe that will help get some interest.
Seriously, could it be a problem connecting 2 systems directly without a switch? Or could it be a problem with something slowing it down since it's transferring?
Firewall rules for the LINK is simple, 1 rule:allow * ** * * * on both pfSense boxes
-
What routing protocol are you using to connect the two routers? Are you doing route summarization? Lets take a look at your routing table.
-
How do I change the routing protocols?
The rows in bold are ones that are sent across the direct link
-
If you want to run RIPv1 or v2 then you want to install a package called routed, if you want to run OSPF then you want to install a package called quagga OSPF. For what you want to do I would probably start with rip. I don't remember how PfSense uses RIP but in the Cisco world you want to apply RIP to all the interfaces that you want advertised on your network. I would use Version 2 as Version 1 doesn't support subnets.
Also I see routes for 10.10.192.0/24 and 10.11.128.0/20 but I don't see anything about 10.0.0.0 and 10.20.0.0 so I'm assuming you made some changes on your router? Lastly I would say make sure your rules allow for traffic from any network on that interface.
-
yes the link between them is now
172.16.32.1 - 172.16.32.210.10 - 10.11 are on one box, 172.16.0.0/20 is on another.
the routing is all accurate. firewall rules on both sides of the link (172.16.32.0/30) are allow * * * *
what routing protocol does pf sense use by default?
I could have sworn that it worked as expected when I originally set it up, maybe I should try rebooting the routers?
what benefit would changing the routing protocol give?
-
By Default Pfsense doesn't use a routing protocol so you would have to setup a static routes on both routers anytime you add networks or interface on either router. When you add a dynamic routing protocol you can make a change on one router and the other router would know about it and traffic will flow. Dynamic Routing protocols make your life alot easier but if not careful they can be a source of a real pain in your butt too.
-
Okay, well I don't think I need a routing protocol, I have static routes for everything.
Connectivity is great, but the speed is the problem.
For example:
Network 1: 172.16.0.0/20
Between me and pfSense 1, there are 3 hops (3rd = pfsense), so between me and pfsense2, there are 4 hops. Using a firewall rule on pfsense1, I control which gateway my ip address goes out (either the wan gateway, or pfsense2).When I got out the normal gateway, internet speed works as expected, it's a slower connection, but I get speeds of around 25Mb - 40Mbps (limited by limiter, normal). When I switch to the pfsense2 gateway, the speed drops drastically, to 4Mb - 6Mb download, and 8 - 18Mbps upload (yes, higher upload usually)
This is using the same speed test provider.
Network 2: 10.10.192.0/24
pfsense is 4 hops away from serverA. A speed test run on serverA shows the fullspeed of the connection, 200- 300 Mbps. This is routed to the wan gateway directly from pfsense2.I have not tried from serverA -> pfsense2 -> pfsense1 -> 100Mb internet, but i don't think its really necessary at this time.
The kicker:
I logged into both pfSense boxes, and scp'd a 150Mb file directly between the two boxes, which it did via the same 172.16.32.1-2 link, this time I got 350 - 400Mbps transfer, between the two routers. So its clearly not the hardware. And I don't see say CPU spikes or anything that would be slowing down traffic, so what could this be?The image is a view of the topology, along with expected link speeds.
Lastly, imagine a serverB connected to the top network, plugged into the first gigabit switch (top left Blue "Switch" box). If serverA tries to send data to this serverB, it also is affected by terribly slow speeds. It seems to affect traffic that crosses both routers
Any ideas?